Develop Reference

C# Azure function add record but check first if it already exist cosmosdb using IMongoCollection

I have the following Azure function using C# that creates/adds a student to cosmosdb and it works just fine
[FunctionName("CreateStudent")]
public static async Task<IActionResult> CreateStudent(
[HttpTrigger(AuthorizationLevel.Function, "post", Route = "student/create")] HttpRequest req,
[CosmosDB(
databaseName: "school",
collectionName: "students",
ConnectionStringSetting = "CosmosAccountEnpoint")] IAsyncCollector<Student> studentCollector,
ILogger log)
{
log.LogInformation("Creating a new student.");
string requestBody = await new StreamReader(req.Body).ReadToEndAsync();
var student = JsonConvert.DeserializeObject<Student>(requestBody);
await studentCollector.AddAsync(student);
return new OkObjectResult(student);
}
Now the 'issue' with this approach is that if the student doesn't exist it creates it which is the expected behavior, but if the student already exists it updates the record with the latest data I'm submitting.
Did some searching and IAsyncCollector only supports the method AddAsync which it doesn't have any type of validations. I found that using IMongoCollection should work because it has methods which will let me put some validations in place.
I changed the code to the following
[FunctionName("CreateStudent")]
public static async Task<IActionResult> CreateStudent(
[HttpTrigger(AuthorizationLevel.Function, "post", Route = "student/create")] HttpRequest req,
[CosmosDB(
databaseName: "school",
collectionName: "students",
ConnectionStringSetting = "CosmosAccountEnpoint")]
IMongoCollection<Student> studentCollection,
ILogger log)
{
log.LogInformation("Creating a new student");
string requestBody = await new StreamReader(req.Body).ReadToEndAsync();
var student = JsonConvert.DeserializeObject<Student>(requestBody);
if (student == null)
{
return new BadRequestObjectResult("Please pass a valid student in the request body");
}
var count = await studentCollection.CountDocumentsAsync(s => s.id == student.id);
if (count > 0)
{
return new BadRequestObjectResult($"Student with ID {student.id} already exists");
}
var studentexist = student.id;
await studentCollection.InsertOneAsync(student);
return new OkObjectResult(student);
}
After this code change i don't get any errors and everything seems to be ok, the problem is that when i run it i get the following error
Function 'CreateStudent' failed indexing and will be disabled.
[2023-02-13T15:14:55.796Z] The 'CreateStudent' function is in error: Microsoft.Azure.WebJobs.Host: Error indexing method 'CreateStudent'. Microsoft.Azure.WebJobs.Host: Can't bind CosmosDB to type 'MongoDB.Driver.IMongoCollection`1[School.Student]'.
what am i doing wrong?

Azure Function and SharePoint webhook: not getting changes from SharePoint list

I'm using a couple of Azure Functions with SharePoint webhook.
The first function is the one used to save messages from SharePoint webhook to a queue (Azure storage queue). This is the function content:
[FunctionName("QueueFunction")]
public static async Task<HttpResponseMessage> Run([HttpTrigger(AuthorizationLevel.Anonymous, "post", Route = null)]HttpRequestMessage req, TraceWriter log)
{
log.Info($"Webhook was triggered!");
// Grab the validationToken URL parameter
string validationToken = req.GetQueryNameValuePairs()
.FirstOrDefault(q => string.Compare(q.Key, "validationtoken", true) == 0)
.Value;
// If a validation token is present, we need to respond within 5 seconds by
// returning the given validation token. This only happens when a new
// web hook is being added
if (validationToken != null)
{
log.Info($"Validation token {validationToken} received");
var response = req.CreateResponse(HttpStatusCode.OK);
response.Content = new StringContent(validationToken);
return response;
}
log.Info($"SharePoint triggered our webhook...great :-)");
var content = await req.Content.ReadAsStringAsync();
log.Info($"Received following payload: {content}");
var notifications = JsonConvert.DeserializeObject<ResponseModel<NotificationModel>>(content).Value;
log.Info($"Found {notifications.Count} notifications");
if (notifications.Count > 0)
{
// get the cloud storage account
string queueName = "MYQUEUE";
CloudStorageAccount storageAccount = CloudStorageAccount.Parse(Environment.GetEnvironmentVariable("AzureWebJobsStorage"));
CloudQueueClient queueClient = storageAccount.CreateCloudQueueClient();
CloudQueue queue = queueClient.GetQueueReference(queueName);
await queue.CreateIfNotExistsAsync();
// store each notification as a queue item
foreach (var notification in notifications)
{
string message = JsonConvert.SerializeObject(notification);
log.Info($"Adding to {queueName}: {message}");
await queue.AddMessageAsync(new CloudQueueMessage(message));
log.Info($"added.");
}
// if we get here we assume the request was well received
return new HttpResponseMessage(HttpStatusCode.OK);
}
The message in queue is correctly added.
Then I've another function triggered by queue. This is the code of the function:
[FunctionName("OCRFunction")]
public static void Run([QueueTrigger("MYQUEUE", Connection = "QueueConn")]string myQueueItem, TraceWriter log)
{
log.Info($"C# Queue trigger function processed: {myQueueItem}");
string siteUrl = "https://MYSHAREPOINT.sharepoint.com/sites/MYSITE";
log.Info($"Processing notifications...");
string json = myQueueItem;
var data = (JObject)JsonConvert.DeserializeObject(json);
string notificationResource = data["resource"].Value<string>();
ClientContext SPClientContext = LoginSharePoint(siteUrl);
log.Info($"Logged in SharePoint");
GetChanges(SPClientContext, notificationResource, log);
}
public static ClientContext LoginSharePoint(string BaseUrl)
{
// Login using UserOnly Credentials (User Name and User PW)
ClientContext cntReturn;
string myUserName = config["spUN"];
string myPassword = config["spPWD"];
SecureString securePassword = new SecureString();
foreach (char oneChar in myPassword) securePassword.AppendChar(oneChar);
SharePointOnlineCredentials myCredentials = new SharePointOnlineCredentials(myUserName, securePassword);
cntReturn = new ClientContext(BaseUrl);
cntReturn.Credentials = myCredentials;
return cntReturn;
}
static void GetChanges(ClientContext SPClientContext, string ListId, TraceWriter log)
{
Web spWeb = SPClientContext.Web;
List myList = spWeb.Lists.GetByTitle("MY LIST");
SPClientContext.Load(myList);
SPClientContext.ExecuteQuery();
ChangeQuery myChangeQuery = GetChangeQueryNew(ListId);
var allChanges = myList.GetChanges(myChangeQuery);
SPClientContext.Load(allChanges);
SPClientContext.ExecuteQuery();
log.Info($"---- Changes found : " + allChanges.Count());
foreach (Change oneChange in allChanges)
{
if (oneChange is ChangeItem)
{
int myItemId = (oneChange as ChangeItem).ItemId;
log.Info($"---- Changed ItemId : " + myItemId);
ListItem myItem = myList.GetItemById(myItemId);
Microsoft.SharePoint.Client.File myFile = myItem.File;
ClientResult<System.IO.Stream> myFileStream = myFile.OpenBinaryStream();
SPClientContext.Load(myFile);
SPClientContext.ExecuteQuery();
byte[] myFileBytes = ConvertStreamToByteArray(myFileStream);
[...] SOME CODE HERE [...]
myItem["OCRText"] = myText;
myItem.Update();
SPClientContext.ExecuteQuery();
log.Info($"---- Text Analyze OCR added to SharePoint Item");
}
}
}
public static ChangeQuery GetChangeQueryNew(string ListId)
{
ChangeToken lastChangeToken = new ChangeToken();
lastChangeToken.StringValue = string.Format("1;3;{0};{1};-1", ListId, DateTime.Now.AddMinutes(-1).ToUniversalTime().Ticks.ToString());
ChangeToken newChangeToken = new ChangeToken();
newChangeToken.StringValue = string.Format("1;3;{0};{1};-1", ListId, DateTime.Now.ToUniversalTime().Ticks.ToString());
ChangeQuery myChangeQuery = new ChangeQuery(false, false);
myChangeQuery.Item = true; // Get only Item changes
myChangeQuery.Add = true; // Get only the new Items
myChangeQuery.ChangeTokenStart = lastChangeToken;
myChangeQuery.ChangeTokenEnd = newChangeToken;
return myChangeQuery;
}
public static Byte[] ConvertStreamToByteArray(ClientResult<System.IO.Stream> myFileStream)
{
Byte[] bytReturn = null;
using (System.IO.MemoryStream myFileMemoryStream = new System.IO.MemoryStream())
{
if (myFileStream != null)
{
myFileStream.Value.CopyTo(myFileMemoryStream);
bytReturn = myFileMemoryStream.ToArray();
}
}
return bytReturn;
}
public static async Task<TextAnalyzeOCRResult> GetAzureTextAnalyzeOCR(byte[] myFileBytes)
{
TextAnalyzeOCRResult resultReturn = new TextAnalyzeOCRResult();
HttpClient client = new HttpClient();
client.DefaultRequestHeaders.Add("Ocp-Apim-Subscription-Key", "XXXXXXXXXXXXXXXXXXXX");
string requestParameters = "language=unk&detectOrientation=true";
/* OCR API */
string uri = "https://MYOCRSERVICE.cognitiveservices.azure.com/vision/v3.0/ocr" + "?" + requestParameters;
string contentString = string.Empty;
HttpResponseMessage response;
using (ByteArrayContent content = new ByteArrayContent(myFileBytes))
{
content.Headers.ContentType = new MediaTypeHeaderValue("application/octet-stream");
response = await client.PostAsync(uri, content);
contentString = await response.Content.ReadAsStringAsync();
resultReturn = JsonConvert.DeserializeObject<TextAnalyzeOCRResult>(contentString);
return resultReturn;
}
}
Before current approach with two functions, I was using a single function where I managed the notifications and I executed some code to update a field in my SharePoint list. This method was having some problem when I was receiving many notifications from SharePoint so I decided to use queue as suggested in Microsoft documentation. This solution was working fine with a single notification received and my SharePoint list item were updated without problem.
To avoid problems with multiple notification, I decided to split functions, one registering notifications in a queue and the other executing some operations and updating a SharePoint field.
The first one function QueueFunction is working fine, the second one is triggering correctly but it is not getting changes from SharePoint list even if I just add one item.
I've tried to check GetChanges code to find why it is always returning no changes, but the code is the same of the one I used when I had only one function, so I can't understand why the behaviour is changed.
What's wrong with my approach? Is there something I could do to correct the second function?

According to the comments, just summarize the solution as below for other communities reference:
Use a function to save the message in a queue and then call an azure web job, the problem was caused by the the running time of the function may exceed 5 minutes.
By the way, the default timeout of azure function(with consumption plan) is 5 minutes, we can see all of the default timeout for different plan on this page (also shown as below screenshot).
If we want longer timeout, we can set the functionTimeout property in host.json of the function(but can not exceed the Maximum timeout). Or we can also use higher plan for the function app, such as Premium plan and App Service plan.

How to call one function app from another locally?

In Visual Studio, I have created 2 Azure function apps f1 and f2.
I have already change the port for both function apps.
I want to call f2 from f1 but I'm getting a NotFound error.
f1 is using http://localhost:999/demo1
f2 is using http://localhost:1212/demo2

I have tried calling one Function to Another Function within the same project and different project too both works fine.
Function 1 Example:
[FunctionName("Function1")]
public static async Task<IActionResult> Run(
[HttpTrigger(AuthorizationLevel.Anonymous, "get", "post", Route = null)] HttpRequest req,
ILogger log)
{
log.LogInformation("C# HTTP trigger function processed a request.");
try
{
//Extract Request Param
var content = await new StreamReader(req.Body).ReadToEndAsync();
QnAMakerQuestion objQnAMakerQuestion = JsonConvert.DeserializeObject<QnAMakerQuestion>(content);
//Global Variable for containing message
dynamic validationMessage;
// Validate param
if (string.IsNullOrEmpty(objQnAMakerQuestion.question))
{
validationMessage = new OkObjectResult("Question is required!");
return (IActionResult)validationMessage;
}
//Selialize Request Param
var json = JsonConvert.SerializeObject(objQnAMakerQuestion);
var stringContent = new StringContent(json, UnicodeEncoding.UTF8, "application/json");
// Call Function 2
HttpClient newClient = new HttpClient();
HttpResponseMessage responseFromAnotherFunction = await newClient.PostAsync("http://localhost:7073/api/Function2FromApp2", stringContent);
dynamic response = "";
if (responseFromAnotherFunction.IsSuccessStatusCode)
{
response = responseFromAnotherFunction.Content.ReadAsStringAsync().Result;
}
validationMessage = new OkObjectResult(response);
return (IActionResult)validationMessage;
}
catch (Exception ex)
{
dynamic validationMessage = new OkObjectResult(string.Format("Something went wrong, please try agian! Reason:{0}", ex.Message));
return (IActionResult)validationMessage;
}
}
Function 2 Example:
[FunctionName("Function2FromApp2")]
public static async Task<IActionResult> Run(
[HttpTrigger(AuthorizationLevel.Anonymous, "get", "post", Route = null)] HttpRequest req,
ILogger log)
{
log.LogInformation("C# HTTP trigger function processed a request.");
try
{
var content = await new StreamReader(req.Body).ReadToEndAsync();
QnAMakerQuestion objQnAMakerQuestion = JsonConvert.DeserializeObject<QnAMakerQuestion>(content);
//Global Variable for containing message
dynamic validationMessage;
// Validate param
if (string.IsNullOrEmpty(objQnAMakerQuestion.question))
{
validationMessage = new OkObjectResult("Question is required!");
return (IActionResult)validationMessage;
}
validationMessage = new OkObjectResult(objQnAMakerQuestion);
return (IActionResult)validationMessage;
}
catch (Exception ex)
{
dynamic validationMessage = new OkObjectResult(string.Format("Something went wrong, please try agian! Reason:{0}", ex.Message));
return (IActionResult)validationMessage;
}
}
Class Used:
public class QnAMakerQuestion
{
public string question { get; set; }
}
Note: If you run in same project then you wouldn't encounter any problem. But if you run in different project encounter a issue
regarding port. To resolve that in local.settings.json file replace
below code :
"Host": {
"LocalHttpPort": 7073
}
And Update Project Properties -> Debug to following
host start --port 7073 --pause-on-error See the screen shot below:
Post Man Test:
I have invoked Function 1 on PostMan it Invoked Function 1 as Function 1 Invoked Function 2 and Send Response or vice-versa from function 2 to function 1. See the screen shot below:
Just plug and play, let me know if you have any additional question.

Correct use of Azure Durable Function - Serializing Complex Objects

So I'm prototyping some Azure Durable Functions, to try and understand to see if they will fit within a proposed solution for our internal API system.
Based on examples, I've created a Orchestrator Client (HelloOrchestratorClient.cs), that responds to a HttpTrigger. This client extracts some information from the original request, then proceeds to fire off a Orchestrator Function (HelloOrchestrator.cs) passing in some of the information extracted:
Complex HelloOrchestratorClient.cs:
[FunctionName("HttpSyncStart")]
public static async Task<HttpResponseMessage> Run(
[HttpTrigger(AuthorizationLevel.Anonymous, methods: "get", Route = "orchestrators/{functionName}/wait")]
HttpRequestMessage req,
[OrchestrationClient] DurableOrchestrationClient starter,
string functionName,
ILogger log)
{
HttpReq originalRequest = new HttpReq() {
DeveloperId = GetDevKey(req,apiHeaderKey),
QueryString = req.RequestUri.Query,
APIName = GetQueryStringValue(req,APIName),
APIVersion = GetQueryStringValue(req,APIVersion)
};
string instanceId = await starter.StartNewAsync(functionName, originalRequest);
TimeSpan timeout = GetTimeSpan(req, Timeout) ?? TimeSpan.FromSeconds(30);
TimeSpan retryInterval = GetTimeSpan(req, RetryInterval) ?? TimeSpan.FromSeconds(1);
return await starter.WaitForCompletionOrCreateCheckStatusResponseAsync(
req,
instanceId,
timeout,
retryInterval);
}
The HelloOrchestrator.cs simply for now is just calling off to one of our internal API's and returning a JsonProduct payload (Simple POCO describing, you guessed it, a title), using a ActivityTigger named HelloOrchestrator.APICall to make the API call itself.
Complex HelloOrchestrator.cs:
[FunctionName("E1_JsonProduct")]
public static async Task<List<JsonProduct>> Run(
[OrchestrationTrigger] DurableOrchestrationContextBase context,
ILogger log)
{
List<JsonProduct> output = new List<JsonProduct>();
HttpReq r = context.GetInput<HttpReq>();
if(r != null)
{
if(r.DeveloperId == null)
{
return output;
}
output.Add(await context.CallActivityAsync<JsonProduct>("E1_CallAPI",r));
return output;
}
return output;
}
[FunctionName("E1_CallAPI")]
public async static Task<JsonProduct> APICall([ActivityTrigger] HttpReq req,
ILogger log)
{
JsonProduct products = null;
string u = $"{baseAddress}{req.APIVersion}/{req.APIName}{req.QueryString}";
var request = new HttpRequestMessage(HttpMethod.Get, u);
request.Headers.Accept.Add(
new MediaTypeWithQualityHeaderValue("application/json")
);
request.Headers.Add("x-apikey",req.DeveloperId);
log.LogInformation($"URL calling = '{request.RequestUri.AbsoluteUri}'.");
HttpResponseMessage response = await client.SendAsync(request);
// return await response.Content.ReadAsStringAsync();
if(response.IsSuccessStatusCode)
{
var formatter = new JsonMediaTypeFormatter
{
SerializerSettings = HelloProj.CosmosDB.Models.Products.Converter.Settings
};
products = await response.Content.ReadAsAsync<JsonProduct>(new [] {formatter});
}
return products;
}
Side Note: The plan is if I can get this to work, is to fan out a bunch of processes to different API's and fan back in again and merge the JSON payload and return it back to the originator.
Issue I'm experiencing
So, when my List<JsonProduct> is returned back from HelloOrchestrator.Run, I receive the following NullReferenceException found on this Gist (Big stack trace) and I receive a 500 response from the Orchestrator Client.
The following proves the output returned does actually have an object at runtime:
Could it be due to the complexity of JsonProduct (Again find the model classes here)? I ask, because when I swap out my Orchestrator Function for a simpler model structure, I don't receive a 500, I receive my JSON Payload.
This example shows the Simple Orchestrator Function HelloOrchestrator.cs, returning a simple TestToDo.cs (Gist for model) flat object that doesn't error:
Simple HelloOrchestrator.cs:
[FunctionName("E1_Todo")]
public static async Task<TestToDo> RunToDo(
[OrchestrationTrigger] DurableOrchestrationContextBase context,
ILogger log)
{
HttpReq r = context.GetInput<HttpReq>();
TestToDo todo = new TestToDo();
if(r != null)
{
todo = await context.CallActivityAsync<TestToDo>("E1_CallAPITodo",r);
}
return todo;
}
[FunctionName("E1_CallAPITodo")]
public async static Task<TestToDo> APITodoCall([ActivityTrigger] HttpReq req,
ILogger log)
{
var request = new HttpRequestMessage(HttpMethod.Get, "https://jsonplaceholder.typicode.com/todos/1");
request.Headers.Accept.Add(
new MediaTypeWithQualityHeaderValue("application/json")
);
log.LogInformation($"URL calling = '{request.RequestUri.AbsoluteUri}'. for {req.QueryString}");
HttpResponseMessage response = await client.SendAsync(request);
return await response.Content.ReadAsAsync<TestToDo>();
}
More Information
If you require my full prototype projects, you can find them here:
Complex Project (Throws 500 and exception)
When you run it, use the following in something like Postman (After F5ing it):
http://localhost:7071/api/orchestrators/E1_JsonProduct/wait?timeout=20&retryInterval=0.25&api=products&apiVersion=v1&filterByImprints=W%26N&N
Simple Project (No 500 or Exception thrown)
When you run it, use the following in something like Postman (after F5ing it):
http://localhost:7071/api/orchestrators/E1_Todo/wait?timeout=20&retryInterval=0.25

Looking at the callstack you posted, the NullReferenceException appears to be a bug in the DurableOrchestrationClient class. Looking at the code (which you can find here) is seems possible that if the query string you're using cannot be parsed correctly, a null-ref is possible.
You mentioned you're using the following URL for testing:
http://localhost:7071/api/orchestrators/E1_JsonProduct/wait?timeout=20&retryInterval=0.25&api=products&apiVersion=v1&filterByImprints=W%26N&N
I wonder if the last two characters (&N) are the source of the problem. Is is possible to encode the & or remove it entirely to isolate the problem?
Either way, it would be great if you could log an issue here: https://github.com/Azure/azure-functions-durable-extension/issues

Get Http response headers and content Angular 6

I have the following web server method, that returns data to our front-end applicaiton.
[FunctionName("SearchCustomerBySearchTerm")]
public static async Task<HttpResponseMessage> SearchCustomerBySearchTerm([HttpTrigger(AuthorizationLevel.Function, WebRequestMethods.Http.Get, Route = "Customer/SearchCustomerBySearchTerm/{searchTerm}/pageSize/{pageSize}")]HttpRequestMessage req, TraceWriter log, string searchTerm, int pageSize)
{
try
{
var continuationToken = req.Headers.TryGetValues("continuationToken", out IEnumerable<string> values) ? values.FirstOrDefault() : null;
PagedResponse<CustomerSearchResult> pagedResponse = await _customerComponent.FindCustomerBy(searchTerm, continuationToken, pageSize);
if (pagedResponse == null) return req.CreateResponse(HttpStatusCode.NoContent, $"Could not find any data related to {searchTerm}");
HttpResponseMessage responseMessage = req.CreateResponse(HttpStatusCode.OK, pagedResponse.Results);
responseMessage.Content.Headers.Add("continuationToken", pagedResponse.Continuation);
responseMessage.Content.Headers.Add("Access-Control-Expose-Headers", "*");
return responseMessage;
}
catch (Exception ex)
{
log.Error(ex.Message);
return req.CreateResponse(HttpStatusCode.InternalServerError, "Something went wrong. Could not search for customers");
}
}
I am allowing all headers to be exposed, by adding the Access-Control-Expose-Headers.
From my Angular application, I am doing the request as follow:
searchCustomersPaged(searchTerm: string, continuationToken: string): Observable<HttpResponse<CustomerSearchResult>> {
let customHeaders = new HttpHeaders().set("continuationToken", this.currentContinuationToken);
const url = "http://localhost:7071/api/Customer/SearchCustomerBySearchTerm/andrew/pageSize/10";
const parsedUrl = encodeURI(url);
return this.http.get<HttpResponse<CustomerSearchResult>>(parsedUrl, { headers: customHeaders });
}
As you can see above, I am expecting an HttpResponse<CustomerSearch> back.
Here is how I try and read my headers:
nextClikcedHandle(continuationToken: string): void {
this.customerService.searchCustomersPaged(this.customerService.searchTerm, this.currentContinuationToken)
.subscribe(resp => {
//add current continuation token, to previous now, as this will be used for 'previous' searching
this.previousContinuationTokens.push(this.currentContinuationToken);
//set next continuation token received by server
this.currentContinuationToken = resp.headers.get('continuationToken');
//return search results
this.customerService.searchResults.next(resp.body);
});
}
With the above code, the resp.headers and the resp.body is always undefined. Why is this happening?
If I look at the Network tab within Chrome, I can see my data is returned, as well as my header.
What am I doing wrong?

I found a useful article here:
By default the HttpClient returns the body of the response. You can
pass-in an object with an observe key set to a value of ‘response’ to
get the full response. This can be useful to inspect for certain
headers:
So I changed my code as follow, with the added observe key.
searchCustomersPaged(searchTerm: string, continuationToken: string): Observable<HttpResponse<CustomerSearchResult>> {
let customHeaders = new HttpHeaders().set("continuationToken", this.currentContinuationToken);
const url = "http://localhost:7071/api/Customer/SearchCustomerBySearchTerm/andrew/pageSize/10";
const parsedUrl = encodeURI(url);
return this.http.get<CustomerSearchResult>(parsedUrl, { headers: customHeaders, observe: 'response' });
}
After changing above method, I could query body and headers as per normal:
nextClikcedHandle(continuationToken: string): void {
this.customerService.searchCustomersPaged(this.customerService.searchTerm, this.currentContinuationToken)
.subscribe(resp => {
//add current continuation token, to previous now, as this will be used for 'previous' searching
this.previousContinuationTokens.push(this.currentContinuationToken);
//set next continuation token received by server
this.currentContinuationToken = resp.headers.get('continuationToken');
//return search results
this.customerService.searchResults.next(resp.body);
});