I'm trying to run a Lambda function from a console application. The idea is for it to run a quick fire & forget lambda function without waiting for lambda function to return. My code doesn't appear to be executing the lambda function at all though. I know the function works because I can run with the test. When I run the below code I just get a task cancelled exception.
var jsonSerializer = new JsonSerializer();
var lambdaConfig = new AmazonLambdaConfig() { RegionEndpoint = RegionEndpoint.USEast2 };
var lambdaClient = new AmazonLambdaClient(lambdaConfig);
using (var memoryStream = new MemoryStream())
{
jsonSerializer.Serialize(myData, memoryStream);
var lambdaRequest = new InvokeRequest
{
FunctionName = "MyFunction",
InvocationType = "Event",
PayloadStream = memoryStream
};
var result = Task.Run(async () => { return await lambdaClient.InvokeAsync(lambdaRequest); }).Result;
Does anyone have some insight into what I'm doing wrong?
Thanks!
You can pass your myData directly instead converting into MemoryStream, if the data is a valid JSON with double quotes.
In function name you can use the ARN or just the name. Both works fine for me in latest version AWSSDK.Lambda -Version 3.3.103.31
static readonly string awsAccessKey = "access key here";
static readonly string awsSecretKey = "secret key here";
private static BasicAWSCredentials awsCredentials = new BasicAWSCredentials(awsAccessKey, awsSecretKey);
private static AmazonLambdaConfig lambdaConfig = new AmazonLambdaConfig() { RegionEndpoint = RegionEndpoint.USEast1 };
private static AmazonLambdaClient lambdaClient = new AmazonLambdaClient(awsCredentials, lambdaConfig);
public async Task<string> GetLambdaResponse(string myData)
{
var lambdaRequest = new InvokeRequest
{
FunctionName = "mylambdafunction",
Payload = myData
};
var response = await lambdaClient.InvokeAsync(lambdaRequest);
if (response != null)
{
using (var sr = new StreamReader(response.Payload))
{
return await sr.ReadToEndAsync();
}
}
return string.Empty;
}
I believe there are two issues in your code:
(Mentioned by #Nikosi) You dispose your memoryStream before it is used for lambda invocation
(The fix that helped in my case) When you serialize you payload into memory stream, the position in the stream points to the byte after its end and thus SDK has nothing to read from it. So I just used memoryStream.Seek(0L, SeekOrigin.Begin)
By the way, assuming that JsonSerializer is the one from Newtonsoft.Json package, I didn't find Serialize method that accepts Stream parameter, only TextWriter or JsonWriter. So it might be necessary to wrap it into StreamWriter and make sure you call Flush or FlushAsync (or dispose StreamWriter before using memory stream as Lambda payload) like this:
await using var stream = new MemoryStream();
await using var streamWriter = new StreamWriter(stream);
var serializer = new JsonSerializer();
serializer.Serialize(streamWriter, payload);
await streamWriter.FlushAsync();
stream.Seek(0L, SeekOrigin.Begin);
log.LogInformation("Batch {0}: sending {1} messages to Lambda", batchId, batch.Count);
var lambdaResponse = await lambda.InvokeAsync(new InvokeRequest
{
InvocationType = InvocationType.RequestResponse,
PayloadStream = stream,
//Payload = JsonConvert.SerializeObject(payload),
FunctionName = lambdaArn
}, stoppingToken);
Mixing blocking calls could be causing a deadlock. If the intent is to fire and for get then just call the desired function. Also why give request a stream only to dispose of it after wards
public static void Main(string[] args) {
var jsonSerializer = new JsonSerializer();
var lambdaConfig = new AmazonLambdaConfig() { RegionEndpoint = RegionEndpoint.USEast2 };
var lambdaClient = new AmazonLambdaClient(lambdaConfig);
var memoryStream = new MemoryStream();
jsonSerializer.Serialize(myData, memoryStream);
var lambdaRequest = new InvokeRequest
{
FunctionName = "MyFunction",
InvocationType = "Event",
PayloadStream = memoryStream
};
lambdaClient.InvokeAsync(lambdaRequest);
Console.ReadLine();
}
The FunctionName in Nkosi's answer should actually be the entire ARN to your lambda function, so taking from Nkosi's answer:
public static void Main(string[] args) {
var jsonSerializer = new JsonSerializer();
var lambdaConfig = new AmazonLambdaConfig() { RegionEndpoint = RegionEndpoint.USEast2 };
var lambdaClient = new AmazonLambdaClient(lambdaConfig);
var memoryStream = new MemoryStream();
jsonSerializer.Serialize(myData, memoryStream);
var lambdaRequest = new InvokeRequest
{
FunctionName = "arn:aws:lambda:ap-southeast-2:{id}:function:MyFunction",
InvocationType = "Event",
PayloadStream = memoryStream
};
lambdaClient.InvokeAsync(lambdaRequest);
Console.ReadLine();
Related
I have an endpoint with functionname (( CallTrafficEndPoint )) this request was developed by RestSharp Methdology
And I call this endpoint from other place in code but the retrieved type was (RequestData)
so How can I retrive a bool value from this request?
namespace Business.TrafficIntegration
{
public static class Calling_API
{
public static RequestData CallTrafficEndPoint(RequestData TrafficModel, CrmDAL crmDal)
{
RequestData IsLicenseValid;
Utility.Utility utility = new Utility.Utility(crmDal);
string serviceURL = utility.GetAdminConfigurationValue("valid-licence");
var client = new RestClient(serviceURL);
var request = new RestRequest();
var body = new RequestData();
body = TrafficModel;
request.AddJsonBody(body);
using (MemoryStream DeSerializememoryStream = new MemoryStream())
{
//Json String that we get from web api
var response = client.Post(request);
//initialize DataContractJsonSerializer object and pass Student class type to it
DataContractJsonSerializer serializer = new DataContractJsonSerializer(typeof(RequestData));
//user stream writer to write JSON string data to memory stream
StreamWriter writer = new StreamWriter(DeSerializememoryStream);
writer.Write(response);
writer.Flush();
DeSerializememoryStream.Position = 0;
//get the Desrialized data
RequestData SerializedObject = (RequestData)serializer.ReadObject(DeSerializememoryStream);
IsLicenseValid = SerializedObject;
// throw new InvalidPluginExecutionException("The retrieved value " + IsLicenseValid);
return IsLicenseValid;
}
}
}
}
I have been trying to write a unit test for the method of service which gets multiple/form-data from the Request. Here is my method:
var boundary = MultipartRequestHelper.GetBoundary(
MediaTypeHeaderValue.Parse(HttpContextAccessor.HttpContext.Request.ContentType),
DEFAULT_MULTIPART_BOUNDARY_LENGTH_LIMIT);
var reader = new MultipartReader(boundary, HttpContextAccessor.HttpContext.Request.Body);
var section = await reader.ReadNextSectionAsync();
var streamedFileContent = Array.Empty<byte>();
var dictionary = new Dictionary<string, string>();
while (section != null)
{
var hasContentDispositionHeader =
ContentDispositionHeaderValue.TryParse(
section.ContentDisposition, out var contentDisposition);
if (hasContentDispositionHeader)
{
if (MultipartRequestHelper.HasFileContentDisposition(contentDisposition))
{
streamedFileContent = await FileHelpers.ProcessStreamedFile(
section, contentDisposition, Configuration.GetFileSizeLimit());
}
else if (MultipartRequestHelper.HasFormDataContentDisposition(contentDisposition))
{
var key = HeaderUtilities
.RemoveQuotes(contentDisposition.Name).Value;
var encoding = GetEncoding(section);
if (encoding == null)
throw new ErrorCodeException("error-invalid-data");
using (var streamReader = new StreamReader(
section.Body,
encoding,
detectEncodingFromByteOrderMarks: true,
bufferSize: 1024,
leaveOpen: true))
{
var value = await streamReader.ReadToEndAsync();
dictionary.Add(key, value);
}
}
}
section = await reader.ReadNextSectionAsync();
}
var file = new File
{
AuthorId = authorStaffId,
Name = dictionary.FirstOrDefault(l => l.Key == "name").Value,
Size = streamedFileContent.Length,
Extension = dictionary.FirstOrDefault(l => l.Key == "extension").Value,
MimeType = dictionary.FirstOrDefault(l => l.Key == "mimeType").Value,
IsTemporary = true,
FileId = Guid.NewGuid()
};
file = await SaveFileContentAsync(streamedFileContent, file);
file.ContentHash = CanculateContentHash(streamedFileContent);
await _fileRepository.AddAsync(file);
Here I am trying to send a request from the Postman and this way it is working fine:
And following way I am trying to test my code with xUnit and Moq:
[Fact]
public async Task UploadFileAsync_WhenFileSizeIsNotBiggerThanLimit_Successful()
{
// Arrange
ConfigureClaims();
ConfigureDatabase();
ConfigureService();
var fakeFileContent = GetFakeFileContent();
var byteContent = Convert.FromBase64String(fakeFileContent);
Stream stream = new MemoryStream(byteContent);
_moqHttpContextAccessor.Setup(x => x.HttpContext.Request.Body).Returns(stream);
_moqHttpContextAccessor.Setup(x => x.HttpContext.Request.ContentType).Returns("multipart/form-data; boundary=----231179732646258011288433");
_moqHttpContextAccessor.Setup(x => x.HttpContext.Request.ContentLength).Returns(0x000000000001d74e);
// Act
var addedFileId = _baseFileService.UploadFileAsync();
var addedFile = _fileRepository.GetById(addedFileId);
// Assert
Assert.NotNull(addedFile);
}
I got the following error message: InvalidDataException: Multipart body length limit 16384 exceeded
And I tried this way:
[Fact]
public async Task UploadFileAsync_WhenFileSizeIsNotBiggerThanLimit_Successful()
{
// Arrange
ConfigureClaims();
ConfigureDatabase();
ConfigureService();
var fakeFileContent = GetFakeFileContent();
var byteContent = Convert.FromBase64String(fakeFileContent);
var byteStringContent = byteContent.ToString();
var forDataDictionary = new Dictionary<string, StringValues>
{
{"file", byteStringContent},
{"name", "test.pdf"},
{"size", "120085"},
{"mimeType", "application/pdf" }
};
var formData = new FormCollection(forDataDictionary);
_moqHttpContextAccessor.Setup(x => x.HttpContext.Request.Form).Returns(formData);
// Act
var addedFileId = _baseFileService.UploadFileAsync();
var addedFile = _fileRepository.GetById(addedFileId);
// Assert
Assert.NotNull(addedFile);
}
I got an error because I am not getting the content from Request.Form. I am getting it from Reques.Body
Then I looked around the Web and could not find any proper answer. What can I do here?
you can take this approach to mock your IformFile.
public static IFormFile AsValidMockIFormFile(this FileInfo physicalFile)
{
var fileMock = new Mock<IFormFile>();
var ms = new MemoryStream();
var writer = new StreamWriter(ms);
writer.Write(physicalFile.OpenRead());
writer.Flush();
ms.Position = 0;
var fileName = physicalFile.Name;
//Setup mock file using info from physical file
fileMock.Setup(_ => _.FileName).Returns(fileName);
fileMock.Setup(_ => _.Length).Returns(ms.Length);
fileMock.Setup(m => m.OpenReadStream()).Returns(ms);
fileMock.Setup(m => m.ContentType).Returns("text/csv");
fileMock.Setup(m => m.ContentDisposition).Returns(string.Format("inline; filename={0}", fileName));
return fileMock.Object;
}
As this above method will give a IformFile object. but to decorate the same you need to pass some data(i.e., you need to hold a sample file in your test data)
var physicalFile = new FileInfo("filePath.txt"); // Put your sample file here
fakeFileContent = FileHelper.AsValidMockIFormFile(physicalFile);
var info=await _controller.UploadAtion(fakeFileContent) as OkObjectResult;
This is my controller code. Here it accepts UpdateMediaDto object that contains IFormFile data ( e.g. images and audios).
[HttpPut("words/{wordId}/medias/{id}")]
public async Task<ActionResult<WordMediaDto>> UpdateWordMedia(Guid wordId, Guid id, [FromForm] UpdateMediaDto mediaDto)
{
WordMedia? media = await _unitOfWork.WordMediaRepository.GetByIdAsync(id);
if (media == null) return NotFound();
if (wordId != media.WordId) return BadRequest("No Word present for this media");
var newMedia = _mediaFileMasterService.UpdateMedias(mediaDto);
//............
}
And this is my testcase
[Fact]
public async Task PUT_Media_with_UpdateMediaDto_results_WordMediaDto_success()
{
var ImagefileMock = new Mock<IFormFile>();
var content = System.Text.Encoding.UTF8.GetBytes("// byte[]....");
var ImagefileName = "sampleImage.jpg";
var ms = new MemoryStream();
var writer = new StreamWriter(ms);
writer.Write(content);
writer.Flush();
ms.Position = 0;
ImagefileMock.Setup(_ => _.OpenReadStream()).Returns(ms);
ImagefileMock.Setup(_ => _.FileName).Returns(ImagefileName);
ImagefileMock.Setup(_ => _.Length).Returns(ms.Length);
UpdateMediaDto mediaDto = new() {
Id = Guid.Parse("f1659b04-85a3-4969-7d20-08da081a9616"),
wordId=Guid.Parse("2caf24aa-4d37-4f64-aa91-4a605798c35b"),
PrimaryImage=ImagefileMock.Object,
SecondaryImage1 = ImagefileMock.Object };
HttpContent httpContent = new StringContent(JsonConvert.SerializeObject(mediaDto), Encoding.UTF8);
httpContent.Headers.ContentType = new MediaTypeHeaderValue("multipart/form-data");
//Act
var response = await _httpClient.PutAsync("api/v1/admin/words/d9823bdd-0d11-42e5-a804-4d59d393d2bc/medias/f47136e3-754a-4f2b-b5cc-08da2d91a92e", httpContent);
//Assert
response.EnsureSuccessStatusCode();
}
But Here I can't serialize the object using SerializeObject. when I execute this test case I get an error like
Newtonsoft.Json.JsonSerializationException : Self referencing loop detected for property 'Object' with type 'Castle.Proxies.IFormFileProxy'. Path 'PrimaryImage.Mock'.
How can I solve this problem... and is there any other ways to test API like this?
The proxy created by MOQ is causing an issue with the serialization in this case.
Consider using an actual FormFile instance which is derived from IFormFile.
[Fact]
public async Task PUT_Media_with_UpdateMediaDto_results_WordMediaDto_success() {
//Setup mock file using a memory stream
string content = System.Text.Encoding.UTF8.GetBytes("// byte[]....");
string ImagefileName = "sampleImage.jpg";
MemoryStream ms = new MemoryStream();
StreamWriter writer = new StreamWriter(ms);
writer.Write(content);
writer.Flush();
ms.Position = 0;
//create FormFile with desired data
IFormFile ImagefileMock = new FormFile(ms, 0, ms.Length, "name_from_form", ImagefileName);
UpdateMediaDto mediaDto = new() {
Id = Guid.Parse("f1659b04-85a3-4969-7d20-08da081a9616"),
wordId=Guid.Parse("2caf24aa-4d37-4f64-aa91-4a605798c35b"),
PrimaryImage=ImagefileMock,
SecondaryImage1 = ImagefileMock
};
HttpContent httpContent = new StringContent(JsonConvert.SerializeObject(mediaDto), Encoding.UTF8);
httpContent.Headers.ContentType = new MediaTypeHeaderValue("multipart/form-data");
//Act
var response = await _httpClient.PutAsync("api/v1/admin/words/d9823bdd-0d11-42e5-a804-4d59d393d2bc/medias/f47136e3-754a-4f2b-b5cc-08da2d91a92e", httpContent);
//Assert
response.EnsureSuccessStatusCode();
}
It seems you need to use MultipartFormDataContent to build your form and add parts. To Stream objects use StreamContent.
var multiPartformData = new MultipartFormDataContent();
var streamContent = new StreamContent(ImagefileMock);
multiPartformData .Add(streamContent);
I was trying to attach a csv file as a body parameter in my test script. But still as per the below code controller expect file and just curious how should I pass that.
I run test script in below order
Method-1
public void AttachedRatesFile(string fileName)
{
_form = string.IsNullOrWhiteSpace(fileName)
? _form = new StringContent(string.Empty)
: _form = new StreamContent(File.OpenRead($"{ResourceFolder}{fileName}"));
_form.Headers.ContentType = new MediaTypeHeaderValue("application/csv");
_form.Headers.ContentDisposition = new ContentDispositionHeaderValue(fileName);
}
Method-2
public void PostRequestExecutes(string resource)
{
var content = new MultipartFormDataContent{_form};
WhenThePostRequestExecutesWithContent(resource, content);
}
Method-3
public async void WhenThePostRequestExecutesWithContent(string resource, HttpContent content)
{
ResponseMessage = await HttpClient.PostAsync(resource, content);
}
I see null in below file parameter
Controller:
public async Task<IActionResult> SeedData(IFormFile file)
{
var result = await _seedDataService.SeedData(file);
return Ok(new { IsUploadSuccesful = result});
}
I would add that to the body as a stream
var memoryContentStream = new MemoryStream();
using (var streamWriter = new StreamWriter(memoryContentStream, Encoding.UTF8, 1000,
true))
{
using (var jsonTextWriter = new JsonTextWriter(streamWriter))
{
var jsonSerializer = new JsonSerializer();
jsonSerializer.Serialize(jsonTextWriter, OBJECT);
jsonTextWriter.Flush();
}
}
if (memoryContentStream.CanSeek)
{
memoryContentStream.Seek(0, SeekOrigin.Begin);
}
Then
using (var streamContent = new StreamContent(memoryContentStream))
{
streamContent.Headers.ContentType = new MediaTypeHeaderValue("application/json");
request.Content = streamContent;
using (var response = await _httpClient.SendAsync(request, HttpCompletionOption.ResponseHeadersRead))
{
var stream = await response.Content.ReadAsStreamAsync();
response.EnsureIsSuccessStatusCode();
}
}
The above would first write the content as a memory stream and then when creating the POST request you can send the stream as a streamContent
I'm getting a web response via an API which converts file (in this case, Powerpoint presentations to PDF).
I do get the response in a string, but when saving this string to a file or a stream (which ends up being saved in a file anyways) I always end up with a blank file, its size is always well over 0 bytes though.
Here's the class that calls the API:
public class CloudConvert
{
private string apiKey;
private static object ProcessResponse;
public CloudConvert(string apiKey)
{
this.apiKey = apiKey;
}
public async Task<string> Convert(string inputFormat, string outputFormat, string fileUrl)
{
var processUrl = await CreateProcess(inputFormat, outputFormat);
return await Upload(processUrl, fileUrl, outputFormat);
}
private async Task<string> CreateProcess(string inputFormat, string outputFormat)
{
var request = new
{
apikey = apiKey,
inputformat = inputFormat,
outputformat = outputFormat
};
var json = await PostJson("https://api.cloudconvert.com/process", request);
dynamic obj = JObject.Parse(json);
ProcessResponse = obj;
return "https:" + obj.url;
}
private static async Task<string> Upload(string processUrl,
string fileUrl, string outputFormat)
{
var request = new
{
input = "download",
file = fileUrl,
outputformat = outputFormat,
download = "false",
wait = "true",
save = "false"
};
return await PostJson(processUrl, request);
}
private static async Task<string> PostJson(string url, object data)
{
var parameters = JsonConvert.SerializeObject(data);
using (var wc = new WebClient())
{
wc.Headers[HttpRequestHeader.ContentType] = "application/json";
try
{
return await wc.UploadStringTaskAsync(url, "POST", parameters);
}
catch (WebException ex)
{
return string.Empty;
}
}
}
}
And how I'm invoking it:
Task<string> task = Task.Run(async () =>
{
return await cloudConvert.Convert("ppt", "pdf", "http://host.com/myfile.ppt");
});
//I have the PDF in the response, as a string.
var pdfContent = task.Result;
//I think it's here that I'm not handling the resulting string
//properly, ending in a blank file. I've tried writing it to a local file as well, same issue.
using (MemoryStream stream = new MemoryStream())
{
StreamWriter writer = new StreamWriter(stream);
writer.Write(pdfContent);
writer.Flush();
stream.Seek(0, SeekOrigin.Begin);
//This below is something that saves to an Azure storage container...
//filePath = fileStorage.SaveFile($"{fileGeneratedName}.pdf", stream);
//But this fails as well. Blank file
using (FileStream fs = new FileStream(#"c:\tmp\output.pdf", FileMode.OpenOrCreate))
{
stream.CopyTo(fs);
fs.Flush();
}
}
So regardless of how I'm trying to save the returned content, it seems my encoding is wrong. I noticed that the resulting blank file always has the same number of pages than the input Powerpoint presentation.
Any ideas?
Have your convert function return an HttpResponseMessage:
public async Task<HttpResponseMessage> convert()
do your conversion into, say, a memory stream, and then:
var result = new HttpResponseMessage(HttpStatusCode.OK)
{
Content = new ByteArrayContent(outStream.GetBuffer(),0,(int)outStream.Length)
};
result.Content.Headers.ContentDisposition =
new System.Net.Http.Headers.ContentDispositionHeaderValue("attachment")
{
FileName = fname
};
result.Content.Headers.ContentType = new MediaTypeHeaderValue("application/pdf");
return result;