Develop Reference

C# FlatBufferBuilder create String from Stream

Suppose you need to read a large string from a stream and you want to put that string into a flatbuffer.
Currently what I do is read the stream into a string and then use the FlatbufferBuilder.CreateString(string s) function.
This works fine but it does have as a drawback that the string is copied and loaded into memory twice: once by reading it from the stream into the string; and then a second time the string is copied into the flatbuffer.
I was wondering if there is a way to fill the flatbuffer string directly from a stream?
For a more concrete example:
Suppose your flatbuffer schema looks like:
table Message
{
_Data: string;
}
root_type Message;
We can then create a flatbuffer like this (with myData a string)
var fbb = new FlatBufferBuilder(myData.Length);
var dataOffset = fbb.CreateString(myData);
var message = Message.CreateMessage(fbb, dataOffset);
Message.FinishMessageBuffer(fbb, message);
So the question is can we somehow do the same thing, where myData is a System.IO.Stream?
Obviously the following works, but I'd like to avoid first reading the Stream into memory.
using (var reader = new StreamReader(myStream)
{
var myData = reader.ReadToEnd();
var fbb = new FlatBufferBuilder(myData.Length);
var dataOffset = fbb.CreateString(myData);
var message = Message.CreateMessage(fbb, dataOffset);
Message.FinishMessageBuffer(fbb, message);
}

There is currently no way to avoid that copy twice, afaik.. it should be relatively simple to implement a version of CreateString that takes a stream and reduces it to one copy. You could have a go at that and open a PR on github with the result.

AddAttachment from MemoryStream

The SendGrid API docs specify you can add attachments from a Stream. The example it gives uses a FileStream object.
I have some blobs in Azure Storage which I'd like to email as attachments. To achieve this I'm trying to use a MemoryStream:
var getBlob = blobContainer.GetBlobReferenceFromServer(fileUploadLink.Name);
if(getBlob != null)
{
// Get file as a stream
MemoryStream memoryStream = new MemoryStream();
getBlob.DownloadToStream(memoryStream);
emailMessage.AddAttachment(memoryStream, fileUploadLink.Name);
}
emailTransport.Deliver(emailMessage);
It sends fine but when the email arrives, the attachment appears to be there but it's actually empty. Looking at the email source, there is no content for the attachment.
Is using a MemoryStream a known limitation when using the SendGrid C# API to send attachments? Or should I be approaching this in some other way?

You probably just need to reset the stream position back to 0 after you call DownloadToStream:
var getBlob = blobContainer.GetBlobReferenceFromServer(fileUploadLink.Name);
if (getBlob != null)
{
var memoryStream = new MemoryStream();
getBlob.DownloadToStream(memoryStream);
memoryStream.Seek(0,SeekOrigin.Begin); // Reset stream back to beginning
emailMessage.AddAttachment(memoryStream, fileUploadLink.Name);
}
emailTransport.Deliver(emailMessage);
You might want to check who cleans up the stream as well and if they don't you should dispose of it after you've called Deliver().

According to their API, they have implemented void AddAttachment(Stream stream, String name).
You are probably using a MemoryStream which you have written to before. I suggest resetting the position inside the stream to the beginning, like:
memoryStream.Seek(0, SeekOrigin.Begin);

I ended up with the following which fixed the issue for me:
fileByteArray = new byte[getBlob.Properties.Length];
getBlob.DownloadToByteArray(fileByteArray, 0);
attachmentFileStream = new MemoryStream(fileByteArray);
emailMessage.AddAttachment(attachmentFileStream, fileUploadLink.Name);

The thread is a bit old, but I use a varient with NReco PDF converter:
private async Task SendGridasyncBid(string from, string to, string displayName, string subject, **byte[] PDFBody**, string TxtBody, string HtmlBody)
{
...
var myStream = new System.IO.MemoryStream(**PDFBody**);
myStream.Seek(0, SeekOrigin.Begin);
myMessage.AddAttachment(myStream, "NewBid.pdf");
...
}
convert the html to pdf and return it instead of writing it for download...
private byte[] getHTML(newBidViewModel model)
{
string strHtml = ...;
HtmlToPdfConverter pdfConverter = new HtmlToPdfConverter();
pdfConverter.CustomWkHtmlArgs = "--page-size Letter";
var pdfBytes = pdfConverter.GeneratePdf(strHtml);
return **pdfBytes**;
}
I am not sure how efficient this is, but it is working for me and I hope it helps someone else get their attachments figured out.

Unable to attach an eml file as an attachment when sending an email using c#?

I am receiving emails through hmailserver and sending those emails as .eml file as an attachment of another report email.
I am having issues in reading and sending those emails as an attachment.
This is what I am doing.
public void addAttachment(string pathname, bool retry)
{
string attachmentname = "";
do
{
try
{
attachmentname = Path.GetFileNameWithoutExtension(pathname);
Stream file = new MemoryStream(File.ReadAllBytes(pathname));
Log.WriteMessage("Size" + file.Length);
dtstreamAttach.Add(attachmentname+".eml", file);
retry = false;
}
catch (ArgumentException e)
{
string strCurrentTs = DateTime.Now.ToString(strDateFormat);
attachmentname = attachmentname + "-" + strCurrentTs+".eml";
}
} while (retry);
}
Then,
MailMessage message = new MailMessage();
.
.
message.Attachments.Add(new Attachment(kvp.Value, kvp.Key)); // I have an attachment dictionary
string contenttype = GetMimeType(".eml");
ContentType cnttype = new ContentType(contenttype);
message.Attachments[0].ContentType = cnttype;
as you see i print the Stream size - which prints out as something like 4790Bytes (4KB)
But when i receive the email, i only get an eml file with size 1KB and the eml file is empty.
I have checked the file paths and also made sure that the email is there until my report mail is sent out.
I have also verified content type is message/rfc822.
Everything seems to check out. Not sure what the issue is.

I was able to resolve it. Looks like the MemoryStream have the correct stream and also the ContentStream of the message object has the correct sizes, but the Stream postion had moved to the end of the stream and hence when the message is actually being sent, it actualy has nothing.
So make sure to reposition the stream back to origin before adding it to AttachmentCollection, something like
Seek(0, SeekOrigin.Begin)
especially when using streams wrapped in dlls, something might have moved them.

Retrieving a stored email attachment as a byte[] from Rabbit MQ doesn't send email

I'm convert an svg object to a byte[] and storing this in a queue with the following class
public class MailMessage
{
public string AttachmentName { get; set; }
public byte[] Attachment { get; set; }
}
However the email never gets sent. I'm thinking that the attachment is corrupt because if we skip adding the attachment to the email then it sends fine.
using (var stream = new MemoryStream(attachment))
{
var mailMessage = new MailMessage(this.from, new MailAddress(recipient)) { Subject = subject, Body = message };
// this is the line which if commented out allows the email to be sent
mailMessage.Attachments.Add(new Attachment(stream, filename));
MailSender().SendAsync(mailMessage, null);
}
A co-worker suggested that the byte[] may have been compromised due to the way rabbit stores messages and to therefore base64 encode the byte[] before storing it using the built in functions
Convert.ToBase64String(bytes)
Convert.FromBase64String(message.Attachment) // to retrieve
However that didn't work either.
Can anyone think why this is failing to send and think of any work-arounds.
I'm contemplating storing the image in the database and deleting it once the email is sent, but this is a last resource.

The issue is with MailSender.SendAsync(). The stream object is get disposed before the mail is sent.
Don't use "using" here. You are destroying the memory stream immediately
after calling SendAsync, e.g. probably before SMTP gets to read it
(since it's async).
Either Destroy your stream in the callback.
or
Use MailSender.Send() instead of MailSender.SendAsync() to send synchronusly.
Refer this for more detail

Get filename while downloading it

We are providing files that are saved in our database and the only way to retrieve them is by going by their id as in:
www.AwesomeURL.com/AwesomeSite.aspx?requestedFileId=23
Everything is working file as I am using the WebClient Class.
There's only one issue that I am facing:
How can I get the real filename?
My code looks like this atm:
WebClient client = new WebClient ();
string url = "www.AwesomeURL.com/AwesomeSite.aspx?requestedFileId=23";
client.DownloadFile(url, "IDontKnowHowToGetTheRealFileNameHere.txt");
All I know is the id.
This does not happen when I try accessing url from the browser where it get's the proper name => DownloadedFile.xls.
What's the proper way to get the correct response?

I had the same problem, and I found this class: System.Net.Mime.ContentDisposition.
using (WebClient client = new WebClient()){
client.OpenRead(url);
string header_contentDisposition = client.ResponseHeaders["content-disposition"];
string filename = new ContentDisposition(header_contentDisposition).FileName;
...do stuff...
}
The class documentation suggests it's intended for email attachments, but it works fine on the server I used to test, and it's really nice to avoid the parsing.

Here is the full code required, assuming the server has applied content-disposition header:
using (WebClient client = new WebClient())
{
using (Stream rawStream = client.OpenRead(url))
{
string fileName = string.Empty;
string contentDisposition = client.ResponseHeaders["content-disposition"];
if (!string.IsNullOrEmpty(contentDisposition))
{
string lookFor = "filename=";
int index = contentDisposition.IndexOf(lookFor, StringComparison.CurrentCultureIgnoreCase);
if (index >= 0)
fileName = contentDisposition.Substring(index + lookFor.Length);
}
if (fileName.Length > 0)
{
using (StreamReader reader = new StreamReader(rawStream))
{
File.WriteAllText(Server.MapPath(fileName), reader.ReadToEnd());
reader.Close();
}
}
rawStream.Close();
}
}
If the server did not set up this header, try debugging and see what ResponseHeaders you do have, one of them will probably contain the name you desire. If the browser show the name, it must come from somewhere.. :)

You need to look at the content-disposition header, via:
string disposition = client.ResponseHeaders["content-disposition"];
a typical example would be:
"attachment; filename=IDontKnowHowToGetTheRealFileNameHere.txt"

I achieve this with the code of wst.
Here is the full code to download the url file in c:\temp folder
public static void DownloadFile(string url)
{
using (WebClient client = new WebClient())
{
client.OpenRead(url);
string header_contentDisposition = client.ResponseHeaders["content-disposition"];
string filename = new ContentDisposition(header_contentDisposition).FileName;
//Start the download and copy the file to the destinationFolder
client.DownloadFile(new Uri(url), #"c:\temp\" + filename);
}
}

You can use HTTP content-disposition header to suggest filenames for the content you are providing:
Content-Disposition: attachment; filename=downloadedfile.xls;
So, in your AwesomeSite.aspx script, you would set the content-disposition header. In your WebClient class you would retrieve that header to save the file as suggested by your AwesomeSite site.

Although the solution proposed by Shadow Wizard works well for text files, I needed to support downloading binary files, such as pictures and executables, in my application.
Here is a small extension to WebClient that does the trick. Download is asynchronous. Also default value for file name is required, because we don't really know if the server would send all the right headers.
static class WebClientExtensions
{
public static async Task<string> DownloadFileToDirectory(this WebClient client, string address, string directory, string defaultFileName)
{
if (!Directory.Exists(directory))
throw new DirectoryNotFoundException("Downloads directory must exist");
string filePath = null;
using (var stream = await client.OpenReadTaskAsync(address))
{
var fileName = TryGetFileNameFromHeaders(client);
if (string.IsNullOrWhiteSpace(fileName))
fileName = defaultFileName;
filePath = Path.Combine(directory, fileName);
await WriteStreamToFile(stream, filePath);
}
return filePath;
}
private static string TryGetFileNameFromHeaders(WebClient client)
{
// content-disposition might contain the suggested file name, typically same as origiinal name on the server
// Originally content-disposition is for email attachments, but web servers also use it.
string contentDisposition = client.ResponseHeaders["content-disposition"];
return string.IsNullOrWhiteSpace(contentDisposition) ?
null :
new ContentDisposition(contentDisposition).FileName;
}
private static async Task WriteStreamToFile(Stream stream, string filePath)
{
// Code below will throw generously, e. g. when we don't have write access, or run out of disk space
using (var outStream = new FileStream(filePath, FileMode.CreateNew))
{
var buffer = new byte[8192];
while (true)
{
int bytesRead = await stream.ReadAsync(buffer, 0, buffer.Length);
if (bytesRead == 0)
break;
// Could use async variant here as well. Probably helpful when downloading to a slow network share or tape. Not my use case.
outStream.Write(buffer, 0, bytesRead);
}
}
}
}

Ok, my turn.
I had a few things in mind when I tried to "download the file":
Use only HttpClient. I had a couple of extension methods over it, and it wasn't desirable to create other extensions for WebClient.
It was mandatory for me also to get a File name.
I had to write the result to MemoryStream but not FileStream.
Solution
So, for me, it turned out to be this code:
// assuming that httpClient created already (including the Authentication cumbersome)
var response = await httpClient.GetAsync(absoluteURL); // call the external API
// reading file name from HTTP headers
var fileName = response.Content.Headers.ContentDisposition.FileNameStar; // also available to read from ".FileName"
// reading file as a byte array
var fileBiteArr = await response.Content
.ReadAsByteArrayAsync()
.ConfigureAwait(false);
var memoryStream = new MemoryStream(fileBiteArr); // memory streamed file
Test
To test that the Stream contains what we have, we can check it by converting it to file:
// getting the "Downloads" folder location, can be anything else
string pathUser = Environment.GetFolderPath(Environment.SpecialFolder.UserProfile);
string downloadPath = Path.Combine(pathUser, "Downloads\\");
using (FileStream file =
new FileStream(
$"{downloadPath}/file.pdf",
FileMode.Create,
FileAccess.Write))
{
byte[] bytes = new byte[memoryStream .Length];
memoryStream.Read(bytes, 0, (int)memoryStream.Length);
file.Write(bytes, 0, bytes.Length);
memoryStream.Close();
}