I have Stream that I need to return through a protobuf message as bytes. How do I convert the Stream into the ByteString that is expected by protobuf? Is it as simple as it appears in the documentation Serialization?
Due to the nature of the project I'm unable to test it well so I'm kinda working blind.
Here is what I'm working with:
Protocol buffer:
message ProtoResponse {
bytes ResponseValue = 1;
}
C#
public ProtoResponse SendResponse(Stream stream)
{
var response = ProtoResponse
{
// this obviously does not work but
// but it conveys the idea of what I am going for
ResponseValue = stream
}
return response;
}
I have attempted to convert the Stream to a string or a byte[] but C# compiler in VS keeps showing this error message:
Cannot implicitly convert type '' to 'Google.Protobuf.ByteString'.
I know I am missing something and my knowledge of Streams and protocol buffers is lacking.
Actually, I may have answered my own question. ByteString has an extension that accepts a byte[].
public ProtoResponse SendResponse(Stream stream)
{
byte[] b;
using (var memoryStream = new MemoryStream())
{
stream.CopyTo(memoryStream);
b = memoryStream.ToArray();
}
var response = ProtoResponse
{
ResponseValue = ByteString.CopyFrom(b)
}
return response;
}
If anyone sees something wrong with this feel free to let me know! Thanks!
Im using C#, and Protobuf syntax = 3; with GRPC. In my case it looks like this:
I found method to change Image to ByteArray, this sample is here to understanding next part of my response.
private static byte[] ImageToByteArray(Bitmap image)
{
using (var ms = new MemoryStream())
{
image.Save(ms, image.RawFormat);
return ms.ToArray();
}
}
But, next i have to change Bytearray to ByteString of Protobuf3
byte[] img = ImageToByteArray(); //its method you can see above
ByteString bytestring;
using (var str = new MemoryStream(img))
{
bytestring = ByteString.FromStream(str);
}
You can simply use ByteString.FromStream(MemoryStream) without CopyFrom method.
If we take a look to receiver of this message, he need change ByteString to ByteArray to for example save photo:
byte[] img = request.Image.ToByteArray(); //this is received message
And thats all. You have exactly the same bytes in both sides.
Related
I've been struggling with this implementation for a few hours now and can't seem to find any solutions wherever I look (SO, Xamarin Forums, Google etc)...
In this current scenario I have a few images in .Droid.Resources.Drawable which I wish to access and convert into a byte[] from my shared code. This is due to the fact that I wish to test the full span of my CRUD functionality on a REST API I've set up as an end-point for our server.
The images show up fine in the application, but for some reason I simply can't seem to warp my head around the process of converting these images to a byte[] in Xamarin. I've done it countless times in 'normal' C#...
Sorry if the code is a bit messy, but I'm sure you get the idea.
I want to get an image from .Droid storage (will be ported for iOS later)
Convert said image into a byte[]
Send that byte[] representation to my API.
In the code's current state I'm getting this error:
C#: An instance of an abstract class can not be created
Where I'm attempting to create a new Stream (new Stream(sauce))
The below example is based on snippets found here and full credit goes to Sten and Vincent.
/*
* Takes an arbitrary string as a token, updates a record with dummy data and a placeholder_image.
*/
public async Task<string> PostUpdateFoundation(string arbitrary, Image img)
{
ImageSource sauce = ImageSource.FromFile("abc.png");
byte[] byte_img = FromStreamToByte(new Stream(sauce)); //error occurs here
Debug.WriteLine("I'm in!");
var client = new System.Net.Http.HttpClient();
client.DefaultRequestHeaders.Add("Accept", "application/json");
var content = new StringContent(arbitrary);
var response = await client.PostAsync(String.Format("http://some.api.to.test.com?s={0}&img={1}", arbitrary, byte_img), content);
var result = response.Content.ReadAsStringAsync().Result;
return result;
}
/*
* Attempts to convert an stream (based on image source) into a byte[].
*/
public static byte[] FromStreamToByte (Stream input)
{
using (MemoryStream ms = new MemoryStream())
{
input.CopyTo(ms);
return ms.ToArray();
}
}
Try using Plugin.Media
byte BImageSource = ReadFully(file.GetStream());
var bytes = new byte[file.GetStream().Length]; //file is from the plugin and contains your image
file.GetStream().Position = 0;
file.GetStream().Read(bytes, 0, (int)file.GetStream().Length);
BImageSource = ReadFully(file.GetStream()); //BImageSource is your resource in bytes
byte[] ReadFully(Stream input)
{
using (MemoryStream ms = new MemoryStream())
{
input.CopyTo(ms);
return ms.ToArray();
}
}
Hope this helps!
The SendGrid API docs specify you can add attachments from a Stream. The example it gives uses a FileStream object.
I have some blobs in Azure Storage which I'd like to email as attachments. To achieve this I'm trying to use a MemoryStream:
var getBlob = blobContainer.GetBlobReferenceFromServer(fileUploadLink.Name);
if(getBlob != null)
{
// Get file as a stream
MemoryStream memoryStream = new MemoryStream();
getBlob.DownloadToStream(memoryStream);
emailMessage.AddAttachment(memoryStream, fileUploadLink.Name);
}
emailTransport.Deliver(emailMessage);
It sends fine but when the email arrives, the attachment appears to be there but it's actually empty. Looking at the email source, there is no content for the attachment.
Is using a MemoryStream a known limitation when using the SendGrid C# API to send attachments? Or should I be approaching this in some other way?
You probably just need to reset the stream position back to 0 after you call DownloadToStream:
var getBlob = blobContainer.GetBlobReferenceFromServer(fileUploadLink.Name);
if (getBlob != null)
{
var memoryStream = new MemoryStream();
getBlob.DownloadToStream(memoryStream);
memoryStream.Seek(0,SeekOrigin.Begin); // Reset stream back to beginning
emailMessage.AddAttachment(memoryStream, fileUploadLink.Name);
}
emailTransport.Deliver(emailMessage);
You might want to check who cleans up the stream as well and if they don't you should dispose of it after you've called Deliver().
According to their API, they have implemented void AddAttachment(Stream stream, String name).
You are probably using a MemoryStream which you have written to before. I suggest resetting the position inside the stream to the beginning, like:
memoryStream.Seek(0, SeekOrigin.Begin);
I ended up with the following which fixed the issue for me:
fileByteArray = new byte[getBlob.Properties.Length];
getBlob.DownloadToByteArray(fileByteArray, 0);
attachmentFileStream = new MemoryStream(fileByteArray);
emailMessage.AddAttachment(attachmentFileStream, fileUploadLink.Name);
The thread is a bit old, but I use a varient with NReco PDF converter:
private async Task SendGridasyncBid(string from, string to, string displayName, string subject, **byte[] PDFBody**, string TxtBody, string HtmlBody)
{
...
var myStream = new System.IO.MemoryStream(**PDFBody**);
myStream.Seek(0, SeekOrigin.Begin);
myMessage.AddAttachment(myStream, "NewBid.pdf");
...
}
convert the html to pdf and return it instead of writing it for download...
private byte[] getHTML(newBidViewModel model)
{
string strHtml = ...;
HtmlToPdfConverter pdfConverter = new HtmlToPdfConverter();
pdfConverter.CustomWkHtmlArgs = "--page-size Letter";
var pdfBytes = pdfConverter.GeneratePdf(strHtml);
return **pdfBytes**;
}
I am not sure how efficient this is, but it is working for me and I hope it helps someone else get their attachments figured out.
I'm currently trying to build an app that involves the user selecting a photo from their library (or taking a photo) and uploading it to Imgur. I have already built a fairly robust C# Imgur client for Windows Forms applications, but unfortunately porting it to the Windows Phone has been a disaster.
Here is the code that I am using:
public void UploadImageAsync(Stream PhotoStream)
{
try
{
WebClient w = new WebClient();
w.Headers["Content-type"] = "application/x-www-form-urlencoded";
string data = "key="+PublicKey+
"&_fake_status=200"+
"&type=base64"+
"&image="+PhotoStreamToBase64(PhotoStream);
w.UploadStringAsync(new Uri("http://api.imgur.com/2/upload", UriKind.Absolute), "POST", data);
}
catch (Exception ex)
{
}
}
string PhotoStreamToBase64(Stream PhotoStream)
{
MemoryStream memoryStream = new MemoryStream();
PhotoStream.CopyTo(memoryStream);
byte[] result = memoryStream.ToArray();
return System.Convert.ToBase64String(result);
}
What is interesting (and frustrating) is that it appears as though everything is working fine, and I receive a successful response after the upload has completed. However, when trying to view the image after being uploaded, the result looks like this: http://i.imgur.com/NWY0R.jpg.
This leads me to believe that somehow the image stream is being converted into the byte array incorrectly, or converted into a base 64 string incorrectly. In any case, I cannot get it to work and I am at a total loss. Does anybody have any idea? Any help would be greatly appreciated.
SpikeX pushed me toward Imgur's C# API example for image uploading. Borrowing the Base64 encoding logic from their example fixed the issue. Here is the now functional PhotoStreamToBase64 method:
string PhotoStreamToBase64(Stream PhotoStream)
{
MemoryStream memoryStream = new MemoryStream();
PhotoStream.CopyTo(memoryStream);
byte[] result = memoryStream.ToArray();
string base64img = System.Convert.ToBase64String(result);
StringBuilder sb = new StringBuilder();
for (int i = 0; i < base64img.Length; i += 32766)
{
sb.Append(Uri.EscapeDataString(base64img.Substring(i, Math.Min(32766, base64img.Length - i))));
}
return sb.ToString();
}
I am trying to send large objects (>30MB) to a MSMQ queue. Due to the large amount of data we are are tring to send the idea was to GZip the objects prior to sending them, then unzipping them on the receiving end.
However, writing the compressed stream to the message.BodyStream property seems to work, but not reading it out from there.
I don't know what's wrong.
Message l_QueueMessage = new Message();
l_QueueMessage.Priority = priority;
using (MessageQueue l_Queue = CreateQueue())
{
GZipStream stream = new GZipStream(l_QueueMessage.BodyStream, CompressionMode.Compress);
Formatter.Serialize(stream, message);
l_Queue.Send(l_QueueMessage);
}
The Formatter is a global property of type BinaryFormatter. This is used to serialize/deserialize to the type of object we want to send/receive, e.g. "ProductItem".
The receving end looks like this:
GZipStream stream = new GZipStream(l_Message.BodyStream, CompressionMode.Decompress);
object decompressedObject = Formatter.Deserialize(stream);
ProductItem l_Item = decompressedObject as ProductItem;
m_ProductReceived(sender, new MessageReceivedEventArgs<ProductItem>(l_Item));
l_ProductQueue.BeginReceive();
I get an EndOfStreamException "{"Unable to read beyond the end of the stream."} trying to deserialize
at System.IO.BinaryReader.ReadByte()
Using the messageBodyStream property I actually circumvent the message.Formatter, which I don't initialize to anything, becasue I'm using my own ser/deser mechanism with the GZipStream. However, I am not sure if that's the correct way of doing this.
What am I missing?
Thanks!
In your original code, the problem is that you need to close the GZipStream in order for a GZip footer to be written correctly, and only then you can send it. If you dont, you end up sending bytes that can not be deserialized. That's also why your new code where sending is done later works.
OK, I made this work. The key was to convert the decompressed stream on the receiver to a byte[] array. Then the deserialization started working.
The sender code (notice the stream is closed before sending the message):
using (MessageQueue l_Queue = CreateQueue())
{
using (GZipStream stream = new GZipStream(l_QueueMessage.BodyStream, CompressionMode.Compress, true))
{
Formatter.Serialize(stream, message);
}
l_Queue.Send(l_QueueMessage);
}
The receiving end (notice how I convert the stream to a byte[] then deserialize):
using (GZipStream stream = new GZipStream(l_QueueMessage.BodyStream, CompressionMode.Decompress))
{
byte[] bytes = ReadFully(stream);
using (MemoryStream ms = new MemoryStream(bytes))
{
decompressedObject = Formatter.Deserialize(ms);
}
}
Still, don't know why this works using the ReadFully() function and not the Stream.CopyTo().
Does anyone?
Btw, ReadFully() is a function that creates a byte[] out of a Stream. I have to credit Jon Skeet for this at http://www.yoda.arachsys.com/csharp/readbinary.html. Thanks!
Try to separate compressing and sending:
byte[] binaryBuffer = null;
using (MemoryStream compressedBody = new MemoryStream())
{
using(GZipStream stream = new GZipStream(compressedBody, CompressionMode.Compress))
{
Formatter.Serialize(compressedBody, message);
binaryBuffer = compressedBody.GetBuffer();
}
}
using (MessageQueue l_Queue = CreateQueue())
{
l_QueueMessage.BodyStream.Write(binaryBuffer, 0, binaryBuffer.Length);
l_QueueMessage.BodyStream.Seek(0, SeekOrigin.Begin);
l_Queue.Send(l_QueueMessage);
}
I'm trying to get an image from an url using a byte stream. But i get this error message:
This stream does not support seek operations.
This is my code:
byte[] b;
HttpWebRequest myReq = (HttpWebRequest)WebRequest.Create(url);
WebResponse myResp = myReq.GetResponse();
Stream stream = myResp.GetResponseStream();
int i;
using (BinaryReader br = new BinaryReader(stream))
{
i = (int)(stream.Length);
b = br.ReadBytes(i); // (500000);
}
myResp.Close();
return b;
What am i doing wrong guys?
You probably want something like this. Either checking the length fails, or the BinaryReader is doing seeks behind the scenes.
HttpWebRequest myReq = (HttpWebRequest)WebRequest.Create(url);
WebResponse myResp = myReq.GetResponse();
byte[] b = null;
using( Stream stream = myResp.GetResponseStream() )
using( MemoryStream ms = new MemoryStream() )
{
int count = 0;
do
{
byte[] buf = new byte[1024];
count = stream.Read(buf, 0, 1024);
ms.Write(buf, 0, count);
} while(stream.CanRead && count > 0);
b = ms.ToArray();
}
edit:
I checked using reflector, and it is the call to stream.Length that fails. GetResponseStream returns a ConnectStream, and the Length property on that class throws the exception that you saw. As other posters mentioned, you cannot reliably get the length of a HTTP response, so that makes sense.
Use a StreamReader instead:
HttpWebRequest myReq = (HttpWebRequest)WebRequest.Create(url);
WebResponse myResp = myReq.GetResponse();
StreamReader reader = new StreamReader(myResp.GetResponseStream());
return reader.ReadToEnd();
(Note - the above returns a String instead of a byte array)
You can't reliably ask an HTTP connection for its length. It's possible to get the server to send you the length in advance, but (a) that header is often missing and (b) it's not guaranteed to be correct.
Instead you should:
Create a fixed-length byte[] that you pass to the Stream.Read method
Create a List<byte>
After each read, call List.AddRange to append the contents of your fixed-length buffer onto your byte list
Note that the last call to Read will return fewer than the full number of bytes you asked for. Make sure you only append that number of bytes onto your List<byte> and not the whole byte[], or you'll get garbage at the end of your list.
If the server doesn't send a length specification in the HTTP header, the stream size is unknown, so you get the error when trying to use the Length property.
Read the stream in smaller chunks, until you reach the end of the stream.
With images, you don't need to read the number of bytes at all. Just do this:
Image img = null;
string path = "http://www.example.com/image.jpg";
WebRequest request = WebRequest.Create(path);
req.Credentials = CredentialCache.DefaultCredentials; // in case your URL has Windows auth
WebResponse resp = req.GetResponse();
using( Stream stream = resp.GetResponseStream() )
{
img = Image.FromStream(stream);
// then use the image
}
Perhaps you should use the System.Net.WebClient API. If already using client.OpenRead(url) use client.DownloadData(url)
var client = new System.Net.WebClient();
byte[] buffer = client.DownloadData(url);
using (var stream = new MemoryStream(buffer))
{
... your code using the stream ...
}
Obviously this downloads everything before the Stream is created, so it may defeat the purpose of using a Stream. webClient.DownloadData("https://your.url") gets a byte array which you can then turn into a MemoryStream.
The length of a stream can not be read from the stream since the receiver does not know how many bytes the sender will send. Try to put a protocol on top of http and send i.e. the length as first item in the stream.