I'm using ServiceStack to create a Service. In one of the methods I write some data in response's output stream like this: await response.OutputStream.WriteAsync(Consts.DATA, 0, Consts.DATA.Length); where data is a byte[]. Ok, now the problem is when using this service from a client and GET the HttpResponseMessage for the specified method, I don't know how to get the data out. I wan't to unit test that the actual response contains the data I passed when writing to output content.
(I've tried to ReadAsByteArray but it throws an exception. When reading as Stream or String I don't know how to make it a byte[] to compare it in the test with the excpected Consts.DATA byte array.)
I'm pretty new to the field, excuse my ignorance if i miss something. Any help is appreciated.
Thanks!
If you want to use the typed client, you can do (from the documentation):
As raw bytes:
byte[] responseBytes = client.Get<byte[]>("/poco/World");
var dto = responseBytes.FromUtf8Bytes().FromJson<PocoResponse>();
dto.Result //Hello, World
Or as a Stream:
using (Stream responseStream = client.Get<Stream>("/poco/World")) {
var dto = responseStream.ReadFully()
.FromUtf8Bytes()
.FromJson<PocoResponse>();
dto.Result //Hello, World
}
Or even access the populated HttpWebResponse object:
HttpWebResponse webResponse = client.Get<HttpWebResponse>("/poco/World");
webResponse.Headers["X-Response"] //World
using (var stream = webResponse.GetResponseStream())
using (var sr = new StreamReader(stream)) {
var dto = sr.ReadToEnd().FromJson<PocoResponse>();
dto.Result //Hello, World
}
You can also use untyped ServiceStack client to access raw response, it is described in the documentation with samples.
If all you need is load the data from an URL as a Stream and youre using a HttpClient you can just do this to get a stream back:
var result = await client.GetStreamAsync(URL);
I think you can also use GetResponseStream() on your HttpWebResponse.
var result = message.GetResponseStream();
Run your output stream through this static method:
public static byte[] ReadFullStream(Stream st)
{
var lockTaken = false;
try
{
Monitor.Enter(_lock, ref lockTaken);
var size = 0;
var continueRead = true;
var buffer = (byte[])Array.CreateInstance(typeof(byte), 0x10000);
using (MemoryStream ms = new MemoryStream())
{
while (continueRead)
{
size = st.Read(buffer, 0, buffer.Length);
if (size > 0)
{
ms.Write(buffer, 0, size);
}
else
{
continueRead = false;
}
}
return ms.ToArray();
}
}
finally
{
if (lockTaken) { Monitor.Exit(_lock); }
}
}
EDIT: forgot, you'll need this defined at class scope:
private static readonly object _lock = new object();
Read response stream as follow-
using (var reader = new System.IO.StreamReader(response.GetResponseStream()))
{
responseContent = reader.ReadToEnd();
if (response.StatusCode == HttpStatusCode.OK && response.StatusDescription.ToUpper() == "OK" && !string.IsNullOrEmpty(responseContent))
{
objFSLstGetAgent = JsonConvert.DeserializeObject<YourObjectClass>(responseContent);
}
}
Related
I'm having troubles to make my program work correctly - here I explain :
I have, on one hand a C# WinForms app which launches an instance of IE by using the "Navigate" method : myWebBrowser.Navigate(myUrl, "_blank", intarray, "");, with intarray defined like this : byte[] intarray = BitConverter.GetBytes(id);. On this side, it works.
On the other side, I have an ASP .NET WebForms application which has to retrieve this intarray. I've tried this.
if (HttpContext.Current != null)
{
if (Session["Authenticated"] == null)
{
var current = HttpContext.Current;
byte[] postdata = getpostdata(current);
}
}
private byte[] getpostdata(HttpContext CurrentContext)
{
MemoryStream ms = new MemoryStream();
CurrentContext.Request.InputStream.CopyTo(ms);
byte[] postdata = ms.ToArray();
return postdata;
}
// Convert a byte array to an Object
public int ByteArrayToInt(byte[] arrBytes)
{
if (BitConverter.IsLittleEndian) Array.Reverse(arrBytes);
int i = BitConverter.ToInt32(arrBytes, 0);
return i;
}
The problem seems to be in retrieving the Data in the getpostdata(HttpContext) function... I get a byte array with length = 0 instead of the one which is sent with length = 4...
Does anyone know how to make it work ?
Yann
var current = HttpContext.Current;
var sr = new StreamReader(Request.InputStream, Encoding.Default);
var postdata = sr.ReadToEnd();
above
I am trying to compress a string (str) using SharpCompress' BZip2Stream but unable to achieve it. Following is the code I have so far,
public static string Compress(string str)
{
var data = Encoding.UTF8.GetBytes(str);
using (MemoryStream stream = new MemoryStream())
{
using (BZip2Stream zip = new BZip2Stream(stream, SharpCompress.Compressor.CompressionMode.Compress))
{
zip.Write(data, 0, data.Length);
var compressed = Encoding.UTF8.GetString(stream.ToArray());
return compressed;
}
}
}
No matter what string i pass to str it always returns BZh.
Any help is greatly appreciated!
I believe you need to finalize/close/flush the bzip2 stream in order to make sure all compressed data is written to the memory stream prior to reading data from the memory stream. Try:
using (MemoryMemoryStream stream = new MemoryStream())
{
using (BZip2Stream zip = new BZip2Stream(stream, SharpCompress.Compressor.CompressionMode.Compress))
{
zip.Write(data, 0, data.Length);
zip.Close();
}
var compressed = Encoding.UTF8.GetString(stream.ToArray());
return compressed;
}
I have been trying to put together an in-memory public-key encryption infrastructure using OpenPGP via Bouncy Castle. One of our vendors uses OpenPGP public key encryption to encrypt all their feeds, and requires us to do the same, so I'm stuck with the technology and the implementation. So now I'm coding an OpenPGP encryption/ decryption toolkit for automating these feeds.
The examples at bouncycastle.org inexplicably default to writing encrypted data to and collecting keys from a file system; this is not what I want to do, so I've been trying to get everything stream-based.
I have gotten to the point where I can actually get my code to compile and run, but my encrypted payload is empty. I think I'm missing something silly, but after several days of trying this and that, I have lost the ability to objectively examine this.
My utility class contains these methods:
public static PgpPublicKey ImportPublicKey(
this Stream publicIn)
{
var pubRings =
new PgpPublicKeyRingBundle(PgpUtilities.GetDecoderStream(publicIn)).GetKeyRings().OfType<PgpPublicKeyRing>();
var pubKeys = pubRings.SelectMany(x => x.GetPublicKeys().OfType<PgpPublicKey>());
var pubKey = pubKeys.FirstOrDefault();
return pubKey;
}
public static Stream Streamify(this string theString, Encoding encoding = null)
{
encoding = encoding ?? Encoding.UTF8;
var stream = new MemoryStream(encoding.GetBytes(theString));
return stream;
}
public static string Stringify(this Stream theStream,
Encoding encoding = null)
{
encoding = encoding ?? Encoding.UTF8;
using (var reader = new StreamReader(theStream, encoding))
{
return reader.ReadToEnd();
}
}
public static byte[] ReadFully(this Stream stream)
{
if (!stream.CanRead) throw new ArgumentException("This is not a readable stream.");
var buffer = new byte[32768];
using (var ms = new MemoryStream())
{
while (true)
{
var read = stream.Read(buffer, 0, buffer.Length);
if (read <= 0)
return ms.ToArray();
ms.Write(buffer, 0, read);
}
}
}
public static void PgpEncrypt(
this Stream toEncrypt,
Stream outStream,
PgpPublicKey encryptionKey,
bool armor = true,
bool verify = true,
CompressionAlgorithmTag compressionAlgorithm = CompressionAlgorithmTag.Zip)
{
if (armor) outStream = new ArmoredOutputStream(outStream);
var compressor = new PgpCompressedDataGenerator(CompressionAlgorithmTag.Zip);
outStream = compressor.Open(outStream);
var data = toEncrypt.ReadFully();
var encryptor = new PgpEncryptedDataGenerator(SymmetricKeyAlgorithmTag.Cast5, verify, new SecureRandom());
encryptor.AddMethod(encryptionKey);
outStream = encryptor.Open(outStream, data.Length);
outStream.Write(data, 0, data.Length);
}
My test method looks like this:
private static void EncryptMessage()
{
var pubKey = #"<public key text>";
var clearText = "This is an encrypted message. There are many like it but this one is cryptic.";
using (var stream = pubKey.Streamify())
{
var key = stream.ImportPublicKey();
using (var clearStream = clearText.Streamify())
using (var cryptoStream = new MemoryStream())
{
clearStream.PgpEncrypt(cryptoStream,key);
cryptoStream.Position = 0;
Console.WriteLine(cryptoStream.Stringify());
Console.WriteLine("Press any key to continue.");
}
}
Console.ReadKey();
}
The result I get looks like this:
-----BEGIN PGP MESSAGE-----
Version: BCPG C# v1.7.4114.6378
Press any key to continue.
Can someone tell me what I am doing wrong?
OK, I managed to get this working. There were several problems with this implementation. One problem was that certain things had to be done in order. Here is what seems to need to happen:
The raw data needs to be put into a PgpLiteralData object
The literal data needs to be encrypted.
The encrypted data needs to be compressed.
The compressed data (optionally) needs to be armored.
The underlying streams need to be closed in order of usage.
There should be a more elegant way to do this, but the streams used by the BouncyCastle library are all frustratingly one-way, and at several points, I needed to convert the stream to a byte array to get another part to work. I include the code I used and independently verified; if someone has a verifyably better way of doing this, I would be quite interested.
public static class OpenPgpUtility
{
public static void ExportKeyPair(
Stream secretOut,
Stream publicOut,
AsymmetricKeyParameter publicKey,
AsymmetricKeyParameter privateKey,
string identity,
char[] passPhrase,
bool armor)
{
if (armor)
{
secretOut = new ArmoredOutputStream(secretOut);
}
var secretKey = new PgpSecretKey(
PgpSignature.DefaultCertification,
PublicKeyAlgorithmTag.RsaGeneral,
publicKey,
privateKey,
DateTime.UtcNow,
identity,
SymmetricKeyAlgorithmTag.Cast5,
passPhrase,
null,
null,
new SecureRandom()
);
secretKey.Encode(secretOut);
if (armor)
{
secretOut.Close();
publicOut = new ArmoredOutputStream(publicOut);
}
var key = secretKey.PublicKey;
key.Encode(publicOut);
if (armor)
{
publicOut.Close();
}
}
public static PgpPublicKey ImportPublicKey(
this Stream publicIn)
{
var pubRings =
new PgpPublicKeyRingBundle(PgpUtilities.GetDecoderStream(publicIn)).GetKeyRings().OfType<PgpPublicKeyRing>();
var pubKeys = pubRings.SelectMany(x => x.GetPublicKeys().OfType<PgpPublicKey>());
var pubKey = pubKeys.FirstOrDefault();
return pubKey;
}
public static PgpSecretKey ImportSecretKey(
this Stream secretIn)
{
var secRings =
new PgpSecretKeyRingBundle(PgpUtilities.GetDecoderStream(secretIn)).GetKeyRings().OfType<PgpSecretKeyRing>();
var secKeys = secRings.SelectMany(x => x.GetSecretKeys().OfType<PgpSecretKey>());
var secKey = secKeys.FirstOrDefault();
return secKey;
}
public static Stream Streamify(this string theString, Encoding encoding = null)
{
encoding = encoding ?? Encoding.UTF8;
var stream = new MemoryStream(encoding.GetBytes(theString));
return stream;
}
public static string Stringify(this Stream theStream,
Encoding encoding = null)
{
encoding = encoding ?? Encoding.UTF8;
using (var reader = new StreamReader(theStream, encoding))
{
return reader.ReadToEnd();
}
}
public static byte[] ReadFully(this Stream stream, int position = 0)
{
if (!stream.CanRead) throw new ArgumentException("This is not a readable stream.");
if (stream.CanSeek) stream.Position = 0;
var buffer = new byte[32768];
using (var ms = new MemoryStream())
{
while (true)
{
var read = stream.Read(buffer, 0, buffer.Length);
if (read <= 0)
return ms.ToArray();
ms.Write(buffer, 0, read);
}
}
}
public static void PgpEncrypt(
this Stream toEncrypt,
Stream outStream,
PgpPublicKey encryptionKey,
bool armor = true,
bool verify = false,
CompressionAlgorithmTag compressionAlgorithm = CompressionAlgorithmTag.Zip)
{
var encryptor = new PgpEncryptedDataGenerator(SymmetricKeyAlgorithmTag.Cast5, verify, new SecureRandom());
var literalizer = new PgpLiteralDataGenerator();
var compressor = new PgpCompressedDataGenerator(compressionAlgorithm);
encryptor.AddMethod(encryptionKey);
//it would be nice if these streams were read/write, and supported seeking. Since they are not,
//we need to shunt the data to a read/write stream so that we can control the flow of data as
//we go.
using (var stream = new MemoryStream()) // this is the read/write stream
using (var armoredStream = armor ? new ArmoredOutputStream(stream) : stream as Stream)
using (var compressedStream = compressor.Open(armoredStream))
{
//data is encrypted first, then compressed, but because of the one-way nature of these streams,
//other "interim" streams are required. The raw data is encapsulated in a "Literal" PGP object.
var rawData = toEncrypt.ReadFully();
var buffer = new byte[1024];
using (var literalOut = new MemoryStream())
using (var literalStream = literalizer.Open(literalOut, 'b', "STREAM", DateTime.UtcNow, buffer))
{
literalStream.Write(rawData, 0, rawData.Length);
literalStream.Close();
var literalData = literalOut.ReadFully();
//The literal data object is then encrypted, which flows into the compressing stream and
//(optionally) into the ASCII armoring stream.
using (var encryptedStream = encryptor.Open(compressedStream, literalData.Length))
{
encryptedStream.Write(literalData, 0, literalData.Length);
encryptedStream.Close();
compressedStream.Close();
armoredStream.Close();
//the stream processes are now complete, and our read/write stream is now populated with
//encrypted data. Convert the stream to a byte array and write to the out stream.
stream.Position = 0;
var data = stream.ReadFully();
outStream.Write(data, 0, data.Length);
}
}
}
}
}
My test method looked like this:
private static void EncryptMessage()
{
var pubKey = #"<public key text here>";
var clearText = #"<message text here>";
using (var stream = pubKey.Streamify())
{
var key = stream.ImportPublicKey();
using (var clearStream = clearText.Streamify())
using (var cryptoStream = new MemoryStream())
{
clearStream.PgpEncrypt(cryptoStream, key);
cryptoStream.Position = 0;
var cryptoString = cryptoStream.Stringify();
Console.WriteLine(cryptoString);
Console.WriteLine("Press any key to continue.");
}
}
Console.ReadKey();
}
Since someone asked, my decryption algorithm looked like this:
public static Stream PgpDecrypt(
this Stream encryptedData,
string armoredPrivateKey,
string privateKeyPassword,
Encoding armorEncoding = null)
{
armorEncoding = armorEncoding ?? Encoding.UTF8;
var stream = PgpUtilities.GetDecoderStream(encryptedData);
var layeredStreams = new List<Stream> { stream }; //this is to clean up/ dispose of any layered streams.
var dataObjectFactory = new PgpObjectFactory(stream);
var dataObject = dataObjectFactory.NextPgpObject();
Dictionary<long, PgpSecretKey> secretKeys;
using (var privateKeyStream = armoredPrivateKey.Streamify(armorEncoding))
{
var secRings =
new PgpSecretKeyRingBundle(PgpUtilities.GetDecoderStream(privateKeyStream)).GetKeyRings()
.OfType<PgpSecretKeyRing>();
var pgpSecretKeyRings = secRings as PgpSecretKeyRing[] ?? secRings.ToArray();
if (!pgpSecretKeyRings.Any()) throw new ArgumentException("No secret keys found.");
secretKeys = pgpSecretKeyRings.SelectMany(x => x.GetSecretKeys().OfType<PgpSecretKey>())
.ToDictionary(key => key.KeyId, value => value);
}
while (!(dataObject is PgpLiteralData) && dataObject != null)
{
try
{
var compressedData = dataObject as PgpCompressedData;
var listedData = dataObject as PgpEncryptedDataList;
//strip away the compression stream
if (compressedData != null)
{
stream = compressedData.GetDataStream();
layeredStreams.Add(stream);
dataObjectFactory = new PgpObjectFactory(stream);
}
//strip the PgpEncryptedDataList
if (listedData != null)
{
var encryptedDataList = listedData.GetEncryptedDataObjects()
.OfType<PgpPublicKeyEncryptedData>().First();
var decryptionKey = secretKeys[encryptedDataList.KeyId]
.ExtractPrivateKey(privateKeyPassword.ToCharArray());
stream = encryptedDataList.GetDataStream(decryptionKey);
layeredStreams.Add(stream);
dataObjectFactory = new PgpObjectFactory(stream);
}
dataObject = dataObjectFactory.NextPgpObject();
}
catch (Exception ex)
{
//Log exception here.
throw new PgpException("Failed to strip encapsulating streams.", ex);
}
}
foreach (var layeredStream in layeredStreams)
{
layeredStream.Close();
layeredStream.Dispose();
}
if (dataObject == null) return null;
var literalData = (PgpLiteralData)dataObject;
var ms = new MemoryStream();
using (var clearData = literalData.GetInputStream())
{
Streams.PipeAll(clearData, ms);
}
ms.Position = 0;
return ms;
}
I'm working on a quick wrapper for the skydrive API in C#, but running into issues with downloading a file. For the first part of the file, everything comes through fine, but then there start to be differences in the file and shortly thereafter everything becomes null. I'm fairly sure that it's just me not reading the stream correctly.
This is the code I'm using to download the file:
public const string ApiVersion = "v5.0";
public const string BaseUrl = "https://apis.live.net/" + ApiVersion + "/";
public SkyDriveFile DownloadFile(SkyDriveFile file)
{
string uri = BaseUrl + file.ID + "/content";
byte[] contents = GetResponse(uri);
file.Contents = contents;
return file;
}
public byte[] GetResponse(string url)
{
checkToken();
Uri requestUri = new Uri(url + "?access_token=" + HttpUtility.UrlEncode(token.AccessToken));
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(requestUri);
request.Method = WebRequestMethods.Http.Get;
WebResponse response = request.GetResponse();
Stream responseStream = response.GetResponseStream();
byte[] contents = new byte[response.ContentLength];
responseStream.Read(contents, 0, (int)response.ContentLength);
return contents;
}
This is the image file I'm trying to download
And this is the image I am getting
These two images lead me to believe that I'm not waiting for the response to finish coming through, because the content-length is the same as the size of the image I'm expecting, but I'm not sure how to make my code wait for the entire response to come through or even really if that's the approach I need to take.
Here's my test code in case it's helpful
[TestMethod]
public void CanUploadAndDownloadFile()
{
var api = GetApi();
SkyDriveFolder folder = api.CreateFolder(null, "TestFolder", "Test Folder");
SkyDriveFile file = api.UploadFile(folder, TestImageFile, "TestImage.png");
file = api.DownloadFile(file);
api.DeleteFolder(folder);
byte[] contents = new byte[new FileInfo(TestImageFile).Length];
using (FileStream fstream = new FileStream(TestImageFile, FileMode.Open))
{
fstream.Read(contents, 0, contents.Length);
}
using (FileStream fstream = new FileStream(TestImageFile + "2", FileMode.CreateNew))
{
fstream.Write(file.Contents, 0, file.Contents.Length);
}
Assert.AreEqual(contents.Length, file.Contents.Length);
bool sameData = true;
for (int i = 0; i < contents.Length && sameData; i++)
{
sameData = contents[i] == file.Contents[i];
}
Assert.IsTrue(sameData);
}
It fails at Assert.IsTrue(sameData);
This is because you don't check the return value of responseStream.Read(contents, 0, (int)response.ContentLength);. Read doesn't ensure that it will read response.ContentLength bytes. Instead it returns the number of bytes read. You can use a loop or stream.CopyTo there.
Something like this:
WebResponse response = request.GetResponse();
MemoryStream m = new MemoryStream();
response.GetResponseStream().CopyTo(m);
byte[] contents = m.ToArray();
As LB already said, you need to continue to call Read() until you have read the entire stream.
Although Stream.CopyTo will copy the entire stream it does not ensure that read the number of bytes expected. The following method will solve this and raise an IOException if it does not read the length specified...
public static void Copy(Stream input, Stream output, long length)
{
byte[] bytes = new byte[65536];
long bytesRead = 0;
int len = 0;
while (0 != (len = input.Read(bytes, 0, Math.Min(bytes.Length, (int)Math.Min(int.MaxValue, length - bytesRead)))))
{
output.Write(bytes, 0, len);
bytesRead = bytesRead + len;
}
output.Flush();
if (bytesRead != length)
throw new IOException();
}
Behold the code:
using (var client = new WebClient())
{
using (var stream = client.OpenWrite("http://localhost/", "POST"))
{
stream.Write(post, 0, post.Length);
}
}
Now, how do I read the HTTP output?
It looks like you have a byte[] of data to post; in which case I expect you'll find it easier to use:
byte[] response = client.UploadData(address, post);
And if the response is text, something like:
string s = client.Encoding.GetString(response);
(or your choice of Encoding - perhaps Encoding.UTF8)
If you want to keep streams everywhere and avoid allocating huge arrays of bytes, which is good practise (for example, if you plan to post big files), you still can do it with a derived version of WebClient. Here is a sample code that does it.
using (var client = new WebClientWithResponse())
{
using (var stream = client.OpenWrite(myUrl))
{
// open a huge local file and send it
using (var file = new FileStream(filePath, FileMode.Open, FileAccess.Read, FileShare.ReadWrite))
{
file.CopyTo(stream);
}
}
// get response as an array of bytes. You'll need some encoding to convert to string, etc.
var bytes = client.Response;
}
And here is the customized WebClient:
public class WebClientWithResponse : WebClient
{
// we will store the response here. We could store it elsewhere if needed.
// This presumes the response is not a huge array...
public byte[] Response { get; private set; }
protected override WebResponse GetWebResponse(WebRequest request)
{
var response = base.GetWebResponse(request);
var httpResponse = response as HttpWebResponse;
if (httpResponse != null)
{
using (var stream = httpResponse.GetResponseStream())
{
using (var ms = new MemoryStream())
{
stream.CopyTo(ms);
Response = ms.ToArray();
}
}
}
return response;
}
}