The following unit test fails:
[TestMethod]
public void Add_file_to_blob_and_retrieve_it()
{
var blobName = Guid.NewGuid().ToString();
var testFileContents = File.ReadAllText(TestFileSpec);
Trace.WriteLine(string.Format("Opening blob container {0}", UnitTestBlobAgentName));
CloudStorageAccount.SetConfigurationSettingPublisher(
(configName, configSetter) => configSetter(ConfigurationManager.AppSettings[configName]));
var cloudStorage = CloudStorageAccount.FromConfigurationSetting("StorageConnectionString");
var blobClient = cloudStorage.CreateCloudBlobClient();
var container = blobClient.GetContainerReference(UnitTestBlobAgentName.ToLower());
try
{
Trace.WriteLine(string.Format("Uploading file {0}", TestFileSpec));
var blob = container.GetBlobReference(blobName);
blob.UploadFile(TestFileSpec);
blob.Properties.ContentType = "ByteArray";
blob.SetProperties();
var blob1 = container.GetBlobReference(blobName);
var found = blob1.DownloadText();
Assert.AreEqual(testFileContents.Trim(), found.Trim());
}
finally
{
if (null != container)
{
Trace.WriteLine(string.Format("Deleting blob {0}", blobName));
var blob2 = container.GetBlobReference(blobName);
blob2.DeleteIfExists(new BlobRequestOptions { DeleteSnapshotsOption = DeleteSnapshotsOption.IncludeSnapshots });
}
}
}
It turns out, the returned string begins with the dword 0xFEFF (the Unicode BOM). I've traced through the Microsoft debug symbols, and the BOM exists in the return stream. AFAICT, it comes from the HttpResponse.GetResponseStream() method call way down in the Microsoft.WindowsAzure.StorageClient.CloudBlob class.
What's the best way to ensure that the input and output are identical? Ensure the input is converted to Unicode before going in? Strip the BOM from the output? Any other ideas?
This is an old one, but if your blob is encoded as Unicode in azure, and you want to download it to a text string, this code will do the trick. Just keep in mind, the weakness here is that you've got to allocate the memory twice. If there's a more efficient way of getting to a Unicode string (synchronously, anyway,) I couldn't find it.
string fileText;
using (var memoryStream = new MemoryStream())
{
cloudBlob.DownloadToStream(memoryStream);
memoryStream.Position = 0;
using (var reader = new StreamReader(memoryStream, Encoding.Unicode))
{
fileText = reader.ReadToEnd();
}
}
Related
I need to save the information from an input page into a JSON File and output the information onto another page reading from the JSON File. I've tried many things and what seemed to work for me is using the specialfolder localapplication data.
Now, I don't quite understand how I can output the information and also check if the data is even put in correctly.
I previously used StreamReader to output the information on the JSON file and then put it on a ListView but this doesn't work if I have the file in the specialfolder. It says "stream cant be null". The commented out code is the code I tried in previous attempts.
Code:
ListPageVM (Read Page)
private ObservableCollection<MainModel> data;
public ObservableCollection<MainModel> Data
{
get { return data; }
set { data = value; OnPropertyChanged(); }
}
public ListPageVM()
{
var assembly = typeof(ListPageVM).GetTypeInfo().Assembly;
Stream stream = assembly.GetManifestResourceStream(Path.Combine(System.Environment.GetFolderPath(Environment.SpecialFolder.LocalApplicationData), "eintraege.json"/"SaveUp.Resources.eintraege.json"/));
//var file = Path.Combine(System.Environment.GetFolderPath(Environment.SpecialFolder.LocalApplicationData), "eintraege.json");
using (var reader = new StreamReader(stream))
{
var json = reader.ReadToEnd();
List<MainModel> dataList = JsonConvert.DeserializeObject<List<MainModel>>(json);
data = new ObservableCollection<MainModel>(dataList);
}
}
MainPageVM (Write Page)
public Command Einfügen
{
get
{
return new Command(() =>
{
// Data ins Json
_mainModels.Add(DModel);
Datum = DateTime.Now.ToString("dd.mm.yyyy");
//var assembly = typeof(ListPageVM).GetTypeInfo().Assembly;
//FileStream stream = new FileStream("SaveUp.Resources.eintraege.json", FileMode.OpenOrCreate, FileAccess.Write);
var file = Path.Combine(System.Environment.GetFolderPath(Environment.SpecialFolder.LocalApplicationData), "eintraege.json");
//Stream stream = assembly.GetManifestResourceStream("SaveUp.Resources.eintraege.json");
if (!File.Exists(file))
{
File.Create(file);
}
using (var writer = File.AppendText(file))
{
string data = JsonConvert.SerializeObject(_mainModels);
writer.WriteLine(data);
}
});
}
}
you are trying to read and write resources, not files. That won't work. Instead do this
var path = Path.Combine(System.Environment.GetFolderPath(Environment.SpecialFolder.LocalApplicationData), "eintraege.json");
File.WriteAllText(path, myjson);
to read the data back
var json = File.ReadAllText(path);
I have an XML file in my blob storage. It contains words like this: Družstevní.
When I download the XML using Azure portal, this word is still correct.
But when I try using DownloadToStreamAsync the result is Dru�stevn�.
How do I fix this?
I found DownloadTextAsync is working because I get set the encoding: Encoding.GetEncoding(1252).
But then I end up with a string and the rest of my code is expecting a stream. Should I read the string again as a stream or exists a more elegant option?
Here's my code:
public Task<string> DownloadAsTextAsync(string code, Encoding encoding)
{
var blockBlob = _container.GetBlockBlobReference(code);
var blobRequestOptions = new BlobRequestOptions
{
MaximumExecutionTime = TimeSpan.FromMinutes(15),
ServerTimeout = TimeSpan.FromHours(1)
};
return blockBlob.DownloadTextAsync(Encoding.GetEncoding(1252), null, blobRequestOptions, null);
}
public async Task<Stream> DownloadAsStreamAsync(string code)
{
var blockBlob = _container.GetBlockBlobReference(code);
var blobRequestOptions = new BlobRequestOptions
{
MaximumExecutionTime = TimeSpan.FromMinutes(15),
ServerTimeout = TimeSpan.FromHours(1)
};
var output = new MemoryStream();
await blockBlob.DownloadToStreamAsync(output, null, blobRequestOptions, null);
return output;
}
Edit, after comment of Zhaoxing Lu:
I changed my unit test and added the encoding to StreamReader and now the unit test is passing:
using (var streamReader = new StreamReader(stream, Encoding.GetEncoding(1252)))
{
string line;
while ((line = streamReader.ReadLine()) != null)
{
if (!line.StartsWith(" <Str>Dru")) continue;
Debug.WriteLine(line);
var street = line.Trim().Replace("<Str>", "").Replace("</Str>", "");
Assert.AreEqual("Družstevní", street);
}
}
But in my 'real' code I'm sending the stream to load as XML:
fileStream.Position = 0;
var xmlDocument = XDocument.Load(fileStream);
The resulting xmlDocument is in the wrong encoding. I can't find how to set the encoding.
The problem seems to be when reading the stream as an XDocument
You could set the encoding as Encoding.GetEncoding("Windows-1252") with the following code to read the stream as XDocument.
XDocument xmlDoc = null;
using (StreamReader oReader = new StreamReader(stream, Encoding.GetEncoding("Windows-1252")))
{
xmlDoc = XDocument.Load(oReader);
}
The result:
Since System.IO.Compression seems to be out of reach for now if I want to use both dotnet core + net461, I've tried with SharpCompress.
The "read zip" part was easy, but I am having trouble finding out how to write to a zip stream.
The wiki of the project is a bit outdated. This is the only example that I've found that applies to writing to streams. I've tried to follow it and adapt it to my needs, but I am stuck at the exception it throws:
using Microsoft.VisualStudio.TestTools.UnitTesting;
using SharpCompress.Common;
using SharpCompress.Compressors.Deflate;
using SharpCompress.Writers;
using System;
using System.IO;
namespace DbManager.DjdbCore.Tests
{
[TestClass]
public class ZipTests
{
public ZipTests()
{
Directory.SetCurrentDirectory(AppContext.BaseDirectory);
}
[TestMethod]
public void Test()
{
var zip = File.OpenWrite(#"..\..\..\..\..\test-resources\zip_file_test.zip");
var writerOptions = new WriterOptions(CompressionType.Deflate);
var zipWriter = WriterFactory.Open(zip, ArchiveType.Zip, writerOptions);
var memoryStream = new MemoryStream();
var binaryWriter = new BinaryWriter(memoryStream);
binaryWriter.Write("Test string inside binary file - text to fill it up: qoiwjqefñlawijfñlaskdjfioqwjefñalskvndñaskvnqo`wiefowainvñaslkfjnwpowiqjfeopwiqjnfjñlaskdjfñlasdfjiowiqjefñaslkdjfñalskjfpqwoiefjqw");
var deflateStream = new DeflateStream(memoryStream, SharpCompress.Compressors.CompressionMode.Compress);
deflateStream.Write(memoryStream.ToArray(), 0, Convert.ToInt32(memoryStream.Length));
// EXCEPTION: SharpCompress.Compressors.Deflate.ZlibException: 'Cannot Read after Writing.'
// Source code: if (_streamMode != StreamMode.Reader) { throw new ZlibException("Cannot Read after Writing."); }
zipWriter.Write("test_file_inside_zip.bin", deflateStream, DateTime.Now);
zip.Flush();
zipWriter.Dispose();
zip.Dispose();
}
}
}
In case it helps, this is what I used (and it worked, but only in dotnet core) using the library System.IO.Compression:
private void WriteAsZipBinary()
{
//Open the zip file if it exists, else create a new one
var zip = ZipPackage.Open(this.FileFullPath, FileMode.OpenOrCreate, FileAccess.ReadWrite);
var zipStream = ZipManager.GetZipWriteStream(zip, nameOfFileInsideZip);
var memoryStream = new MemoryStream();
var binaryWriter = new BinaryWriter(memoryStream);
// Here is where strings etc are written to the binary file:
WriteStuffInBinaryStream(ref binaryWriter);
//Read all of the bytes from the file to add to the zip file
byte[] bites = new byte[Convert.ToInt32(memoryStream.Length - 1) + 1];
memoryStream.Position = 0;
memoryStream.Read(bites, 0, Convert.ToInt32(memoryStream.Length));
binaryWriter.Dispose();
binaryWriter = null;
memoryStream.Dispose();
memoryStream = null;
zipStream.Position = 0;
zipStream.Write(bites, 0, bites.Length);
zip.Close();
}
public static Stream GetZipWriteStream(Package zip, string renamedFileName)
{
//Replace spaces with an underscore (_)
string uriFileName = renamedFileName.Replace(" ", "_");
//A Uri always starts with a forward slash "/"
string zipUri = string.Concat("/", Path.GetFileName(uriFileName));
Uri partUri = new Uri(zipUri, UriKind.Relative);
string contentType = "Zip"; // System.Net.Mime.MediaTypeNames.Application.Zip;
//The PackagePart contains the information:
// Where to extract the file when it's extracted (partUri)
// The type of content stream (MIME type): (contentType)
// The type of compression: (CompressionOption.Normal)
PackagePart pkgPart = zip.CreatePart(partUri, contentType, CompressionOption.Normal);
//Compress and write the bytes to the zip file
return pkgPart.GetStream();
}
I'll post here the answer on github from #adamhathcock (the owner of the project):
[TestMethod]
public void Test()
{
var writerOptions = new WriterOptions(CompressionType.Deflate);
using(var zip = File.OpenWrite(#"..\..\..\..\..\test-resources\zip_file_test.zip"))
using(var zipWriter = WriterFactory.Open(zip, ArchiveType.Zip, writerOptions))
{
var memoryStream = new MemoryStream();
var binaryWriter = new BinaryWriter(memoryStream);
binaryWriter.Write("Test string inside binary file - text to fill it up: qoiwjqefñlawijfñlaskdjfioqwjefñalskvndñaskvnqo`wiefowainvñaslkfjnwpowiqjfeopwiqjnfjñlaskdjfñlasdfjiowiqjefñaslkdjfñalskjfpqwoiefjqw");
memoryStream.Position = 0;
zipWriter.Write("test_file_inside_zip.bin", memoryStream, DateTime.Now);
}
}
2 things:
You forgot to reset the MemoryStream after writing to it so it can be read.
You don't need to manually use the DeflateStream. You've told the ZipWriter what compression to use. If it worked, you would have double compressed the bytes which would be garbage really.
This is a very strange question.
I using C# to create a pass.json and save it to memoryStream, it work normally. After that I create the manifest.json SHA1 data which including that pass.json, the string of manifest.json like this and it is totally correct.
{"icon.png": "9423bd00e2b01c59a3265c38b5062fac7da0752d",
"icon#2x.png": "4d1db55bdaca70b685c013529a1c0dcbd7046524",
"logo.png": "ee5b053e63dbfe3b78378c15d163331d68a0ede8",
"logo#2x.png": "2f9e3a55bded1163620719a4d6c1ad496ed40c17",
"pass.json": "fd68bf77757d3057263a9aca0e5110ddd933934a"}
After generate pkpass as my phone, it can't open. I change the pass.json SHA1 code as "fd68bf77757d3057263a9aca0e5110ddd933934a" without using a value to save it, it work.
The coding like following:
// This version run success
var strPass = JavascriptSerialize(details);
var sw = new StreamWriter(assetsFolder + #"pass.json");
sw.Write(strPass);
sw.Close();
manifest.passjson = GetSha1Hash(assetsFolder + manifest.GetAssetBoardingPass(libPkPass_object_boardingPass.JsonObjects.AssetTypes.passjson));
//manifest.passjson = "2f9e3a55bded1163620719a4d6c1ad496ed40c17"
// end
// This version run fail
var strPass = JavascriptSerialize(details);
MemoryStream stream = new MemoryStream();
StreamWriter writer = new StreamWriter(strPass);
writer.Write(s);
writer.Flush();
stream.Position = 0;
var a = GetSha1HashMemory(passStream);
private static string GetSha1HashMemory(Stream passStream)
{
//var bs = new BufferedStream(passStream);
using (SHA1Managed sha = new SHA1Managed())
{
byte[] checksum = sha.ComputeHash(passStream);
string sendCheckSum = BitConverter.ToString(checksum)
.Replace("-", string.Empty);
return sendCheckSum.ToString().ToLower();
}
}
manifest.passjson = a;
//manifest.passjson = "2f9e3a55bded1163620719a4d6c1ad496ed40c17" (same data )
//end
What is going on?????? I can find out any question that string is wrong.
The pkpass provide in here (sendspace).
Can any body told me where is wrong?
Big Thank!
Two mistakes :
ComputeHash(Stream) and using Stream
ComputeHash(Stream) : ComputeHash stream only using System.IO.Stream, but not MemoryStream, change to ComputeHash(bytes[]) can handle it
using Stream: I try to pass the stream to other function, it is not a good example, the stream need to create a new one and it may replace some bytes at your computer stream. In this case, I just need to call this function will out open new one, it will fix
StringBuilder formatted;
using (var sha1 = new SHA1Managed())
{
//var bytePass = ReadFully(passStream);
var bytePass = passStream.ToArray();
var hash = sha1.ComputeHash(bytePass);
formatted = new StringBuilder(2 * hash.Length);
foreach (var b in hash)
{
formatted.AppendFormat("{0:X2}", b);
}
}
manifest.passjson = formatted.ToString().ToLower();
I'm trying to create a resx file and write it to a stream so that I might return it as a string instead of immediately saving it to a file. However, when I try to read that stream, it is empty. What am I doing wrong here? i did verify that the entries are not null. I can actually use the ResXResourceWriter constructor that saves it to disk successfully, but I'm trying to avoid using temp files. Also, I can see the stream is 0k before the loop and about 8k in length after the loop.
using (var stream = new MemoryStream())
{
using (var resx = new ResXResourceWriter(stream))
{
// build the resx and write to memory
foreach (var entry in InputFile.Entries.Values)
{
resx.AddResource(new ResXDataNode(entry.Key, entry.Value) { Comment = entry.Comment });
}
var reader = new StreamReader(stream);
var text = reader.ReadToEnd(); // text is an empty string here!
return null;
}
}
You need to flush and reset the output/stream before trying to read it. This should work, using Generate and Position:
resx.Generate();
stream.Position = 0;
var reader = new StreamReader(stream);
var text = reader.ReadToEnd();
return text;