We files in svn which we extract in byte format with a webservice.
I am trying to move us over to getting these files from an S3 bucket.
When we extract the file from svn it comes out in bytes which we later decode using LZ4.
Like this:
Encoding.UTF8.GetString(LZHelper.ReadFromLZ4Stream(bytes returned by svn))
However, the file for our bucket is just uploaded without compression or anything. I extracted the bytes and tried to encode them so they would match the svn version. I assumed it would be UTF8 as that is the decode used above but it does not seem to match.
In this function, not one of the compressedByte arrays matches the svn reply.
public byte[] GetFile(string pstrPath)
{
AmazonS3Client s3Client = new AmazonS3Client("blah", "blah blah", Amazon.RegionEndpoint.whoknows);
GetObjectRequest gRequest = new GetObjectRequest();
gRequest.BucketName = "test";
gRequest.Key = "the file path and name";
string contents;
GetObjectResponse gresponse = s3Client.GetObjectAsync(gRequest).Result;
byte[] compressedBytes;
using (MemoryStream memoryStream = new MemoryStream())
{
gresponse.ResponseStream.CopyTo(memoryStream);
compressedBytes = memoryStream.ToArray();
}
byte[] compressedBytes1 = LZ4Codec.Wrap(Encoding.Default.GetBytes(contents));
byte[] compressedBytes2 = LZ4Codec.Wrap(Encoding.UTF8.GetBytes(contents));
byte[] compressedBytes3 = LZ4Codec.Wrap(Encoding.UTF7.GetBytes(contents));
byte[] compressedBytes4 = LZ4Codec.Wrap(Encoding.UTF32.GetBytes(contents));
byte[] compressedBytes5 = LZ4Codec.Wrap(Encoding.BigEndianUnicode.GetBytes(contents));
byte[] compressedBytes6 = LZ4Codec.Wrap(Encoding.ASCII.GetBytes(contents));
byte[] svnres = Channel.GetFile(pstrPath);
return Channel.GetFile(pstrPath);
}
Is there a way I can find out what encoding these files are using?
Ideally I will change the file upload to be this encoding but I cannot do that unless I work out what encoding svn is using.
Related
I have a .docx file located on my virtual machine desktop which I want to write down to a stream.
So far this is what I have tried
byte[] buffer = new byte[32768];
string path = #"\\officeblrhome.somedomain\UserData$\username\Desktop\filename.docx";
var memoryStr = new MemoryStream();
memoryStr.Write(System.IO.File.ReadAllBytytes(path), 0 , buffer.Length);
using (WordprocessingDocumenet doc = WordprocessingDocument.Open(memoryStr, true)
And I get the an error that the file contains corrupted data. Is it possible that my path is wrong? If it is how to get the valid path from the VM? The word file itself is not corrupted.
I made few changes to the code and seems that it's working
byte[] buffer = System.IO.File.ReadAllBytytes(path);
stream.Flush();
stream.Position = 0;
I am using AWS KMS Encrypt API to encrypt a text. The Encrypt API returns a EncryptResponse object that has a MemoryStream attribute which is the encrypted value.
I am using the AWS .Net SDK and trying to save this value to DynamoDB.
How can I save this value? I tried converting this to a String but while the conversion saving DynamoDB and retrieval gave me back the MemoryStream object, the resultant cipher text from the recreated MemoryStream object cannot be decrypted with an error "Invalid ciphertext."
Here is the code used to save and retrieve the string
// convert stream to string
StreamReader reader = new StreamReader(ciphertextBlob);
string text = reader.ReadToEnd();
// convert string to stream
byte[] byteArray = Encoding.UTF8.GetBytes(text);
MemoryStream stream = new MemoryStream(byteArray);
When retrieving the stream it was noticed that the ciphertext was different for the object before saving and that during retrieval.
The EncryptFunction
The EncryptResponse
This issue was solved by converting the memory stream object into a byte[] and saving it as a String in DynamoDB.
byte[] byteArray = ReadFully(ciphertextBlob);
string AccessKeycipherText = Convert.ToBase64String(byteArray);
The base64 encoding was used to get a String representation of the byte array.
The memory stream object was retrieved using the reverse.
byte[] temp_backToBytes = Convert.FromBase64String(AccessKey);
MemoryStream stream = new MemoryStream(temp_backToBytes);
If i read byte array from a file and write it using below code
byte[] bytes = File.ReadAllBytes(filePath);
File.WriteAllBytes(filePath, byteArr);
works perfectly fine.I can open and view the written file properly.
But if i read file contents into a string and then convert it to byte array using below function
string s = File.ReadAllText(filePath);
var byteArr = System.Text.Encoding.UTF8.GetBytes(s);
the size of byte array is more than the previous array read directly from file and the values are also different, hence if i write the file using this array the cannot be read when opened
Note:- File is utf-8 encoded
i found out that using below code
using (StreamReader reader = new StreamReader(filePath, Encoding.UTF8, true))
{
reader.Peek(); // you need this!
var encoding = reader.CurrentEncoding;
}
Unable to understand why both the array differs??
I was using the below attached image for converting and then writing
With
using (StreamReader reader = new StreamReader(filePath, Encoding.UTF8, true))
{
reader.Peek(); // you need this!
var encoding = reader.CurrentEncoding;
}
your var encoding will just echo the Encoding.UTF8 parameter. You are deceiving yourself there.
A binary file just has no text encoding.
Need to save a file may be anything an image or a text
Then just use ReadAllBytes/WriteAllBytes. A text file is always also a byte[], but not all file types are text. You would need Base64 encoding first and that just adds to the size.
The safest way to convert byte arrays to strings is indeed encoding it in something like base64.
Like:
string s= Convert.ToBase64String(bytes);
byte[] bytes = Convert.FromBase64String(s);
i have tried every possible solution that is given on website.
private byte[] GetBinaryFile()
{
Stream fs = FileUpload1.PostedFile.InputStream;
BinaryReader br = new BinaryReader(fs);
Byte[] bytes = br.ReadBytes((Int32)fs.Length);
return bytes;
}
This is what i was trying to Read from the fileuploader which didn't work. Then i changed the idea i uploaded the file on the server first and then i was trying to read but that again gave me the same error of system.byte(). Thing is it does not return the byte format of the pdf but it worked perfectly on my local system does it has anything to do with the server? Any help will be appreciated
you can try this:-
byte[] bytes = new byte[FileUpload1.PostedFile.ContentLength];
bytes = FileUpload1.FileBytes;
I have a scenario where by I want to zip an email attachment using SharpZipLib. Then the end user will open the attachment and will unzip the attached file.
Will the file originally zipped file using SharpZipLib be easily unzipped by other programs for my end user?
It depends on how you use SharpZipLib. There is more than one way to compress the data with this library.
Here is example of method that will create a zip file that you will be able to open in pretty much any zip aware application:
private static byte[] CreateZip(byte[] fileBytes, string fileName)
{
using (var memoryStream = new MemoryStream())
using (var zipStream = new ZipOutputStream(memoryStream))
{
var crc = new Crc32();
crc.Reset();
crc.Update(fileBytes);
var zipEntry =
new ZipEntry(fileName)
{
Crc = crc.Value,
DateTime = DateTime.Now,
Size = fileBytes.Length
};
zipStream.PutNextEntry(zipEntry);
zipStream.Write(fileBytes, 0, fileBytes.Length);
zipStream.Finish();
zipStream.Close();
return memoryStream.ToArray();
}
}
Usage:
var fileBytes = File.ReadAllBytes(#"C:/1.xml");
var zipBytes = CreateZip(fileBytes, "MyFile.xml");
File.WriteAllBytes(#"C:/2.zip", zipBytes);
This CreateZip method is optimized for the cases when you already have bytes in memory and you just want to compress them and send without even saving to disk.