I have a file input control.
<input type="file" name="file" id="SaveFileToDB"/>
Lets say I browse to C:/Instruction.pdf document and click on submit. On Submit, I want to save the document in RavenDB and also later retrieve it for download purposes. I saw this link http://ravendb.net/docs/client-api/attachments
that says.. do this..
Stream data = new MemoryStream(new byte[] { 1, 2, 3 });
documentStore.DatabaseCommands.PutAttachment("videos/2", null, data,
new RavenJObject {{"Description", "Kids play in the garden"}});
I am not following what 1,2,3 mean here and what it means to say videos/2 in the command... how I can use these two lines to use it in my case.. to save word/pdfs in ravendb.. if any one has done such thing before, please advise.
I am not clear on one thing.. how the attachment is stored. If I want to store the attachment itself (say pdf) it is stored independently in ravendb.. and I just store the key of the attachment in the main document that it is associated with? If that is so, where is the pdf stored physically in ravendb? can I see it?
The 1,2,3 is just example data. What it is trying to get across is that you create a memory stream of whatever you want then use that memory stream in the PutAttachment method. Below is ad-hoc and not tested but should work:
using (var mem = new MemoryStream(file.InputStream)
{
_documentStore.DatabaseCommands.PutAttachment("upload/" + YourUID, null, mem,
new RavenJObject
{
{ "OtherData", "Can Go here" },
{ "MoreData", "Here" }
});
}
Edited for the rest of the questions
How is attachment stored? I believe it is a json document with one property holding the byte array of the attachment
Is the "document" stored independently? Yes. An attachment is a special document that is not indexed but it is part of the database so that tasks like replication work.
"Should I" store the key of the attachment in the main document that it is associated with? Yes you would reference the Key and anytime you want to get that you would just ask Raven for the attachment with that id.
Is the pdf stored physically in ravendb? Yes.
Can you see it? No. It does even show up in the studio (at least as far as I know)
Edit Corrected and Updated Sample
[AcceptVerbs(HttpVerbs.Post)]
public ActionResult Upload(HttpPostedFileBase file)
{
byte[] bytes = ReadToEnd(file.InputStream);
var id = "upload/" + DateTime.Now.Second.ToString(CultureInfo.InvariantCulture);
using (var mem = new MemoryStream(bytes))
{
DocumentStore.DatabaseCommands.PutAttachment(id, null, mem,
new RavenJObject
{
{"OtherData", "Can Go here"},
{"MoreData", "Here"},
{"ContentType", file.ContentType}
});
}
return Content(id);
}
public FileContentResult GetFile(string id)
{
var attachment = DocumentStore.DatabaseCommands.GetAttachment("upload/" + id);
return new FileContentResult(ReadFully(attachment.Data()), attachment.Metadata["ContentType"].ToString());
}
public static byte[] ReadToEnd(Stream stream)
{
long originalPosition = 0;
if (stream.CanSeek)
{
originalPosition = stream.Position;
stream.Position = 0;
}
try
{
var readBuffer = new byte[4096];
int totalBytesRead = 0;
int bytesRead;
while ((bytesRead = stream.Read(readBuffer, totalBytesRead, readBuffer.Length - totalBytesRead)) > 0)
{
totalBytesRead += bytesRead;
if (totalBytesRead == readBuffer.Length)
{
int nextByte = stream.ReadByte();
if (nextByte != -1)
{
var temp = new byte[readBuffer.Length*2];
Buffer.BlockCopy(readBuffer, 0, temp, 0, readBuffer.Length);
Buffer.SetByte(temp, totalBytesRead, (byte) nextByte);
readBuffer = temp;
totalBytesRead++;
}
}
}
byte[] buffer = readBuffer;
if (readBuffer.Length != totalBytesRead)
{
buffer = new byte[totalBytesRead];
Buffer.BlockCopy(readBuffer, 0, buffer, 0, totalBytesRead);
}
return buffer;
}
finally
{
if (stream.CanSeek)
{
stream.Position = originalPosition;
}
}
}
public static byte[] ReadFully(Stream input)
{
byte[] buffer = new byte[16 * 1024];
using (MemoryStream ms = new MemoryStream())
{
int read;
while ((read = input.Read(buffer, 0, buffer.Length)) > 0)
{
ms.Write(buffer, 0, read);
}
return ms.ToArray();
}
}
How is attachment stored?
It is stored as binary data inside RavenDB. It is NOT stored as json.
Is the "document" stored independently?
There isn't a document here, you have some metadata that is associated with the attachment, it isn't a seaprate document.
"Should I" store the key of the attachment in the main document that it is associated with?
Yes, there is no way to query for that.
Is the pdf stored physically in ravendb?
Yes
Can you see it?
Only if you go to the attachment directly, such as http://localhost:8080/static/ATTACHMENT_KEY
It won't show in the UI
Related
I know that this is may be a question without one 'right' answer
I have a C# windows application that has an embedded resource included in the assembly. I've been trying to come up with a way to compare the contents of my resource stream to determine if the contents of that stream matches a particular file on the file system.
e.g.
using(var resourceStream = Assembly.GetExecutingAssembly().GetManifestResourceStream(#"Manifest/Resource/Path/thing.exe"))
using(var fileStream = new FileStream(#"File/System/Path/thing.exe", FileMode.Read))
// Compare Contents (thing.exe may be an older version)
if(CompareStreamContents(resourceStream, fileStream))
{
/* Do a thing */
}
else
{
/* Do another thing*/
}
Is there a better way than simply doing a byte-by-byte comparison? Thoughts? (and thanks in advance!)
Per my comment:
private bool CompareStreamContents(Stream resourceStream, Stream fileStream)
{
var sha = new SHA256CryptoServiceProvider();
var hash1 = Convert.ToBase64String(sha.ComputeHash(ReadToEnd(resourceStream)));
var hash2 = Convert.ToBase64String(sha.ComputeHash(ReadToEnd(fileStream)));
return hash1 == hash2;
}
private byte[] ReadToEnd(Stream stream)
{
var continueRead = true;
var buffer = new byte[0x10000];
var ms = new MemoryStream();
while (continueRead)
{
var size = stream.Read((byte[])buffer, 0, buffer.Length);
if (size > 0)
{
ms.Write(buffer, 0, size);
}
else
{
continueRead = false;
}
}
return ms.ToArray();
}
If you plan on doing something else with the streams after the compare, you may want to set the stream position back to origin before returning from the compare method.
I'm attempting to read an email attachment and I'm getting a "Memory Stream is not expandable" error. I researched this some and most of the solutions seemed related to determining the size of the buffer dynamically, but I'm already doing that. I'm not very experienced with memory streams, so I'd like to know WHY this is a problem. Thanks.
foreach (MailMessage m in messages)
{
byte[] myBuffer = null;
if (m.Attachments.Count > 0)
{
//myBuffer = new byte[25 * 1024]; old way
myBuffer = new byte[m.Attachments[0].ContentStream.Length];
int read;
while ((read = m.Attachments[0].ContentStream.Read(myBuffer, 0, myBuffer.Length)) > 0)
{
// error occurs on executing next statement
m.Attachments[0].ContentStream.Write(myBuffer, 0, read);
}
... more unrelated code ...
If you create a MemoryStream over a pre-allocated byte array, it can't expand (ie. get longer than the size you specified when you started). Instead, why not just use:
using (var ms = new MemoryStream())
{
// Do your thing, for example:
m.Attachments[0].ContentStream.CopyTo(ms);
return ms.ToArray(); // This gives you the byte array you want.
}
You need to replace the line
m.Attachments[0].ContentStream.Write(myBuffer, 0, read);
with a line that writes to a previously created MemoryStream, e.g.
foreach (MailMessage m in messages)
{
byte[] myBuffer = null;
if (m.Attachments.Count > 0)
{
//myBuffer = new byte[25 * 1024]; old way
myBuffer = new byte[m.Attachments[0].ContentStream.Length];
int read;
MemoryStream ms = new MemoryStream();
while ((read = m.Attachments[0].ContentStream.Read(myBuffer, 0, myBuffer.Length)) > 0)
{
ms.Write(myBuffer, 0, read);
}
I'm looking for a good way to get filesize from the Microsoft.SharePoint.Client.File object.
The Client object does not have a Length member.
I tried this:
foreach (SP.File file in files)
{
string path = file.Path;
path = path.Substring(this.getTeamSiteUrl().Length);
FileInformation fileInformation = SP.File.OpenBinaryDirect(this.Context, path);
using (MemoryStream memoryStream = new MemoryStream())
{
CopyStream(fileInformation.Stream, memoryStream);
file.Size = memoryStream.Length;
}
}
Which gave me a length through using the MemoryStream, but it's not good for performance. This file also does not belong to a document library. Since it's an attached file, I can't convert it to a ListItem object using ListItemAllFields. If I could convert it to a ListItem, I could get its size using: ListItem["File_x0020_Size"]
How do I get the filesize of the Client object in SharePoint using C#?
Load the File_x0020_Size field information to get it.
This is what I do when I want to list all the files in a Sharepoint 2010 folder:
//folderPath is something like /yoursite/yourlist/yourfolder
Microsoft.SharePoint.Client.Folder spFolder = _ctx.Web.GetFolderByServerRelativeUrl(folderPath);
_ctx.Load(spFolder);
_ctx.ExecuteQuery();
FileCollection fileCol = spFolder.Files;
_ctx.Load(fileCol);
_ctx.ExecuteQuery();
foreach (Microsoft.SharePoint.Client.File spFile in fileCol)
{
//In here, specify all the fields you want retrieved, including the file size one...
_ctx.Load(spFile, file => file.Author, file => file.TimeLastModified, file=>file.TimeCreated,
file => file.Name, file => file.ServerRelativeUrl, file => file.ListItemAllFields["File_x0020_Size"]);
_ctx.ExecuteQuery();
int fileSize = int.Parse((string)spFile.ListItemAllFields["File_x0020_Size"]);
}
_ctx is obviously the ClientContext you have initiated.
Here's an extended list of all Sharepoint internal fields
Can't you just use the Stream property's length?
file.Size = fileInformation.Stream.Length;
I don't know if this question was ever solved, but for the people who are looking for an answer (like I did) :
... CODE FOR GETTING THE SP.FILE ...
SP.FileInformation fileInfo = SP.File.OpenBinaryDirect(ctx, mySPFile.ServerRelativeUrl);
byte[] bodyString = ReadToEnd(fileInfo.Stream);
int length = bodyString.Length;
Console.Write(length.ToString());
... DO THE OTHER STUFF YOU NEED TO DO ....
public static byte[] ReadToEnd(System.IO.Stream stream)
{
long originalPosition = 0;
if (stream.CanSeek)
{
originalPosition = stream.Position;
stream.Position = 0;
}
try
{
byte[] readBuffer = new byte[4096];
int totalBytesRead = 0;
int bytesRead;
while ((bytesRead = stream.Read(readBuffer, totalBytesRead, readBuffer.Length - totalBytesRead)) > 0)
{
totalBytesRead += bytesRead;
if (totalBytesRead == readBuffer.Length)
{
int nextByte = stream.ReadByte();
if (nextByte != -1)
{
byte[] temp = new byte[readBuffer.Length * 2];
Buffer.BlockCopy(readBuffer, 0, temp, 0, readBuffer.Length);
Buffer.SetByte(temp, totalBytesRead, (byte)nextByte);
readBuffer = temp;
totalBytesRead++;
}
}
}
byte[] buffer = readBuffer;
if (readBuffer.Length != totalBytesRead)
{
buffer = new byte[totalBytesRead];
Buffer.BlockCopy(readBuffer, 0, buffer, 0, totalBytesRead);
}
return buffer;
}
finally
{
if (stream.CanSeek)
{
stream.Position = originalPosition;
}
}
}
Hope this will help other people, because I couldn't find a direct answer on the internet!
Original compressed data can be correctly inflated back. However, if I inflate data, deflate, and again inflate, resulting data are incorrect (e.g. simple data extraction, its modification and again compression - only now when testing no modification occurs, so I can test it).
Resulting data are somehow "damaged". The starting (about) 40 bytes are OK, and then "block" of incorrect data follows (remnants of original data are still there, but many bytes are missing).
Changing compression level doesn't help (except setting NO_COMPRESSION creates somehow incomplete stream).
Question is simple: why is that happening?
using ICSharpCode.SharpZipLib.Zip.Compression;
public byte[] Inflate(byte[] inputData)
{
Inflater inflater = new Inflater(false);
using (var inputStream = new MemoryStream(inputData))
using (var ms = new MemoryStream())
{
var inputBuffer = new byte[4096];
var outputBuffer = new byte[4096];
while (inputStream.Position < inputData.Length)
{
var read = inputStream.Read(inputBuffer, 0, inputBuffer.Length);
inflater.SetInput(inputBuffer, 0, read);
while (inflater.IsNeedingInput == false)
{
var written = inflater.Inflate(outputBuffer, 0, outputBuffer.Length);
if (written == 0)
break;
ms.Write(outputBuffer, 0, written);
}
if (inflater.IsFinished == true)
break;
}
inflater.Reset();
return ms.ToArray();
}
}
public byte[] Deflate(byte[] inputData)
{
Deflater deflater = new Deflater(Deflater.BEST_SPEED, false);
deflater.SetInput(inputData);
deflater.Finish();
using (var ms = new MemoryStream())
{
var outputBuffer = new byte[65536 * 4];
while (deflater.IsNeedingInput == false)
{
var read = deflater.Deflate(outputBuffer);
ms.Write(outputBuffer, 0, read);
if (deflater.IsFinished == true)
break;
}
deflater.Reset();
return ms.ToArray();
}
}
Edit: My bad, by mistake I rewrote first several bytes of the original compressed data. This isn't SharpZipLib fault, but mine.
I know this is a tangential answer, but the exact same thing happened to me, I abandoned SharpZipLib and went to DotNetZip :
http://dotnetzip.codeplex.com/
Easier API, no corrupt or strange byte order files.
I am trying to check if a file is an image before I upload it to the image server.
I am doing it with the following function, which works exceptionally well:
static bool IsValidImage(Stream imageStream)
{
bool isValid = false;
try
{
// Read the image without validating image data
using (Image img = Image.FromStream(imageStream, false, false))
{
isValid = true;
}
}
catch
{
;
}
return isValid;
}
The problem is that when the below is called immediately afterwards, The line:
while ((bytesRead = request.FileByteStream.Read(buffer, 0, bufferSize)) > 0)
evalueates to zero and no bytes are read. I notice that when I remove the
IsValidImage function, bytes are read and the file is written. It seems
that bytes can only be read once? Any idea how to fix this?
using (FileStream outfile = new FileStream(filePath, FileMode.Create))
{
const int bufferSize = 65536; // 64K
int bytesRead = 0;
Byte[] buffer = new Byte[bufferSize];
while ((bytesRead = request.FileByteStream.Read(buffer, 0, bufferSize)) > 0)
{
outfile.Write(buffer, 0, bytesRead);
}
outfile.Close(); //necessary?
}
UPDATE: Thanks for your help Marc. I am new to stream manipulation and could use a little
more help here. I took a shot but may be mixing up the use of filestream and memorystream.
Would you mind taking a look? Thanks again.
using (FileStream outfile = new FileStream(filePath, FileMode.Create))
using (MemoryStream ms = new MemoryStream())
{
byte[] buffer = new byte[1024];
int bytesRead;
while ((bytesRead = request.FileByteStream.Read(buffer, 0, buffer.Length)) > 0)
{
ms.Write(buffer, 0, bytesRead);
}
// ms now has a seekable/rewindable copy of the data
// TODO: read ms the first time
// I replaced request.FileByteStream with ms but am unsure about
// the using statement in the IsValidImage function.
if (!IsValidImage(ms) == true)
{
ms.Close();
request.FileByteStream.Close();
return;
}
ms.Position = 0;
// TODO: read ms the second time
byte[] m_buffer = new byte[ms.Length];
while ((bytesRead = ms.Read(m_buffer, 0, (int)ms.Length)) > 0)
{
outfile.Write(m_buffer, 0, bytesRead);
}
}
static bool IsValidImage(MemoryStream imageStream)
{
bool isValid = false;
try
{
// Read the image without validating image data
using (Image img = Image.FromStream(imageStream, false, false))
{
isValid = true;
}
}
catch
{
;
}
return isValid;
}
As you read from any stream, the position increases. If you read a stream to the end (as is typical), and then try to read again, then it will return EOF.
For some streams, you can seek - set the Position to 0, for example. However, you should try to avoid relying on this as it is not available for many streams (especially when network IO is involved). You can query this ability via CanSeek, but it would be simpler to avoid this - partly as if you are branching based on this, you suddenly have twice as much code to maintain.
If you need the data twice, then the options depends on the size of the data. For small streams, buffer it in-memory, as either a byte[] or a MemoryStream. For larger streams (or if you don't know the size) then writing to a scratch file (and deleting afterwards) is a reasonable approach. You can open and read the file as many times (in series, not in parallel) as you like.
If you are happy the stream isn't too large (although maybe add a cap to prevent people uploading swap-files, etc):
using (MemoryStream ms = new MemoryStream()) {
byte[] buffer = new byte[1024];
int bytesRead;
while ((bytesRead = inputStream.Read(buffer, 0, buffer.Length)) > 0) {
ms.Write(buffer, 0, bytesRead);
}
// ms now has a seekable/rewindable copy of the data
// TODO: read ms the first time
ms.Position = 0;
// TODO: read ms the second time
}
Indeed Stream instances remember where the current "cursor" is. Some streams support "rewinding". The "CanSeek" property will then return true. In the case of a HTTP request Ithis won't work (CanSeek = false).
Isn't a MIME-type sent from the browser as well?
If you really want to keep your way of checking you'll have to go with Marc's proposition
In your update, you have a problem reading the stream a second time.
byte[] m_buffer = new byte[ms.Length];
while ((bytesRead = ms.Read(m_buffer, 0, (int)ms.Length)) > 0)
{
outfile.Write(m_buffer, 0, bytesRead);
}
The solution is simple:
byte[] m_buffer = ms.ToArray();
outfile.Write(m_buffer, 0, m_buffer.Length);
See also MemoryStream.ToArray
public static bool IsImagen(System.IO.Stream stream, String fileName)
{
try
{
using (Image img = Image.FromStream(stream, false, false))
{
if (fileName.ToLower().IndexOf(".jpg") > 0)
return true;
if (fileName.ToLower().IndexOf(".gif") > 0)
return true;
if (fileName.ToLower().IndexOf(".png") > 0)
return true;
}
}
catch (ArgumentException){}
return false;
}