I have a website where photographers can upload photos. The site is being hosted on an shared azure web app and The photos and thumbnails of the photos are uploaded to Azure Blob storage and a record is written to the db. Photographers can upload potentially up to 700mb of photos at a time.
My problem
I had a synchronous method for the upload that A) took ages to run and B) failed with the "There is not enough space on disk" error message. I'm guessing this is because the temp folder for a Azure shared web app is restricted to 200mb.
I tried to implement a Asynchronous method to help speed up the upload but it completes the first photo successfully (ie blobs and db records exist) and then It appears to just hang. This is my first attempt at writing an asynchronous method.
I also don't know how to fix the temp folder size issue.
My calling method
public static async Task<Tuple<bool, string>> UploadAsync(HttpPostedFileBase[] photos, Bookings booking, string photoType, ApplicationUser user)
{
// For each file to be uploaded
foreach (HttpPostedFileBase file in photos)
{
try
{
await UploadPhotoFromFileAsync(file, user, booking.BookingsId, photoType);
}
catch (Exception ex)
{
// Not Implemented
}
}
return new Tuple<bool, string>(true, "Photos uploaded successfully");
}
My Photo Upload method
public static Task UploadPhotoFromFileAsync(HttpPostedFileBase file, ApplicationUser user, int bookingId, string photoType)
{
return Task.Run(() =>
{
using (ApplicationDbContext dbt = new ApplicationDbContext())
{
Bookings booking = dbt.Bookings.Find(bookingId);
// Craete a new record in the UserFiles table
Photos photo = new Photos();
photo.BookingsId = booking.BookingsId;
photo.PhotoType = photoType;
photo.FileName = Path.GetFileName(file.FileName);
string confirmedDate = string.Empty;
if (booking.ConfirmedDate.HasValue)
{
DateTime actualConfirmedDate = booking.ConfirmedDate.Value;
confirmedDate = actualConfirmedDate.Year.ToString() + actualConfirmedDate.Month.ToString() + actualConfirmedDate.Day.ToString();
}
string blobName = string.Empty;
string blobThumbName = string.Empty;
if (photoType == "SamplePhoto")
{
// Get the count of the sample photos in the gallery
List<Photos> samplePhotos = dbt.Photos.Where(m => m.BookingsId == booking.BookingsId && m.PhotoType == "SamplePhoto").ToList();
blobName = "TS_" + booking.location.Location.Replace(" ", "") + "_" + booking.BookingsId.ToString() + "_" + confirmedDate + "_" + (samplePhotos.Count).ToString() + "_sample" + Path.GetExtension(file.FileName);
blobThumbName = "TS_" + booking.location.Location.Replace(" ", "") + "_" + booking.BookingsId.ToString() + "_" + confirmedDate + "_" + (samplePhotos.Count).ToString() + "_sample_thumb" + Path.GetExtension(file.FileName);
}
else
{
// Get the count of the sample photos in the gallery
List<Photos> photos = dbt.Photos.Where(m => m.BookingsId == booking.BookingsId && m.PhotoType == "GalleryPhoto").ToList();
blobName = "TS_" + booking.location.Location.Replace(" ", "") + "_" + booking.BookingsId.ToString() + "_" + confirmedDate + "_" + (photos.Count).ToString() + Path.GetExtension(file.FileName);
blobThumbName = "TS_" + booking.location.Location.Replace(" ", "") + "_" + booking.BookingsId.ToString() + "_" + confirmedDate + "_" + (photos.Count).ToString() + "_thumb" + Path.GetExtension(file.FileName);
}
// Create the Thumbnail image.
CloudBlobContainer thumbnailBlobContainer = _blobStorageService.GetCloudBlobContainer("thumbnails");
if (CreateThumbnailImageFromHttpPostedFileBase(file, blobThumbName, photo))
{
photo.ThumbnailBlobName = blobThumbName;
photo.ThumbnailBlobUrl = thumbnailBlobContainer.Uri + "/" + blobThumbName;
}
CloudBlobContainer blobContainer = _blobStorageService.GetCloudBlobContainer("photos");
photo.BlobName = blobName;
photo.BlobUrl = blobContainer.Uri + "/" + blobName;
photo.DateCreated = DateTime.Now;
photo.CreatedBy = user.Id;
dbt.Photos.Add(photo);
dbt.SaveChanges();
booking.Photos.Add(photo);
dbt.SaveChanges();
//Upload to Azure Blob Storage
CloudBlockBlob blob = blobContainer.GetBlockBlobReference(blobName);
blob.UploadFromStream(file.InputStream);
}
});
}
The Azure Storage library includes Async-methods and you should take advantage of those. And instead of wrapping your controller method with Task.Run, you can use ASP.NET MVC's built-in async-capabilities.
So, first, make the controller's method async:
public async Task<ContentResult> UploadPhotoFromFileAsync...
Then remove all the Task.Runs.
Lastly, call Azure Storage Library's async-methods:
var blob = blobContainer.GetBlockBlobReference(blobName);
await blob.UploadFromStreamAsync(file.InputStream);
The problem was actually being caused by the request length set in the Web.Config. It wasn't high enough to allow for the size of the photos being uploaded. I simply added the following code to the Web.Config.
<system.web>
<httpRuntime targetFramework="4.5" maxRequestLength="1048576" executionTimeout="900" />
</system.web>
<system.webServer>
<modules>
<remove name="FormsAuthentication" />
</modules>
</system.webServer>
Related
Hello I have a method in my controller that looks like this.
[HttpPost]
[ValidateAntiForgeryToken]
public ActionResult UploadImage(int? id)
{
if (id == null)
return HttpNotFound();
Component c = db.Components.Find((int)id);
HttpPostedFileBase photo = Request.Files["image"];
if (photo != null && photo.ContentLength > 0)
{
var filename = IGT.imagePath + "\\Components\\" + id.ToString() + ".jpg";
photo.SaveAs(filename);
c.Image_Url = IGT.baseUrl + "/Content/images/Components/" + id.ToString() + ".jpg";
db.SaveChanges();
}
else
{
if (Request["imageurl"] != null && Request["imageurl"].Length > 0)
{
// download this file
WebClient wc = new WebClient();
wc.DownloadFile(Request["imageurl"], IGT.imagePath + "\\Components\\" + id.ToString() + ".jpg");
c.Image_Url = IGT.baseUrl + "/Content/images/Components/" + id.ToString() + ".jpg";
db.SaveChanges();
}
}
HttpPostedFileBase reference = Request.Files["referencefile"];
if (reference != null && reference.ContentLength > 0)
{
// Upload the origin file and create a URL
var filename = IGT.contentPath + "\\uploads\\Comp-" + id.ToString() + "-" + System.IO.Path.GetFileName(reference.FileName);
reference.SaveAs(filename);
c.Reference_Url = IGT.baseUrl + "/Content/uploads/Comp-" + id.ToString() + "-" + System.IO.Path.GetFileName(reference.FileName);
db.SaveChanges();
}
return RedirectToAction("Edit", new { id = id });
}
But currently when it gets to
photo.SaveAs(filename);
I receive the error message
System.IO.DirectoryNotFoundException: 'Could not find a part of the path 'C:\Users\chris\Source\Repos\inventory2.0\PIC_Program_1.0\Content\images\Components\498.jpg
How can I make a try catch block so that if the folder doesn't exist in IIS Express, it will create it?
You could use the below code to create directory programmatically:
if (!Directory.Exists(appDataPath)) {
Directory.CreateDirectory(appDataPath);
}
and use directory.SetAccessControl(security); method to set the permission to that folder.
please refer the below links for more detail:
https://learn.microsoft.com/en-us/dotnet/api/system.io.directory.createdirectory?view=netframework-4.8
C# Creating directory and setting the permissions
https://www.kunal-chowdhury.com/2016/02/folder-permission.html
Can you try this code:
if (!System.IO.Directory.Exists("your folder")) {
System.IO.Directory.CreateDirectory("Your Folder");
}
Also, make sure your IIS user has a read/write access to that folder directory.
I am working on an IOS App which I create with Unity in C# (Unity 2018.2.5f1). Now I'm trying to upload an image to Firebase Storage to get familiar with it. Unfortunately this doesn't work.
I assume there is a problem with the path to the local file. I get the following error in XCode when running the app.
Firebase.Storage.Storage.StorageException: There is no object at the desired reference.
The file swift.jpg definitively exists on the iPhone.
Your help is very welcome.
using UnityEngine;
using Firebase.Storage;
using System.Threading.Tasks;
public class Upload : MonoBehaviour {
FirebaseStorage storage;
StorageReference storage_ref;
// Use this for initialization
void Start () {
storage = FirebaseStorage.DefaultInstance;
storage_ref = storage.GetReferenceFromUrl("gs://[My Storage URL]");
// File located on disk
string local_file = Application.persistentDataPath + "/" + "swift.jpg";
// Create a reference to the file you want to upload
StorageReference image_ref = storage_ref.Child("images/swift.jpeg");
// Upload the file to the path
image_ref.PutFileAsync(local_file)
.ContinueWith((Task<StorageMetadata> task) => {
if (task.IsFaulted || task.IsCanceled) {
Debug.Log(task.Exception.ToString());
// Uh-oh, an error occurred!
}
else {
// Metadata contains file metadata such as size, content-type, and download URL.
// StorageMetadata metadata = task.Result;
// string download_url = metadata.DownloadUrl.ToString();
// Debug.Log("Finished uploading...");
// Debug.Log("download url = " + download_url);
}
});
}
}
Try
string local_file = "file:///" + Application.persistentDataPath + "/" + "swift.jpg";
Or
string local_file = "file://" + Application.persistentDataPath + "/" + "swift.jpg";
Or
string local_file = "file:\\" + Application.persistentDataPath + "/" + "swift.jpg";
Hope one of these will work.
I am looking to allow a person to to export journal entries into a text file. I can create a file with all the data but rather strictly saving the file somewhere specific I want to allow a user to download and save the file where they want on their computer. How to I force a download of a file after I create it with StreamWriter. I currently have the following code:
string fileName = "Journal.txt";
using (StreamWriter journalExport = new StreamWriter(fileName))
{
foreach (JournalEntryView entry in journalEnteries)
{
//write each journal entery to file/document
journalExport.WriteLine(entry.timestamp + " - " + entry.author + " (" + entry.authorRole + ")");
journalExport.WriteLine(entry.text);
journalExport.WriteLine("");
journalExport.WriteLine("");
}
}
I am also trying to put this into an ActionResult and return the file.
EDIT:
The following code is my new current code and the direction I am looking to go in, but when I use an ActionLink to call this method, i just get redirected to a new page rather than downloading the file.
string fileName = "Journal.txt";
string filepath = ConfigurationManager.AppSettings["DocumentRoot"] + "\\" + id + "\\" + fileName;
using (StreamWriter journalExport = new StreamWriter(filepath))
{
foreach (JournalEntryView entry in journalEnteries)
{
//write each journal entery to file/document
journalExport.WriteLine(entry.timestamp + " - " + entry.author + " (" + entry.authorRole + ")");
journalExport.WriteLine(entry.text);
journalExport.WriteLine("");
journalExport.WriteLine("");
}
}
byte[] fileData = System.IO.File.ReadAllBytes(filepath);
string contentType = MimeMapping.GetMimeMapping(filepath);
var cd = new System.Net.Mime.ContentDisposition
{
FileName = fileName,
Inline = true,
};
Response.AppendHeader("Content-Disposition", cd.ToString());
return File(fileData, contentType);
This might be what you are looking for:
public ActionResult GetFile()
{
...processing stuff...
return File("/files/file.pdf", "application/pdf");
//or
return File("/files/file.pdf", "application/force-download", "donwloadname.pdf");
}
I have been looking for some time now and have not been able to find this. How can I set my program up to write or update a file from multiple users but only one group is allowed to open the read what is in the folder?
class Log_File
{
string LogFileDirectory = #"\\server\boiseit$\TechDocs\Headset Tracker\Weekly Charges\Log\Log Files";
string PathToXMLFile = #"\\server\boiseit$\scripts\Mikes Projects\Headset-tracker\config\Config.xml";
string AdditionToLogFile = #"\Who.Did.It_" + DateTime.Now.Month + "-" + DateTime.Now.Day + "-" + DateTime.Now.Year + ".txt";
XML XMLFile = new XML();
public void ConfigCheck()
{
if (!File.Exists(PathToXMLFile))
{
XMLFile.writeToXML(PathToXMLFile, LogFileDirectory + AdditionToLogFile);
}
}
public void CreateLogFile()
{
if (Directory.GetFiles(LogFileDirectory).Count() == 0)
{
XMLFile.writeToXML(PathToXMLFile, LogFileDirectory + AdditionToLogFile);
CreateFileOrAppend("");
}
else if (!File.Exists(XMLFile.readingXML(PathToXMLFile)))
{
XMLFile.writeToXML(PathToXMLFile, LogFileDirectory + AdditionToLogFile);
CreateFileOrAppend("");
}
else
{
FileInfo dateOfLastLogFile = new FileInfo(XMLFile.readingXML(PathToXMLFile));
DateTime dateOfCreation = dateOfLastLogFile.CreationTime;
if (dateOfLastLogFile.CreationTime <= DateTime.Now.AddMonths(-1))
{
XMLFile.writeToXML(PathToXMLFile, LogFileDirectory + AdditionToLogFile);
CreateFileOrAppend("");
}
}
}
public void CreateFileOrAppend(string whoDidIt)
{
using (IsolatedStorageFile storage = IsolatedStorageFile.GetStore((IsolatedStorageScope.Domain | IsolatedStorageScope.Assembly | IsolatedStorageScope.User), null, null))
{
using (StreamWriter myWriter = new StreamWriter(XMLFile.readingXML(PathToXMLFile), true))
{
if (whoDidIt == "")
{
}
else
{
myWriter.WriteLine(whoDidIt);
}
}
}
}
This is my path where it needs to go. I have the special permission to open and write to the folder but my co workers do not. I am not allow to let them have this permission.
If I where to set up a database how would i change this code
LoggedFile.CreateFileOrAppend(Environment.UserName.ToUpper() + "-" + Environment.NewLine + "Replacement Headset To: " + AgentName + Environment.NewLine + "Old Headset Number: " + myDatabase.oldNumber + Environment.NewLine + "New Headset Number: " + HSNumber + Environment.NewLine + "Date: " + DateTime.Now.ToShortDateString() + Environment.NewLine);
I need it to pull current user, the agents name that is being affected the old headset and the new headset, and the time it took place.
While you create file, you have to set access rules to achieve your requirements. .
File.SetAccessControl(string,FileSecurity)
The below link has example
https://msdn.microsoft.com/en-us/library/system.io.file.setaccesscontrol(v=vs.110).aspx
Also the "FileSecurity" class object, which is an input parameter, has functions to set access rules, including group level control.
Below link has example
https://msdn.microsoft.com/en-us/library/system.security.accesscontrol.filesecurity(v=vs.110).aspx
This question will be opened under a new question since I am going to take a different route for recording the data I need Thank you all for the help
I am using the HTML5 canvas element and the new HTML5 file i\o function to drop multiple files on it and have them upload. It works fine, but now I need to generate a new filename if no files are in the destination directory (It's a 7 digit integer) or get the name of the last uploaded file, convert it to int32 and increment that by one for every new file being uploaded to the same directory. This is where the GetFileName(dir); comes in. The first image always uploads fine but the problem begins once the second file is saved and the process hits ImageJob.Build(), I presume this is because once the new file is starting to write, the GetFile() method runs for second file in line simultaneously and is checking for last written file, which is still being written and this creates the conflict. How can I fix this, maybe I can somehow itterate with a foreach over the Request.InputStream data or implement some kind process watch that waits for the process to finish?
Update: I tried using TempData to store the generated filename, and just increment on the int value in TempData for all the next file names and it appears to do better, gets more images in but still errors at some point. But TempData is not for that as it gets erased after each read, reassigning to it again does not help. Maybe I'll try storing it in session.
The process cannot access the file 'C:\Users\Admin\Documents\Visual Studio
2010\Projects\myproj\myproj\Content\photoAlbums\59\31\9337822.jpg'
because it is being used by another process.
public PartialViewResult Upload()
{
string fileName = Request.Headers["filename"];
string catid = Request.Headers["catid"];
string pageid = Request.Headers["pageid"];
string albumname = Request.Headers["albumname"];
var dir = "~/Content/photoAlbums/" + catid + "/" + pageid + "/" + (albumname ?? null);
var noex = GetFileName(dir);
var extension = ".jpg";
string thumbFile = noex + "_t" + extension;
fileName = noex + extension;
byte[] file = new byte[Request.ContentLength];
Request.InputStream.Read(file, 0, Request.ContentLength);
string imgdir;
string thumbimgdir;
string imageurl;
if (albumname != null)
{
imgdir = Server.MapPath("~/Content/photoAlbums/" + catid + "/" + pageid + "/" + albumname + "/" + fileName);
thumbimgdir = Server.MapPath("~/Content/photoAlbums/" + catid + "/" + pageid + "/" + albumname + "/" + thumbFile);
imageurl = "/Content/photoAlbums/" + catid + "/" + pageid + "/" + albumname + "/" + thumbFile;
}
else
{
imgdir = Server.MapPath("~/Content/photoAlbums/" + catid + "/" + pageid + "/" + fileName);
thumbimgdir = Server.MapPath("~/Content/photoAlbums/" + catid + "/" + pageid + "/" + thumbFile);
imageurl = "/Content/photoAlbums/" + catid + "/" + pageid + "/" + thumbFile;
}
ImageJob b = new ImageJob(file, imgdir, new ResizeSettings("maxwidth=1024&maxheight=768&format=jpg")); b.CreateParentDirectory = true; b.Build();
ImageJob a = new ImageJob(file, thumbimgdir, new ResizeSettings("w=100&h=100&mode=crop&format=jpg")); a.CreateParentDirectory = true; a.Build();
ViewBag.CatID = catid;
ViewBag.PageID = pageid;
ViewBag.FileName = fileName;
return PartialView("AlbumImage", imageurl);
}
public string GetFileName(string dir)
{
var FullPath = Server.MapPath(dir);
var dinfo = new DirectoryInfo(FullPath);
string FileName;
if (dinfo.Exists)
{
var Filex = dinfo.EnumerateFiles().OrderBy(x => x.Name).LastOrDefault();
FileName = Filex != null ? Path.GetFileNameWithoutExtension(Filex.Name) : null;
if (FileName != null)
{
FileName = FileName.Contains("_t") ? FileName.Substring(0, FileName.Length - 2) : FileName;
int fnum;
Int32.TryParse(FileName, out fnum);
FileName = (fnum + 1).ToString();
if (fnum > 999999) { return FileName; } //Check that TryParse produced valid int
else
{
var random = new Random();
FileName = random.Next(1000000, 9999000).ToString();
}
}
else
{
var random = new Random();
FileName = random.Next(1000000, 9999000).ToString();
}
}
else
{
var random = new Random();
FileName = random.Next(1000000, 9999000).ToString();
}
return FileName;
}
You simply cannot use the Random class if you want to generate unique filenames. It uses the current time as the seed, so two exactly concurrent requests will always produce the same 'random' number.
You could use a cryptographic random number generator,
but you would still have to ensure that (a) only one thread would generate it at a time, and (b) you used a sufficiently long identifier to prevent the Birthday paradox.
Thus, I suggest that everyone use GUID identifiers for their uploads, as they solve all of the above issues inherently (I believe an OS-level lock is used to prevent duplicates).
Your method also doesn't handle multiple file uploads per-request, although that may be intentional. You can support those by looping through Request.Files and passing each HttpPostedFile instance directly into the ImageJob.
Here's a simplified version of your code that uses GUIDs and won't encounter concurrency issues.
public PartialViewResult Upload()
{
string albumname = Request.Headers["albumname"];
string baseDir = "~/Content/photoAlbums/" + Request.Headers["catid"] + "/" + Request.Headers["pageid"] + "/" (albumname != null ? albumname + "/" : "");
byte[] file = new byte[Request.ContentLength];
Request.InputStream.Read(file, 0, Request.ContentLength);
ImageJob b = new ImageJob(file, baseDir + "<guid>.<ext>", new ResizeSettings("maxwidth=1024&maxheight=768&format=jpg")); b.CreateParentDirectory = true; b.Build();
ImageJob a = new ImageJob(file, baseDir + "<guid>_t.<ext>", new ResizeSettings("w=100&h=100&mode=crop&format=jpg")); a.CreateParentDirectory = true; a.Build();
//Want both the have the same GUID? Pull it from the previous job.
//string ext = PathUtils.GetExtension(b.FinalPath);
//ImageJob a = new ImageJob(file, PathUtils.RemoveExtension(a.FinalPath) + "_t." + ext, new ResizeSettings("w=100&h=100&mode=crop&format=jpg")); a.CreateParentDirectory = true; a.Build();
ViewBag.CatID = Request.Headers["catid"];
ViewBag.PageID = Request.Headers["pageid"];
ViewBag.FileName = Request.Headers["filename"];
return PartialView("AlbumImage", PathUtils.GuessVirtualPath(a.FinalPath));
}
If the process is relatively quick (small files) you could go in a loop, check for that exception, sleep the thread for a couple of seconds, and try again (up to a maximum number of iterations). One caveat is that if the upload is asynchronous you might miss a file.
A couple of other suggestions:
Make the GetFileName to be a private method so that it doesn't get triggered from the web.
The OrderBy in the Filex query might not do what you expect once the it goes to 8 digits (possible if the first Random() is a very high number).
The Random() should probably be seeded to produce better randomness.