I really need some rubber ducking...
I have a file that is at least 2.3 GiB.
I am currently downloading this file to a temp directory.
But when the download is interrupted (connection error, or windows crash) I want the user to resume download where it stopped. And not download the whole file all over again.
The code works in the fact that it continues downloading the file, but I see that the download stream is starting from the beginning again. So that means that the file ends to be (2.3 GiB + the amount bytes that were downloaded previously), which ofc corrupts my file.
I used the following snippet to resume downloading, so I hoped the stream would resume, where it stopped
localStream.Seek(positionInFile, SeekOrigin.Begin);
Any ideas on what I am missing here?
Here is my code.
BlobContainerClient containerClient = new BlobContainerClient(connectionString, container);
var blobClient = containerClient.GetBlobClient(downloadFile);
fullOutputPath = createOutputFilePath(updateFileUri.OriginalString, outputFolder);
downloadFileInfo = new FileInfo(fullOutputPath);
var response = blobClient.Download(cts.Token);
contentLength = response.Value.ContentLength;
if (contentLength.HasValue && contentLength.Value > 0)
{
if (_fileSystemService.FileExists(fullOutputPath))
{
from = downloadFileInfo.Length;
to = contentLength;
if (from == to)
{
//file is already downloaded
//skip it
progress.Report(1);
return;
}
fileMode = FileMode.Open;
positionInFile = downloadFileInfo.Length;
}
using FileStream localStream = _fileSystemService.CreateFile(fullOutputPath, fileMode, FileAccess.Write);
localStream.Seek(positionInFile, SeekOrigin.Begin);
bytesDownloaded = positionInFile;
double dprog = ((double)bytesDownloaded / (double)(contentLength.Value + positionInFile));
do
{
bytesRead = await response.Value.Content.ReadAsync(buffer, 0, buffer.Length, cts.Token);
await localStream.WriteAsync(buffer, 0, bytesRead, cts.Token);
await localStream.FlushAsync();
bytesDownloaded += bytesRead;
dprog = ((double)bytesDownloaded / (double)(contentLength.Value + positionInFile));
progress.Report(dprog);
} while (bytesRead > 0);
}
I did some test for you, in my case, I use a .txt file to demo your requirement. You can see the .txt file here.
As you can see, at line 151, I made an end mark:
I also created a local file that ends with this end mark to emulate that download is interrupted and we will continue to download from storage:
This is my code for fast demo below:
static void Main(string[] args)
{
string containerName = "container name";
string blobName = ".txt file name";
string storageConnStr = "storage account conn str";
string localFilePath = #"local file path";
var localFileStream = new FileStream(localFilePath, FileMode.Append);
var localFileLength = new FileInfo(localFilePath).Length;
localFileStream.Seek(localFileLength, SeekOrigin.Begin);
var blobServiceClient = new BlobServiceClient(storageConnStr);
var blobClient = blobServiceClient.GetBlobContainerClient(containerName).GetBlobClient(blobName);
var stream = blobClient.Download(new Azure.HttpRange(localFileLength)).Value.Content;
var contentStrting = new StreamReader(stream).ReadToEnd();
Console.WriteLine(contentStrting);
localFileStream.Write(Encoding.ASCII.GetBytes(contentStrting));
localFileStream.Flush();
}
Result:
We only downloaded the content behind the end mark:
Content has been downloaded to local .txt file:
Pls let me know if you have any more questions.
Related
private async Task DoDownloadFile(ChatMessageListRefDataModel chatMessage)
{
var status = await Permissions.RequestAsync<Permissions.StorageWrite>();
if(status == PermissionStatus.Granted)
{
await HttpRequestHelper.DownloadFile(chatMessage.FileUrl, chatMessage.FileName);
}
}
public static async Task DownloadFile(string url, string fileName) {
var path = Path.Combine(Environment.GetFolderPath(Environment.SpecialFolder.Personal), fileName);
using (var downloadStream = await client.GetStreamAsync(url))
{
using (var memoryStream = new MemoryStream())
{
await downloadStream.CopyToAsync(memoryStream);
using(FileStream file = new FileStream(path, FileMode.OpenOrCreate, FileAccess.Write, FileShare.None))
{
byte[] bytes = new byte[memoryStream.Length];
memoryStream.Read(bytes, 0, (int)memoryStream.Length);
file.Write(bytes, 0, bytes.Length);
memoryStream.Close();
}
}
}
The code produces no error it is just the file was not found on the phone's directory. What could have gone wrong. Thanks.
I have created a new sample to test your code. And meet the same problem as yours. But the problem just appeared on the physical device. When I run it on the andorid emulator, I can find the file by the Android Studio's emulator device file explorer.
The path Path.Combine(Environment.GetFolderPath(Environment.SpecialFolder.Personal), fileName); seems be hidden. I can't find it by the device's file manager.
So you can try to use the following code to get the path, such as:
var filename1 = Android.App.Application.Context.GetExternalFilesDir("").AbsolutePath;
var filename = System.IO.Path.Combine(filename1, "xxx.txt");
using (System.IO.FileStream os = new System.IO.FileStream(filename, System.IO.FileMode.OpenOrCreate))
{
}
This file path can be found by the device's file manager. You can have a try.
Turns Out the Launcher.OpenAsync(fileUrl) is just okay to me.
How to get the path to the internal memory of the smartphone and the DCIM folder, then to save the photo there?
string directory = System.IO.Path.Combine(Android.OS.Environment.ExternalStorageDirectory.AbsolutePath, Android.OS.Environment.DirectoryDownloads);
FileStream file = new FileStream(System.IO.Path.Combine(directory, "newProdict_" + product.Width + "x" + product.Height + ".png"), FileMode.OpenOrCreate);
You can get the DCIM folder like that:
var dcimFolder = Android.OS.Environment.GetExternalStoragePublicDirectory(System.Environment.DirectoryDcim).Path;
For writing files you can use the nuget package PCLStorage so you already have everything in a cross platform manner:
var folder = await FileSystem.Current.GetFolderFromPathAsync(dcimFolder);
var file = await folder.CreateFileAsync("image.jpg", CreationCollisionOption.ReplaceExisting);
byte[] buffer = new byte[100];
using (System.IO.Stream stream = await file.OpenAsync(FileAccess.ReadAndWrite))
{
stream.Write(buffer, 0, 100);
}
I need to download files like ".Doc , .pdf , .xls , .Jpeg, .PNG etc" from the server and store into phone memory not to Isolated. I have search a lot but can not get any things for .doc , .pdf. I got a link Downloading and saving a file Async in Windows Phone 8
but can not work. So if any one can do this please let me know.
Thanks in advance.
I have done with FileSavePicker , here is code
public void DownloadFiles(Uri url)
{
var wc = new WebClient();
wc.OpenReadCompleted +=async (s, e) =>
{
Stream st = e.Result;
buf = ReadFully(st);
FileSavePicker savePicker = new FileSavePicker();
savePicker.SuggestedStartLocation = PickerLocationId.DocumentsLibrary;
// Dropdown of file types the user can save the file as
savePicker.FileTypeChoices.Add("PDF", new List<string>() { ".pdf" });
// Default file name if the user does not type one in or select a file to replace
savePicker.SuggestedFileName = "New Document";
savePicker.PickSaveFileAndContinue();
StorageFile SF = await KnownFolders.PicturesLibrary.CreateFileAsync
("Guide.pdf", CreationCollisionOption.ReplaceExisting);
var fs = await SF.OpenAsync(FileAccessMode.ReadWrite);
StorageStreamTransaction transaction = await SF.OpenTransactedWriteAsync();
DataWriter dataWriter = new DataWriter(transaction.Stream);
dataWriter.WriteBytes(buf);
transaction.Stream.Size = await dataWriter.StoreAsync(); // reset stream size to override the file
await transaction.CommitAsync();
};
wc.OpenReadAsync(url);
}
public static byte[] ReadFully(Stream input)
{
byte[] buffer = new byte[16 * 1024];
using (MemoryStream ms = new MemoryStream())
{
int read;
while ((read = input.Read(buffer, 0, buffer.Length)) > 0)
{
ms.Write(buffer, 0, read);
}
return ms.ToArray();
}
}
private async void ContinueFileOpenPicker(FileSavePickerContinuationEventArgs args)
{
StorageFile file = args.File;
if (file != null)
{
// Prevent updates to the remote version of the file until we finish making changes and call CompleteUpdatesAsync.
CachedFileManager.DeferUpdates(file);
// write to file
await FileIO.WriteBytesAsync(file, buf);
// Let Windows know that we're finished changing the file so the other app can update the remote version of the file.
// Completing updates may require Windows to ask for user input.
FileUpdateStatus status = await CachedFileManager.CompleteUpdatesAsync(file);
if (status == FileUpdateStatus.Complete)
{
Debug.WriteLine("File " + file.Name + " was saved.");
}
else
{
Debug.WriteLine("File " + file.Name + " couldn't be saved.");
}
}
else
{
Debug.WriteLine("Operation cancelled.");
}
await Windows.System.Launcher.LaunchFileAsync(file);
}
For More information Please go with this url How to continue your Windows Phone app after calling a file picker
I don't think you'll be able to directly download and store the files into the memory card as they have restricted access for security purposes. I guess Isolated Storage could be the option.
Reference
I am trying to load a .gz file out of a bucket.
Connection and authentication work finde, I even do get a file, but the problem is, the file is a lot bigger then the file should be. it is, original size, 155MB within the bucket but when it comes onto my hard disk it gets up to about 288MB
here is the function code:
public bool SaveBucketToFile(string Filename)
{
//Response check into file
using (StreamReader StRead = new StreamReader(_ObjResponse.ResponseStream))
{
string TempFile = Path.GetTempFileName();
StreamWriter StWrite = new StreamWriter(TempFile, false);
StWrite.Write(StRead.ReadToEnd());
StWrite.Close();
StRead.Close();
// Move to real destination
if (File.Exists(Filename))
{
File.Delete(Filename);
}
File.Move(TempFile, Filename);
}
return true;
}
the download and filling of _ObjResponse is made over usage of the AmazonS3 Client from their SDK. I am using a proxy but the same code on a different machine without proxy brings back the same result.
Any hints what to do here? the object request is simple:
_ObjRequest = new GetObjectRequest
{
BucketName = BucketName,
Key = Key
};
glad for any help...
for everyone to stumble upon this.
I needed to first save the stream via bufferedStream into a memorystream.
the code looks like this:
MemoryStream MemStream = new MemoryStream();
BufferedStream Stream2 = new BufferedStream(_ObjResponse.ResponseStream);
byte[] Buffer = new byte[0x2000];
int Count;
while ((Count = Stream2.Read(Buffer, 0, Buffer.Length)) > 0)
{
MemStream.Write(Buffer, 0, Count);
}
// Pfad auslesen
string TempFile = Path.GetTempFileName();
//Stream zum Tempfile öffnen
FileStream Newfile = new FileStream(TempFile,FileMode.Create);
//Stream wieder auf Position 0 ziehen
MemStream.Position = 0;
// in Tempdatei speichern
MemStream.CopyTo(Newfile);
Newfile.Close();
// Endgültigen Speicherpunkt prüfen und Tempdatei dorthin schieben
if (File.Exists(Filename))
{
File.Delete(Filename);
}
File.Move(TempFile, Filename);
I found this somewhere here:
http://www.codeproject.com/Articles/186132/Beginning-with-Amazon-S under the Caption "Get a file from Amazon S3"
I am using this code to write into my file:
private async void play_Click(object sender, RoutedEventArgs e)
{
String MyScore;
Double previousScore = 0;
StorageFolder local = Windows.Storage.ApplicationData.Current.LocalFolder;
var dataFolder1 = await local.CreateFolderAsync("MyFolder", CreationCollisionOption.OpenIfExists);
var file1 = await dataFolder1.CreateFileAsync("MyFile.txt", CreationCollisionOption.OpenIfExists);
var file = await dataFolder1.OpenStreamForReadAsync("MyFile.txt");
using (StreamReader streamReader = new StreamReader(file))
{
MyScore = streamReader.ReadToEnd();
}
if (MyScore != null && !MyScore.Equals(""))
{
previousScore = Convert.ToDouble(MyScore);
}
Double CurerentScore = 0;
Double Total = 0;
String scoreText = this.ScoreTB.Text;
CurerentScore = Convert.ToDouble(scoreText);
Total = previousScore - CurerentScore;
using (var s = await file1.OpenStreamForWriteAsync())
{
byte[] fileBytes = System.Text.Encoding.UTF8.GetBytes(Convert.ToString(Total));
s.Write(fileBytes, 0, fileBytes.Length);
}
}
But before writing into it, I want that my file should get cleared. What should I do?
This is what i have tried so far but the problem is that it writes the file up to the filebytes.length and due to that if the new information to be writed in file is less in terms of length in comparison to the privous length then some garbage value or unnecessay thing comes after the end of the new file
You can use this snippet :
var folder = ApplicationData.Current.LocalFolder;
// You are going to replace the file
var file = await folder.CreateFileAsync(fileName, CreationCollisionOption.ReplaceExisting);
using (var stream = await file.OpenStreamForWriteAsync())
{
var content = System.Text.Encoding.UTF8.GetBytes(Convert.ToString(Total));
await stream.WriteAsync(content, 0, content.Length);
}
To quote the documentation :
ReplaceExisting : Create the new file or folder with the desired name,
and replaces any file or folder that already exists with that name.
I have clear the file by writing a empty string to it and then i have written what i wanted in my file This solved my issue as nothing was there in the file so whatever i wanted to write to it came up successfully.
Simply use Stream.SetLength like this:
using (var s = await file1.OpenStreamForWriteAsync())
{
// Add this line
s.SetLength(0);
// Then write new bytes. use 's.SetLength(fileBytes.Length)' if needed.
byte[] fileBytes = System.Text.Encoding.UTF8.GetBytes(Convert.ToString(Total));
s.Write(fileBytes, 0, fileBytes.Length);
}