Develop Reference

Unity Firebase Storage - Check if file exists Async

After some advise around calls to firebase storage primarily how (from a sync POV) I should be checking to see if a file exists.
To set the context I am reading in a state file as json using the DownloadCoroutine function below and passing this into a JsonTextReader, this is working fine as long as the file exists in the first place (which as a new user it will not).
So then I wrote the below checkIfFile exists function which also works in a standalone capacity (Grabs the URL to prove that this file does in fact exist). Once this function has completed I then set a bool (SaveFileExists) to say this is/is not an existing file then go create one dependent on the state.
Where my problem lies is the order in which these functions are executed, I need the check to happen before any other methods are executed, they are both called in the LoadScene function currently. What I think I need to do is make the check an Async method returning a task? If so how would this look and where should I be calling it from, I have tried this but I think it keeps locking the main thread.
So the state right now is that because that bool doesnt change in time, the download of the storage file doesnt happen and the Json is never read in and throws an error, at the end of the console output the checkfile URL is outputted, any help would be great ,thanks.
private IEnumerator DownloadCoroutine(string path)
{
var storage = FirebaseStorage.DefaultInstance;
var storageReference = storage.GetReference(path);
if (SaveFileExists == true)
{
var DownloadTask = storageReference.GetBytesAsync(long.MaxValue);
yield return new WaitUntil(predicate: () => DownloadTask.IsCompleted);
byte[] fileContents = DownloadTask.Result;
retrievedSaveFile = Encoding.Default.GetString(fileContents);
Debug.Log("Downloading the save file");
}
else
{
createNewSaveFile(path);
}
}
Check to see if the Json file exists
private void CheckIfFileExists(string path)
{
var storage = FirebaseStorage.DefaultInstance;
var storageReference = storage.GetReference(path);
storageReference.GetDownloadUrlAsync().ContinueWith(task => {
if (!task.IsFaulted && !task.IsCanceled) {
Debug.Log("Download URL: " + task.Result);
SaveFileExists = true;
}
else{
Debug.Log("file doesnt exist so we create one");
}
});
}
Load scene
public IEnumerator LoadLastScene()
{
var User = FirebaseAuth.DefaultInstance.CurrentUser;
Debug.Log("USERID IS " + User.UserId.ToString());
CheckIfFileExists("Saves://" + User.UserId.ToString() + "saveFile.json");
yield return DownloadCoroutine("Saves://" + User.UserId.ToString() +
"saveFile.json");
}

The things you have to do is that
User authentication
Everything must be inside an async task function
check if the file exists
await storageRef.Root.Child(fUser.UserId).Child("Data").GetDownloadUrlAsync().ContinueWith(async task2 =>
{
if (task2.IsFaulted || task2.IsCanceled)
{
Debug.Log("<color=Red>File Not Exists</color>");
}
else
{
Debug.Log("<color=green>File Exists</color>");
await DownloadData();
}
});

Error: 'The process cannot access the file because it is being used by another process' in Visual Studio c#

I am having some trouble with this error in Visual Studio:
An unhandled exception of type 'System.IO.IOException' occurred in mscorlib.dll
Additional information: The process cannot access the file 'C:\Users\aheij\Desktop\KinectOutput\swipe.txt' because it is being used by another process.
What I Have tried:
I have tried using these codes obtained from other solved StackOverflow questions similar to mine to try to solve the problem - but even that didn't seem to work?
Ive tried checking if the file is in use, but with no success.
I also run Visual Studio as administrator.
The file is not read-only.
The folder is not open, and the file is not being used in any other program when executing the code - at least not that I can see/know of.
The code:
So, to add some context to my code: I am writing some quick gesture detection code to the Kinect c# BodyBasics SDK v2 code freely available. This is a helper method that I added, that gets called in when a person is in view. If that person is executing the gesture, the method writes the frame time and gesture name to a text file.
Half the time, when the code does work, the gesture recognition works well. However, the other half of the time, the code breaks/stops because the writing to file bit causes the error.
Below is my code to see if the person is standing in the neutral position - its a bit waffly, so apologies about that. I have commented 'ERROR' where the error is (unsurprisingly):
private void Neutral_stance(VisualStyleElement.Tab.Body body, IReadOnlyDictionary<JointType, Joint> joints, IDictionary<JointType, Point> jointPoints, BodyFrame bf)
{
CameraSpacePoint left_hand = joints[JointType.HandLeft].Position;
CameraSpacePoint left_elbow = joints[JointType.ElbowLeft].Position;
CameraSpacePoint left_shoulder = joints[JointType.ShoulderLeft].Position;
CameraSpacePoint left_hip = joints[JointType.HipLeft].Position;
CameraSpacePoint right_hand = joints[JointType.HandRight].Position;
CameraSpacePoint right_elbow = joints[JointType.ElbowRight].Position;
CameraSpacePoint right_shoulder = joints[JointType.ShoulderRight].Position;
CameraSpacePoint right_hip = joints[JointType.HipRight].Position;
double vertical_error = 0.15;
double shoulderhand_xrange_l = Math.Abs(left_hand.X - left_shoulder.X);
double shoulderhand_xrange_r = Math.Abs(right_hand.X - right_shoulder.X);
if (bf != null)
{
TimeSpan frametime = bf.RelativeTime;
string path_p = #"C:\Users\aheij\Desktop\KinectOutput\Punch.txt"; //write to punch file
string path_s = #"C:\Users\aheij\Desktop\KinectOutput\swipe.txt"; //write to swipe file
if (left_hand.Y < left_elbow.Y)
{
if (right_hand.Y < right_elbow.Y)
{
if (shoulderhand_xrange_l < vertical_error)
{
if (shoulderhand_xrange_r < vertical_error)
{
Gesture_being_done.Text = " Neutral";
File.AppendAllText(path_p, frametime.ToString() + " Neutral" + Environment.NewLine); //ERROR
File.AppendAllText(path_s, frametime.ToString() + " Neutral" + Environment.NewLine); //ERROR
}
}
}
}
else
{
Gesture_being_done.Text = " Unknown";
File.AppendAllText(path_p, frametime.ToString() + " Unknown" + Environment.NewLine); //ERROR
File.AppendAllText(path_s, frametime.ToString() + " Unknown" + Environment.NewLine); //ERROR
}
}
}
Any solutions/ideas/suggestions to point me on the right track would be appreciated. I think that it would be good to use the 'using streamwriter' method as opposed to the method I am using here - but I am not sure how? Any help would be appreciated.
Additonal Info: Using Visual Studio 2015; Using windows 10.
Sidenote: I read somewhere that the Windows Search tool in Windows 10 can interfere and cause problems like this so I need to disable it?

As suggested to me I used the Filestream method & ensured the file was closed after use. But, even this still caused the same error.
Thus, I also got rid of having two file-writing actions in rapid succession of each other. I dont know if this is technically right or even true, but based off of this post here: link, my error could be coming up because I am trying to execute the second 'write to text file' line whilst the previous 'write to text file' line is still executing/writing to that same folder & location - hence the clash? Please someone, correct me if I am wrong.
Either way, this seems to have worked.
See below for my edited/corrected method:
private void Neutral_stance(Body body, IReadOnlyDictionary<JointType, Joint> joints, IDictionary<JointType, Point> jointPoints, BodyFrame bf)
{
//cameraspace point joint stuff here again (see original post for this bit leading up to the if statements.)
if (bf != null)
{
TimeSpan frametime = bf.RelativeTime;
string path_s = #"C:\Users\aheij\Desktop\KinectOutput\swipe.txt";
if (left_hand.Y < left_elbow.Y)
{
if (right_hand.Y < right_elbow.Y)
{
if (shoulderhand_xrange_l < vertical_error)
{
if (shoulderhand_xrange_r < vertical_error)
{
Gesture_being_done.Text = " Neutral";
FileStream fs_s = new FileStream(path_s, FileMode.Append); //swipe
byte[] bdatas = Encoding.Default.GetBytes(frametime.ToString() + " Neutral" + Environment.NewLine);
fs_s.Write(bdatas, 0, bdatas.Length);
fs_s.Close();
}
}
}
}
else
{
Gesture_being_done.Text = " Unknown";
FileStream fs_s = new FileStream(path_s, FileMode.Append);
byte[] bdatas = Encoding.Default.GetBytes(frametime.ToString() + " Unknown" + Environment.NewLine);
fs_s.Write(bdatas, 0, bdatas.Length);
fs_s.Close();
}
}
}
Do let me know if there is any way I can make this more elegant or anything else I should be aware of w.r.t this answer.
The code is based off of the code found here: FileStream Tutorial website

Why does my file sometimes disappear in the process of reading from it or writing to it?

I have an app that reads from text files to determine which reports should be generated. It works as it should most of the time, but once in awhile, the program deletes one of the text files it reads from/writes to. Then an exception is thrown ("Could not find file") and progress ceases.
Here is some pertinent code.
First, reading from the file:
List<String> delPerfRecords = ReadFileContents(DelPerfFile);
. . .
private static List<String> ReadFileContents(string fileName)
{
List<String> fileContents = new List<string>();
try
{
fileContents = File.ReadAllLines(fileName).ToList();
}
catch (Exception ex)
{
RoboReporterConstsAndUtils.HandleException(ex);
}
return fileContents;
}
Then, writing to the file -- it marks the record/line in that file as having been processed, so that the same report is not re-generated the next time the file is examined:
MarkAsProcessed(DelPerfFile, qrRecord);
. . .
private static void MarkAsProcessed(string fileToUpdate, string
qrRecord)
{
try
{
var fileContents = File.ReadAllLines(fileToUpdate).ToList();
for (int i = 0; i < fileContents.Count; i++)
{
if (fileContents[i] == qrRecord)
{
fileContents[i] = string.Format("{0}{1} {2}"
qrRecord, RoboReporterConstsAndUtils.COMPLETED_FLAG, DateTime.Now);
}
}
// Will this automatically overwrite the existing?
File.Delete(fileToUpdate);
File.WriteAllLines(fileToUpdate, fileContents);
}
catch (Exception ex)
{
RoboReporterConstsAndUtils.HandleException(ex);
}
}
So I do delete the file, but immediately replace it:
File.Delete(fileToUpdate);
File.WriteAllLines(fileToUpdate, fileContents);
The files being read have contents such as this:
Opas,20170110,20161127,20161231-COMPLETED 1/10/2017 12:33:27 AM
Opas,20170209,20170101,20170128-COMPLETED 2/9/2017 11:26:04 AM
Opas,20170309,20170129,20170225-COMPLETED
Opas,20170409,20170226,20170401
If "-COMPLETED" appears at the end of the record/row/line, it is ignored - will not be processed.
Also, if the second element (at index 1) is a date in the future, it will not be processed (yet).
So, for these examples shown above, the first three have already been done, and will be subsequently ignored. The fourth one will not be acted on until on or after April 9th, 2017 (at which time the data within the data range of the last two dates will be retrieved).
Why is the file sometimes deleted? What can I do to prevent it from ever happening?
If helpful, in more context, the logic is like so:
internal static string GenerateAndSaveDelPerfReports()
{
string allUnitsProcessed = String.Empty;
bool success = false;
try
{
List<String> delPerfRecords = ReadFileContents(DelPerfFile);
List<QueuedReports> qrList = new List<QueuedReports>();
foreach (string qrRecord in delPerfRecords)
{
var qr = ConvertCRVRecordToQueuedReport(qrRecord);
// Rows that have already been processed return null
if (null == qr) continue;
// If the report has not yet been run, and it is due, add i
to the list
if (qr.DateToGenerate <= DateTime.Today)
{
var unit = qr.Unit;
qrList.Add(qr);
MarkAsProcessed(DelPerfFile, qrRecord);
if (String.IsNullOrWhiteSpace(allUnitsProcessed))
{
allUnitsProcessed = unit;
}
else if (!allUnitsProcessed.Contains(unit))
{
allUnitsProcessed = allUnitsProcessed + " and "
unit;
}
}
}
foreach (QueuedReports qrs in qrList)
{
GenerateAndSaveDelPerfReport(qrs);
success = true;
}
}
catch
{
success = false;
}
if (success)
{
return String.Format("Delivery Performance report[s] generate
for {0} by RoboReporter2017", allUnitsProcessed);
}
return String.Empty;
}
How can I ironclad this code to prevent the files from being periodically trashed?
UPDATE
I can't really test this, because the problem occurs so infrequently, but I wonder if adding a "pause" between the File.Delete() and the File.WriteAllLines() would solve the problem?
UPDATE 2
I'm not absolutely sure what the answer to my question is, so I won't add this as an answer, but my guess is that the File.Delete() and File.WriteAllLines() were occurring too close together and so the delete was sometimes occurring on both the old and the new copy of the file.
If so, a pause between the two calls may have solved the problem 99.42% of the time, but from what I found here, it seems the File.Delete() is redundant/superfluous anyway, and so I tested with the File.Delete() commented out, and it worked fine; so, I'm just doing without that occasionally problematic call now. I expect that to solve the issue.

// Will this automatically overwrite the existing?
File.Delete(fileToUpdate);
File.WriteAllLines(fileToUpdate, fileContents);
I would simply add an extra parameter to WriteAllLines() (which could default to false) to tell the function to open the file in overwrite mode, and not call File.Delete() at all then.
Do you currently check the return value of the file open?
Update: ok, it looks like WriteAllLines() is a .Net Framework function and therefore cannot be changed, so I deleted this answer. However now this shows up in the comments, as a proposed solution on another forum:
"just use something like File.WriteAllText where if the file exists,
the data is just overwritten, if the file does not exist it will be
created."
And this was exactly what I meant (while thinking WriteAllLines() was a user defined function), because I've had similar problems in the past.
So, a solution like that could solve some tricky problems (instead of deleting/fast reopening, just overwriting the file) - also less work for the OS, and possibly less file/disk fragmentation.

C# Unity3D JSON Hanging

Firstly I must point out that I am very new to C#.
I am developing an application using Unity3D, and part of the application requires that I parse a JSON file stored on my server.
The problem that I am having is that sometimes everything works perfectly, and other times the app hangs on the downloading the JSON. I don't receive any errors, the script just never reaches 100% on the progress.
Here is my code:
public IEnumerator DownloadJSONFile(string url)
{
Debug.Log("JSON URL: "+ url);
mJsonInfo = new WWW(url);
yield return mJsonInfo;
mIsJSONRequested = true;
}
private void LoadJSONData(string jsonUrl)
{
Debug.LogWarning("LoadJSONData, url= "+jsonUrl);
if(!mIsJSONRequested){
StartCoroutine(DownloadJSONFile(jsonUrl));
} else {
if(mJsonInfo.progress >= 1)
{
if(mJsonInfo.error == null )
{
//** PARSE THE JSON HERE **//
}else
{
Debug.LogError("Error downloading JSON");
mIsLoadingData = false;
}
} else {
Debug.LogWarning("!! ### JSON DOWNLOADING: "+mJsonInfo.progress+"%");
if(mJsonInfo.error != null )
{
Debug.LogError("Error downloading JSON");
Debug.LogError("JSON Error:"+mJsonInfo.error);
mIsLoadingData = false;
}
}
}
}
Like I said, 50% of the time the JSON data gets loaded nearly instantly, and 50% of the time the progress never reaches 1. I never receive an error in form the mJsonInfo.error variable.
Any suggestions as to what I am doing wrong would be greatly appreciated!

You need to wait until the download is complete.
As stated in the documentation you need to:
var www = new WWW(...);
yield return www;
So you need to modify the return type of your method from void to IEnumerator.
private IEnumerator LoadJSONData(string jsonUrl)
{
Debug.LogWarning("LoadJSONData, url= "+jsonUrl);
if(!mIsJSONRequested)
{
// Gets the json book info from the url
mJsonInfo = new WWW(jsonUrl);
yield return mJsonInfo; //Wait for download to complete
mIsJSONRequested = true;
}
else
{
...
}
}

isDone is that you need,
WWW lWWW = new WWW(...)
if(lWWW.isDone)
then parse it

I found the problem, I thought I would post my solution for anyone else experiencing the same problem.
The solution in the end was the JSON file on the server. I was using PHP (CakePHP) to generate the JSON, when opening the PHP generated file via the browser the response time was instant, but from my mobile app for some reason it would hang. So I changed my server side code to actually create and updated an actual JSON file, and now everything works fine.

Search String takes a long time the first time only?

No shortage of search for string performance questions out there yet I still can not make heads or tails out of what the best approach is.
Long story short, I have committed to moving from 4NT to PowerShell. In leaving the 4NT I am going to miss the console super quick string searching utility that came with it called FFIND. I have decided to use my rudimentary C# programming skills to try an create my own utility to use in PowerShell that is just as quick.
So far search results on a string search in 100's of directories across a few 1000 files, some of which are quite large, are FFIND 2.4 seconds and my utility 4.4 seconds..... after I have ran mine at least once????
The first time I run them FFIND does it near the same time but mine takes over a minute? What is this? Loading of libraries? File indexing? Am I doing something wrong in my code? I do not mind waiting a little longer but the difference is extreme enough that if there is a better language or approach I would rather start down that path now before I get too invested.
Do I need to pick another language to write a string search that will be lighting fast
I have the need to use this utility to search through 1000 of files for strings in web code, C# code, and another propitiatory language that uses text files. I also need to be able to use this utility to find strings in very large log files, MB size.
class Program
{
public static int linecounter;
public static int filecounter;
static void Main(string[] args)
{
//
//INIT
//
filecounter = 0;
linecounter = 0;
string word;
// Read properties from application settings.
string filelocation = Properties.Settings.Default.FavOne;
// Set Args from console.
word = args[0];
//
//Recursive search for sub folders and files
//
string startDIR;
string filename;
startDIR = Environment.CurrentDirectory;
//startDIR = "c:\\SearchStringTestDIR\\";
filename = args[1];
DirSearch(startDIR, word, filename);
Console.WriteLine(filecounter + " " + "Files found");
Console.WriteLine(linecounter + " " + "Lines found");
Console.ReadKey();
}
static void DirSearch(string dir, string word, string filename)
{
string fileline;
string ColorOne = Properties.Settings.Default.ColorOne;
string ColorTwo = Properties.Settings.Default.ColorTwo;
ConsoleColor valuecolorone = (ConsoleColor)Enum.Parse(typeof(ConsoleColor), ColorOne);
ConsoleColor valuecolortwo = (ConsoleColor)Enum.Parse(typeof(ConsoleColor), ColorTwo);
try
{
foreach (string f in Directory.GetFiles(dir, filename))
{
StreamReader file = new StreamReader(f);
bool t = true;
int counter = 1;
while ((fileline = file.ReadLine()) != null)
{
if (fileline.Contains(word))
{
if (t)
{
t = false;
filecounter++;
Console.ForegroundColor = valuecolorone;
Console.WriteLine(" ");
Console.WriteLine(f);
Console.ForegroundColor = valuecolortwo;
}
linecounter++;
Console.WriteLine(counter.ToString() + ". " + fileline);
}
counter++;
}
file.Close();
file = null;
}
foreach (string d in Directory.GetDirectories(dir))
{
//Console.WriteLine(d);
DirSearch(d,word,filename);
}
}
catch (System.Exception ex)
{
Console.WriteLine(ex.Message);
}
}
}
}

If you want to speed up your code run a performance analysis and see what is taking the most time. I can almost guaruntee the longest step here will be
fileline.Contains(word)
This function is called on every line of the file, on every file. Naively searching for a word in a string can taken len(string) * len(word) comparisons.
You could code your own Contains method, that uses a faster string comparison algorithm. Google for "fast string exact matching". You could try using a regex and seeing if that gives you a performance enhancement. But I think the simplest optimization you can try is :
Don't read every line. Make a large string of all the content of the file.
StreamReader streamReader = new StreamReader(filePath, Encoding.UTF8);
string text = streamReader.ReadToEnd();
Run contains on this.
If you need all the matches in a file, then you need to use something like Regex.Matches(string,string).
After you have used regex to get all the matches for a single file, you can iterate over this match collection (if there are any matches). For each match, you can recover the line of the original file by writing a function that reads forward and backward from the match object index attribute, to where you find the '\n' character. Then output that string between those two newlines, to get your line.
This will be much faster, I guarantee it.
If you want to go even further, some things I've noticed are :
Remove the try catch statement from outside the loop. Only use it exactly where you need it. I would not use it at all.
Also make sure your system is running, ngen. Most setups usually have this, but sometimes ngen is not running. You can see the process in process explorer. Ngen generates a native image of the C# managed bytecode so the code does not have to be interpreted each time, but can be run natively. This speeds up C# a lot.
EDIT
Other points:
Why is there a difference between first and subsequent run times? Seems like caching. The OS could have cached the requests for the directories, for the files, for running and loading programs. Usually one sees speedups after a first run. Ngen could also be playing a part here, too, in generating the native image after compilation on the first run, then storing that in the native image cache.
In general, I find C# performance too variable for my liking. If the optimizations suggested are not satisfactory and you want more consistent performance results, try another language -- one that is not 'managed'. C is probably the best for your needs.