Memory usage of strings (or any other objects) in .Net - c#

I wrote this little test program:
using System;
namespace GCMemTest
{
class Program
{
static void Main(string[] args)
{
System.GC.Collect();
System.Diagnostics.Process pmCurrentProcess = System.Diagnostics.Process.GetCurrentProcess();
long startBytes = pmCurrentProcess.PrivateMemorySize64;
double kbStart = (double)(startBytes) / 1024.0;
System.Console.WriteLine("Currently using " + kbStart + "KB.");
{
int size = 2000000;
string[] strings = new string[size];
for(int i = 0; i < size; i++)
{
strings[i] = "blabla" + i;
}
}
System.GC.Collect();
pmCurrentProcess = System.Diagnostics.Process.GetCurrentProcess();
long endBytes = pmCurrentProcess.PrivateMemorySize64;
double kbEnd = (double)(endBytes) / 1024.0;
System.Console.WriteLine("Currently using " + kbEnd + "KB.");
System.Console.WriteLine("Leaked " + (kbEnd - kbStart) + "KB.");
System.Console.ReadKey();
}
}
}
The output in Release build is:
Currently using 18800KB.
Currently using 118664KB.
Leaked 99864KB.
I assume that the GC.collect call will remove the allocated strings since they go out of scope, but it appears it does not. I do not understand nor can I find an explanation for it. Maybe anyone here?
Thanks,
Alex

You're looking at the private memory size - the managed heap will have expanded to accommodate the strings, but it won't release the memory back to the operating system when the strings are garbage collected. Instead, the managed heap will be bigger, but have lots of free space - so if you create more objects, it won't require the heap to expand.
If you want to look at the memory used within the managed heap, look at GC.GetTotalMemory. Note that due to the complexities of garbage collection, there's a certain amount of woolliness within all of this.

Indeed I used the private mem size because that's the one that's closest to the one in Process Explorer
if I rewrite the program with the GC.GetTotalMemory like this:
using System;
namespace GCMemTest
{
class Program
{
static void Main(string[] args)
{
System.GC.Collect();
long startBytes = System.GC.GetTotalMemory(true);
{
string[] strings = new string[2000000];
for (int i = 0; i < 2000000; i++)
{
strings[i] = "blabla" + i;
}
strings = null;
}
System.GC.Collect();
long endBytes = System.GC.GetTotalMemory(true);
double kbStart = (double)(startBytes) / 1024.0;
double kbEnd = (double)(endBytes) / 1024.0;
System.Console.WriteLine("Leaked " + (kbEnd - kbStart) + "KB.");
System.Console.ReadKey();
}
}
}
Then the output is:
Leaked 0KB.
Only when I have 'strings = null;' this is the case, remove it and I leak 100MB. This means that the local scope in the main routine does not cause the array to be freed. If I move that part into a static method Test, and call that one instead, I leak a few bytes. I guess what I should learn from this is that local scopes are ignored by the GC.

Related

How to use the ARCore camera image in OpenCV in an Unity Android app?

I am trying to use OpenCV for hand gesture recognition in my Unity ARCore game. However, with the deprecation of TextureReaderAPI, the only way to capture the image from the camera is by using Frame.CameraImage.AcquireCameraImageBytes(). The problem with that is not only that the image is in 640x480 resolution (this cannot be changed AFAIK), but it is also in YUV_420_888 format.
As if that were not enough, OpenCV does not have free C#/Unity packages, so if I do not want to cash out 20$ for a paid package, I need to use available C++ or python versions. How do I move the YUV image to OpenCV, convert it to an RGB (or HSV) color space, and then either do some processing on it or return it back to Unity?
In this example, I will use C++ OpenCV libraries and Visual Studio 2017 and I will try to capture ARCore camera image, move it to OpenCV (as efficiently as possible), convert it to RGB color space, then move it back to Unity C# code and save it in the phone's memory.
Firstly, we have to create a C++ dynamic library project to use with OpenCV. For this, I highly recommend to follow both Pierre Baret's and Ninjaman494's answers on this question: OpenCV + Android + Unity. The process is rather straightforward, and if you will not deviate from their answers too much (i.e. you can safely download a newer than 3.3.1 version of OpenCV, but be careful when compiling for ARM64 instead of ARM, etc.), you should be able to call a C++ function from C#.
In my experience, I had to solve two problems - firstly, if you made the project part of your C# solution instead of creating a new solution, Visual Studio will keep messing with your configuration, like trying to compile a x86 version instead of an ARM version. To save yourself the hassle, create a completely separate solution. The other problem is that some functions failed to link for me, thus throwing a undefined reference linker error (undefined reference to 'cv::error(int, std::string const&, char const*, char const*, int, to be exact). If this happens and the problem is with a function that you do not really need, just recreate the function in your code - for instance if you have problems with cv::error, add this code in the end of your .cpp file:
namespace cv {
__noreturn void error(int a, const String & b, const char * c, const char * d, int e) {
throw std::string(b);
}
}
Sure, this is ugly and dirty way to do things, so if you know how to fix the linker error, please do so and let me know.
Now, you should have a working C++ code that compiles and can be run from a Unity Android application. However, what we want is for OpenCV to not return a number, but to convert an image. So change your code to this:
.h file
extern "C" {
namespace YOUR_OWN_NAMESPACE
{
int ConvertYUV2RGBA(unsigned char *, unsigned char *, int, int);
}
}
.cpp file
extern "C" {
int YOUR_OWN_NAMESPACE::ConvertYUV2RGBA(unsigned char * inputPtr, unsigned char * outputPtr, int width, int height) {
// Create Mat objects for the YUV and RGB images. For YUV, we need a
// height*1.5 x width image, that has one 8-bit channel. We can also tell
// OpenCV to have this Mat object "encapsulate" an existing array,
// which is inputPtr.
// For RGB image, we need a height x width image, that has three 8-bit
// channels. Again, we tell OpenCV to encapsulate the outputPtr array.
// Thanks to specifying existing arrays as data sources, no copying
// or memory allocation has to be done, and the process is highly
// effective.
cv::Mat input_image(height + height / 2, width, CV_8UC1, inputPtr);
cv::Mat output_image(height, width, CV_8UC3, outputPtr);
// If any of the images has not loaded, return 1 to signal an error.
if (input_image.empty() || output_image.empty()) {
return 1;
}
// Convert the image. Now you might have seen people telling you to use
// NV21 or 420sp instead of NV12, and BGR instead of RGB. I do not
// understand why, but this was the correct conversion for me.
// If you have any problems with the color in the output image,
// they are probably caused by incorrect conversion. In that case,
// I can only recommend you the trial and error method.
cv::cvtColor(input_image, output_image, cv::COLOR_YUV2RGB_NV12);
// Now that the result is safely saved in outputPtr, we can return 0.
return 0;
}
}
Now, rebuild the solution (Ctrl + Shift + B) and copy the libProjectName.so file to Unity's Plugins/Android folder, as in the linked answer.
The next thing is to save the image from ARCore, move it to C++ code, and get it back. Let us add this inside the class in our C# script:
[DllImport("YOUR_OWN_NAMESPACE")]
public static extern int ConvertYUV2RGBA(IntPtr input, IntPtr output, int width, int height);
You will be prompted by Visual Studio to add System.Runtime.InteropServices using clause - do so.
This allows us to use the C++ function in our C# code. Now, let's add this function to our C# component:
public Texture2D CameraToTexture()
{
// Create the object for the result - this has to be done before the
// using {} clause.
Texture2D result;
// Use using to make sure that C# disposes of the CameraImageBytes afterwards
using (CameraImageBytes camBytes = Frame.CameraImage.AcquireCameraImageBytes())
{
// If acquiring failed, return null
if (!camBytes.IsAvailable)
{
Debug.LogWarning("camBytes not available");
return null;
}
// To save a YUV_420_888 image, you need 1.5*pixelCount bytes.
// I will explain later, why.
byte[] YUVimage = new byte[(int)(camBytes.Width * camBytes.Height * 1.5f)];
// As CameraImageBytes keep the Y, U and V data in three separate
// arrays, we need to put them in a single array. This is done using
// native pointers, which are considered unsafe in C#.
unsafe
{
for (int i = 0; i < camBytes.Width * camBytes.Height; i++)
{
YUVimage[i] = *((byte*)camBytes.Y.ToPointer() + (i * sizeof(byte)));
}
for (int i = 0; i < camBytes.Width * camBytes.Height / 4; i++)
{
YUVimage[(camBytes.Width * camBytes.Height) + 2 * i] = *((byte*)camBytes.U.ToPointer() + (i * camBytes.UVPixelStride * sizeof(byte)));
YUVimage[(camBytes.Width * camBytes.Height) + 2 * i + 1] = *((byte*)camBytes.V.ToPointer() + (i * camBytes.UVPixelStride * sizeof(byte)));
}
}
// Create the output byte array. RGB is three channels, therefore
// we need 3 times the pixel count
byte[] RGBimage = new byte[camBytes.Width * camBytes.Height * 3];
// GCHandles help us "pin" the arrays in the memory, so that we can
// pass them to the C++ code.
GCHandle YUVhandle = GCHandle.Alloc(YUVimage, GCHandleType.Pinned);
GCHandle RGBhandle = GCHandle.Alloc(RGBimage, GCHandleType.Pinned);
// Call the C++ function that we created.
int k = ConvertYUV2RGBA(YUVhandle.AddrOfPinnedObject(), RGBhandle.AddrOfPinnedObject(), camBytes.Width, camBytes.Height);
// If OpenCV conversion failed, return null
if (k != 0)
{
Debug.LogWarning("Color conversion - k != 0");
return null;
}
// Create a new texture object
result = new Texture2D(camBytes.Width, camBytes.Height, TextureFormat.RGB24, false);
// Load the RGB array to the texture, send it to GPU
result.LoadRawTextureData(RGBimage);
result.Apply();
// Save the texture as an PNG file. End the using {} clause to
// dispose of the CameraImageBytes.
File.WriteAllBytes(Application.persistentDataPath + "/tex.png", result.EncodeToPNG());
}
// Return the texture.
return result;
}
To be able to run unsafe code, you also need to allow it in Unity. Go to Player Settings (Edit > Project Settings > Player Settings and check the Allow unsafe code checkbox.)
Now, you can call the CameraToTexture() function, let's say, every 5 seconds from Update(), and the camera image should be saved as /Android/data/YOUR_APPLICATION_PACKAGE/files/tex.png. The image will probably be landscape oriented, even if you held the phone in portrait mode, but this is not that hard to fix anymore. Also, you might notice a freeze everytime the image is saved - I recommend calling this function in a separate thread because of this. Also, the most demanding operation here is saving the image as an PNG file, so if you need it for any other reason, you should be fine (still use the separate thread, though).
If you want to undestand the YUV_420_888 format, why you need a 1.5*pixelCount array, and why we modified the arrays the way we did, read https://wiki.videolan.org/YUV/#NV12. Other websites seem to have incorrect information about how this format works.
Also, feel free to comment me with any issues you might have, and I will try to help with them, as well as any feedback for both the code and answer.
APPENDIX 1: According to https://docs.unity3d.com/ScriptReference/Texture2D.LoadRawTextureData.html, you should use GetRawTextureData instead of LoadRawTextureData, to prevent copying. To do this, just pin the array returned by GetRawTextureData instead of the RGBimage array (which you can remove). Also, do not forget to call result.Apply(); afterwards.
APPENDIX 2: Do not forget to call Free() on both GCHandles when you are done using them.
I figured out how to get the full resolution CPU image in Arcore 1.8.
I can now get the full camera resolution with cameraimagebytes.
put this in your class variables:
private ARCoreSession.OnChooseCameraConfigurationDelegate m_OnChoseCameraConfiguration = null;
put this in Start()
m_OnChoseCameraConfiguration = _ChooseCameraConfiguration; ARSessionManager.RegisterChooseCameraConfigurationCallback(m_OnChoseCameraConfiguration); ARSessionManager.enabled = false; ARSessionManager.enabled = true;
Add this callback to the class:
private int _ChooseCameraConfiguration(List<CameraConfig> supportedConfigurations) { return supportedConfigurations.Count - 1; }
Once you add those, you should have cameraimagebytes returning the full resolution of the camera.
For everyone who want to try this with OpencvForUnity:
public Mat getCameraImage()
{
// Use using to make sure that C# disposes of the CameraImageBytes afterwards
using (CameraImageBytes camBytes = Frame.CameraImage.AcquireCameraImageBytes())
{
// If acquiring failed, return null
if (!camBytes.IsAvailable)
{
Debug.LogWarning("camBytes not available");
return null;
}
// To save a YUV_420_888 image, you need 1.5*pixelCount bytes.
// I will explain later, why.
byte[] YUVimage = new byte[(int)(camBytes.Width * camBytes.Height * 1.5f)];
// As CameraImageBytes keep the Y, U and V data in three separate
// arrays, we need to put them in a single array. This is done using
// native pointers, which are considered unsafe in C#.
unsafe
{
for (int i = 0; i < camBytes.Width * camBytes.Height; i++)
{
YUVimage[i] = *((byte*)camBytes.Y.ToPointer() + (i * sizeof(byte)));
}
for (int i = 0; i < camBytes.Width * camBytes.Height / 4; i++)
{
YUVimage[(camBytes.Width * camBytes.Height) + 2 * i] = *((byte*)camBytes.U.ToPointer() + (i * camBytes.UVPixelStride * sizeof(byte)));
YUVimage[(camBytes.Width * camBytes.Height) + 2 * i + 1] = *((byte*)camBytes.V.ToPointer() + (i * camBytes.UVPixelStride * sizeof(byte)));
}
}
// Create the output byte array. RGB is three channels, therefore
// we need 3 times the pixel count
byte[] RGBimage = new byte[camBytes.Width * camBytes.Height * 3];
// GCHandles help us "pin" the arrays in the memory, so that we can
// pass them to the C++ code.
GCHandle pinnedArray = GCHandle.Alloc(YUVimage, GCHandleType.Pinned);
IntPtr pointer = pinnedArray.AddrOfPinnedObject();
Mat input = new Mat(camBytes.Height + camBytes.Height / 2, camBytes.Width, CvType.CV_8UC1);
Mat output = new Mat(camBytes.Height, camBytes.Width, CvType.CV_8UC3);
Utils.copyToMat(pointer, input);
Imgproc.cvtColor(input, output, Imgproc.COLOR_YUV2RGB_NV12);
pinnedArray.Free();
return output;
}
}
Here is an implementation of this which just uses the free plugin OpenCV Plus Unity. Very simple to set up and great documentation if you are familiar with OpenCV.
This implementation rotates the image properly using OpenCV, stores them into memory and upon exiting the app, saves them to file. I have tried to strip all Unity aspects from the code so that the function GetCameraImage() can be run on a separate thread.
I can confirm it works on Andoird (GS7), I presume it will work pretty universally.
using System;
using System.Collections.Generic;
using GoogleARCore;
using UnityEngine;
using OpenCvSharp;
using System.Runtime.InteropServices;
public class CamImage : MonoBehaviour
{
public static List<Mat> AllData = new List<Mat>();
public static void GetCameraImage()
{
// Use using to make sure that C# disposes of the CameraImageBytes afterwards
using (CameraImageBytes camBytes = Frame.CameraImage.AcquireCameraImageBytes())
{
// If acquiring failed, return null
if (!camBytes.IsAvailable)
{
return;
}
// To save a YUV_420_888 image, you need 1.5*pixelCount bytes.
// I will explain later, why.
byte[] YUVimage = new byte[(int)(camBytes.Width * camBytes.Height * 1.5f)];
// As CameraImageBytes keep the Y, U and V data in three separate
// arrays, we need to put them in a single array. This is done using
// native pointers, which are considered unsafe in C#.
unsafe
{
for (int i = 0; i < camBytes.Width * camBytes.Height; i++)
{
YUVimage[i] = *((byte*)camBytes.Y.ToPointer() + (i * sizeof(byte)));
}
for (int i = 0; i < camBytes.Width * camBytes.Height / 4; i++)
{
YUVimage[(camBytes.Width * camBytes.Height) + 2 * i] = *((byte*)camBytes.U.ToPointer() + (i * camBytes.UVPixelStride * sizeof(byte)));
YUVimage[(camBytes.Width * camBytes.Height) + 2 * i + 1] = *((byte*)camBytes.V.ToPointer() + (i * camBytes.UVPixelStride * sizeof(byte)));
}
}
// GCHandles help us "pin" the arrays in the memory, so that we can
// pass them to the C++ code.
GCHandle pinnedArray = GCHandle.Alloc(YUVimage, GCHandleType.Pinned);
IntPtr pointerYUV = pinnedArray.AddrOfPinnedObject();
Mat input = new Mat(camBytes.Height + camBytes.Height / 2, camBytes.Width, MatType.CV_8UC1, pointerYUV);
Mat output = new Mat(camBytes.Height, camBytes.Width, MatType.CV_8UC3);
Cv2.CvtColor(input, output, ColorConversionCodes.YUV2BGR_NV12);// YUV2RGB_NV12);
// FLIP AND TRANPOSE TO VERTICAL
Cv2.Transpose(output, output);
Cv2.Flip(output, output, FlipMode.Y);
AllData.Add(output);
pinnedArray.Free();
}
}
}
I then call ExportImages() when exiting the program to save to file.
private void ExportImages()
{
/// Write Camera intrinsics to text file
var path = Application.persistentDataPath;
StreamWriter sr = new StreamWriter(path + #"/intrinsics.txt");
sr.WriteLine(CameraIntrinsicsOutput.text);
Debug.Log(CameraIntrinsicsOutput.text);
sr.Close();
// Loop through Mat List, Add to Texture and Save.
for (var i = 0; i < CamImage.AllData.Count; i++)
{
Mat imOut = CamImage.AllData[i];
Texture2D result = Unity.MatToTexture(imOut);
result.Apply();
byte[] im = result.EncodeToJPG(100);
string fileName = "/IMG" + i + ".jpg";
File.WriteAllBytes(path + fileName, im);
string messge = "Succesfully Saved Image To " + path + "\n";
Debug.Log(messge);
Destroy(result);
}
}
Seems you already fix this.
But for anyone who want to combine AR with hand gesture recognition and tracking, try Manomotion: https://www.manomotion.com/
free SDk and work prefect in 12/2020.
Use the SDK Community Edition and Download ARFoundation version

Reading Text file data through Multi threading in C#.net

I am reading the text file consisting of 6 columns. Among 6 columns each 3 columns show one object information I want to access these columns in parallel through multithreading. Like 3 columns for one object, altogether 2 threads have created except main thread.
The text file looks like this:
I tried it but I face difficulty in passing data from the main thread to other threads error occurs at string variable "part". (variable part doesn't exist in the current context)
I want to do multithreading for tag1 and tag2.
I am sharing the block of my code, please suggest me where I am mistaken
As I am new to multithread programming.
namespace MultiTag_Simulation_ConsoleApp
{
class Program
{
static void Main(string[] args)
{
string line;
string[] part;
StreamReader File = new StreamReader("2Tags_Points.txt");
while((line = File.ReadLine()) !=null)
{
part = line.Split('\t');
Thread TAG1 = new Thread(new ThreadStart(Tag1));
TAG1.Start();
}
}
void Tag1()
{
double w, x;
w = Convert.ToDouble(part[1]);
x = Convert.ToDouble(part[2]);
Console.WriteLine("Tag1 x:" + w + "\t" + "Tag1 y:" + x);
Console.ReadKey();
}
}
}
Thank you, everyone, for your time. I had mistaken in thread synchronization. Now I solved the issue, by initializing the "part" variable as static variable above the main thread.
static string [] part
Your solution, although it might compile, still has a lot of hidden problems. You need to synchronize access to shared variables for example and right now, if you did that, it would defeat the purpose of having multiple threads. I would suggest using a simpler framework, that will do the multi-threading for you, because multi-threading is hard to get right, but using multiple processors for your workload is way easier when you leave the hard stuff to the framework.
For example, this will calculate your stuff in parallel. Although just in parallel per line, not per tag, but as long as all your processors are used optimally, it really does not matter.
namespace MultiTag_Simulation_ConsoleApp
{
using System;
using System.IO;
using System.Linq;
using System.Threading.Tasks;
internal static class Program
{
internal static void Main()
{
Parallel.ForEach(
File.ReadLines("2Tags_Points.txt").Select(line => line.Split('\t')),
parts =>
{
var w = Convert.ToDouble(parts[1]);
var x = Convert.ToDouble(parts[2]);
Console.WriteLine("Tag1 x:" + w + "\t" + "Tag1 y:" + x);
var y = Convert.ToDouble(parts[4]);
var z = Convert.ToDouble(parts[5]);
Console.WriteLine("Tag2 x:" + y + "\t" + "Tag2 y:" + z);
});
}
}
}

OpenCL Cloo: Out of Resources Error

While running some test code in OpenCL (using Cloo C#), I started getting these OutOfResource errors from OpenCL and sometimes Unity just crashes entirely before I get an exception. I am basically re-calling a kernel function over and over with varying number of global/local work items to check timing. I leave the arguments the same and call the kernel starting with 2x2x2 global and 2x2x2 local and iterating uperwards checking only valid sizes. It works fine occasionally, but most of the time it completes about 30 or 40 Execute() calls and then crashes on the next Execute() call.
Note: Execute refers to the OpenCL.dll on the computer. The stack trace Unity returns is NULL I assume because of the native code.
Anyone have any idea what could be causing this?
Note: This version of Cloo is Cloo-Unity from GitHub and I am using it in Unity. The equivalent OpenCL function being called when I get the error is clEnqueueNDRangeKernel(), but it is called Execute() in Cloo.
Code Sample:
//Setup inputs one time...
foreach (var input in p_inputs)
{
inputs.Add(input.Function, input);
profiles.Add(input.Function, new RunProfile(input.Function, input.Weight));
input.Input.Prepare(package[input.Function]);
}
//Profile...
DateTime start;
int g_state = 0;
int l_state = 0;
long[] g = new long[3] { 2, 2, 2 };
long[] l = new long[3] { 2, 2, 2 };
while(g[0] * g[1] * g[2] < Device.MaxWorkGroupSize)
{
l[0] = 2; l[1] = 2; l[2] = 2; l_state = 0; //Reset locals
bool proceed = true;
while(proceed)
{
proceed = (l[0] != g[0] || l[1] != g[1] || l[2] != g[2]);
if (CLUtilities.ValidateExecutionParameters(Device, g, l))
{
Debug.Log("Profiling Start: " + g.ToEnumeratedString() + " / " + l.ToEnumeratedString());
foreach (var profile in profiles)
{
start = DateTime.Now;
//Exception here when on (g=6x4x4, l=6x4x4)
package.Execute(package[profile.Key], g, l);
package.Commands.Flush();
package.Commands.Finish();
float time = (float)(DateTime.Now - start).TotalMilliseconds;
profile.Value.AddRun(g, l, time);
}
Debug.Log("Profiling Ending: " + g.ToEnumeratedString() + " / " + l.ToEnumeratedString());
}
l[l_state] += 2;
l_state = (l_state == 2) ? 0 : l_state + 1;
}
g[g_state] += 2;
g_state = (g_state == 2) ? 0 : g_state + 1;
}
Sorry i cannot comment cause less than 50 rep. but which operating system do you use? gpu? driver?
i got similar problems caused by opencl.dll i used win10 and Nvidia (x64).
Also have a look on https://social.technet.microsoft.com/Forums/en-US/85680348-c2c4-40bc-9f39-9dcfeea331c0/windows-10-opencldll-error?forum=win10itprogeneral
It seems that there is/was a issue with the memory compression in win10.
My problem was caused by updating win7 to win10, without updating the nvidia drivers.
I just got back around to posting this, but the issue turned out be related to the fact that I didn't recall Kernel.SetArgument() each time I called the Execute() method. I originally did this because I was worried it would re-copy the buffer, but as it turns out the buffer copy doesn't occur in this method anyway (so the overhead was small anyway).
Does your nvidia graphics card for display?
If nvidia is main graphics card, you have to edit registry to turn off watchdog.
for windows 7
system/current/control/graphicsdriver
TdrLevel(DWORL) : 0

Why does some file get missed out if i use Parallel.ForEach()?

Following is the code which processes about 10000 files.
var files = Directory.GetFiles(directorypath, "*.*", SearchOption.AllDirectories).Where(
name => !name.EndsWith(".gif") && !name.EndsWith(".jpg") && !name.EndsWith(".png")).ToList();
Parallel.ForEach(files,Countnumberofwordsineachfile);
And the Countnumberofwordsineachfile function prints the number of words in each file into the text.
Whenever i implement Parallel.ForEach(), i miss about 4-5 files everytime while processing.
Can anyone suggest as to why this happens?
public void Countnumberofwordsineachfile(string filepath)
{
string[] arrwordsinfile = Regex.Split(File.ReadAllText(filepath).Trim(), #"\s+");
Charactercount = Convert.ToInt32(arrwordsinfile.Length);
filecontent.AppendLine(filepath + "=" + Charactercount);
}
fileContent is probably not threadsafe. So if two (or more) tasks attempt to append to it at the same time one will win, the other will not. You need to remember to either lock the sections that are shared, or don't used shared data.
This is probably the easiest solution for your code. Locking, synchronises access (other tasks have to queue up to access the locked section) so it will slow down the algorithm, but since this is very short compared to the part that counts the words is likely to be then it isn't really going to be much of an issue.
private object myLock = new object();
public void Countnumberofwordsineachfile(string filepath)
{
string[] arrwordsinfile = Regex.Split(File.ReadAllText(filepath).Trim(), #"\s+");
Charactercount = Convert.ToInt32(arrwordsinfile.Length);
lock(myLock)
{
filecontent.AppendLine(filepath + "=" + Charactercount);
}
}
The cause has already been found, here is an alternative implementation:
//Parallel.ForEach(files,Countnumberofwordsineachfile);
var fileContent = files
.AsParallel()
.Select(f=> f + "=" + Countnumberofwordsineachfile(f));
and that requires a more useful design for the count method:
// make this an 'int' function, more reusable as well
public int Countnumberofwordsineachfile(string filepath)
{ ...; return characterCount; }
But do note that going parallel won't help you much here, your main function (ReadAllText) is I/O bound so you will most likely see a degradation from using AsParallel().
The better option is to use Directory.EnumerateFiles and then collect the results without parallelism:
var files = Directory.EnumerateFiles(....);
var fileContent = files
//.AsParallel()
.Select(f=> f + "=" + Countnumberofwordsineachfile(f));

c# search in another process's memory - sudden lockout

I need to keep track of another program's memory, constantly looking for a sequence of bytes to appear in there, and when they do, i need to remember their location so i later know where to write to.
I used the following post to learn how to look for byte[] in another process's memory:
C#: Search a byte[] array in another process's memory
My program is very simple: It launches process (using Process.Start), and then repeatedly runs function from the linked thread's one of the answers:
private static int GetMemoryAddressOfString(byte[] searchedBytes)
{
IntPtr hProcess = OpenProcess(ProcessAccessFlags.VMOperation | ProcessAccessFlags.VMRead | ProcessAccessFlags.VMWrite, false, Program.ArtemisProcess.Id);
if (hProcess == IntPtr.Zero)
throw new Win32Exception(Marshal.GetLastWin32Error());
int addr = 0;
int speed = 1024 * 64;
for (int j = 0x00400000; j < 0x11000000; j += speed)
{
byte[] bigMem = new byte[speed + searchedBytes.Length];
IntPtr unmanagedPointer = Marshal.AllocHGlobal(4);
ReadProcessMemory(hProcess, (IntPtr)j, bigMem, new UIntPtr((uint)(speed + searchedBytes.Length)), unmanagedPointer);
int result = Marshal.ReadInt32(unmanagedPointer);
Marshal.DestroyStructure(unmanagedPointer, typeof(int));
for (int k = 0; k < bigMem.Length - searchedBytes.Length; k++)
{
bool found = true;
for (int l = 0; l < searchedBytes.Length; l++)
{
if (bigMem[k + l] != searchedBytes[l])
{
found = false;
break;
}
}
if (found)
{
addr = k + j;
break;
}
}
if (addr != 0)
break;
}
return addr;
}
where ArtemisProcess is the Process i ran with .Start()
Most of the times, it works fine. As soon as i do the action in the watched process that puts the searched sequence of bytes to the memory, the next search finds it. However, sometimes, it wont.
I was wondering if i'm right and used Cheat Engine to be sure that the searched data IS there.
Then i added the part where i create an unmanaged pointer to know how many bytes there were read - and thats when i found out that exactly the place in the memory where the searched bytes appear (that Cheat Engine correctly identifies) returns 0! It wont let me read memory there. This "lockout" happens for about a minute or two, and only then it allows me to read the memory (just out of sudden, the next attempt to read the memory at that location is a success and the sequence of bytes is found all right).
Now, i read on the msdn that "he function fails if the requested read operation crosses into an area of the process that is inaccessible" but how do i know which part of the process memory is accessible and which isnt?
Why is Cheat Engine able to read that memory, and my program isnt?
Why does it suddenly allow me to read the process memory again?
I am at a loss here...

Categories

Resources