The C# app searches for a matching string in a folder with various excel files, each file with various rows and there might be various match strings found in each file. The result will be kept in a global variable. it worked perfectly
without async task but when I tried async/await, the global variable receive some wrong result. can I know how to solve this problem.
public IList<SearchList> SearchResult { get; private set; }
public async Task ReadExcelData(string stringSearch)
{
string filePath = Path.Combine("D:/test/", "Packing List Folder");
SearchResult = new List<SearchList>();
string[] fileArray = Directory.GetFiles(filePath, "*.xls*", SearchOption.AllDirectories);
List<Task> tasks = new List<Task>();
foreach (var file in fileArray)
{
tasks.Add(Task.Run(() => GetXlData(file, stringSearch)));
}
await Task.WhenAll(tasks);
}
private void GetXlData(string file, string stringSearch)
{
using (ExcelEngine excelEngine = new ExcelEngine())
{
// open the file and read every cells here.
if (str == stringSearch)
{
PList =sheet[row, col].Value2;
unit =sheet[row, col+2].Value2;
WriteData(PList, unit);
}
workbook.Close();
stream.Close();
}//how to made a return SearchResult in each task??
private void WriteData(string cde, string unt)
{
SearchResult.Add(new SearchList
{
Code = cde,
Unit = unt,
});
}
public class SearchList
{
public string Code { get; set; }
public string Unit { get; set; }
public override string ToString()
{
return Code;
}
}
this is the app structure I made..
First option is to use a lock
https://learn.microsoft.com/en-us/dotnet/csharp/language-reference/keywords/lock-statement
so you set the lock, access the global variable and release the lock again. So only one thread at a time can write to the variable. But I would not do this. Don't use global variables when dealing with threads.
Second option:
A task can return something.
after the Task.WhenAll you can iterate over your tasks and get the results (Result property in the task) and add them to your global variable.
I rewrote code to return SearchList from GetXlData as plan but it show error CS0029 Cannot implicitly convert type 'System.Collections.Generic.List' to 'System.Threading.Tasks.Task', at
tasks.Add(Task.Run(() => GetXlData(file, stringSearch)));
Can help me solve this error. Thanks.
public IList<SearchList> SearchResult { get; private set; }
public async Task ReadExcelData(string stringSearch)
{
string filePath = Path.Combine("D:/test/", "Packing List Folder");
SearchResult = new List<SearchList>();
string[] fileArray = Directory.GetFiles(filePath, "*.xls*", SearchOption.AllDirectories);
List<Task<SearchList>> tasks = new List<Task<SearchList>>();
foreach (var file in fileArray)
{
tasks.Add(Task.Run(() => GetXlData(file, stringSearch)));
}
var results = await Task.WhenAll(tasks);
foreach (var item in results)
{
SearchResult.Add(item);
}
}
private List<SearchList> GetXlData(string file, string stringSearch)
{
//i guess there is error here but unable fix this.
List<SearchList> container = new List<SearchList>();
using (ExcelEngine excelEngine = new ExcelEngine())
{
// open the file and read every cells here.
if (str == stringSearch)
{
WriteData(PList, unit);
}
container.AddRange((IEnumerable<SearchList>)WriteData(PList, unit));
}
return container;
}
private SearchList WriteData(string cde, string unt)
{
SearchList output = new SearchList
{
Code = cde,
Unit = unt
};
return output;
}
grateful everyone support me..Thanks.
Related
I am trying to get the object value but I don't know how to do it. I'm new to C# and its giving me syntax error. I want to print it separately via the method "PrintSample" How can I just concatenate or append the whatData variable . Thank you.
PrintSample(getData, "name");
PrintSample(getData, "phone");
PrintSample(getData, "address");
//Reading the CSV file and put it in the object
string[] lines = File.ReadAllLines("sampleData.csv");
var list = new List<Sample>();
foreach (var line in lines)
{
var values = line.Split(',');
var sampleData = new Sample()
{
name = values[0],
phone = values[1],
address = values[2]
};
list.Add(sampleData);
}
public class Sample
{
public string name { get; set; }
public string phone { get; set; }
public string adress { get; set; }
}
//Method to call to print the Data
private static void PrintSample(Sample getData, string whatData)
{
//THis is where I'm having error, how can I just append the whatData to the x.?
Console.WriteLine( $"{getData. + whatData}");
}
In C# it's not possible to dynamically evaluate expression like
$"{getData. + whatData}"
As opposed to languages like JavaScript.
I'd suggest to use rather switch expression or Dictionary<string, string>
public void PrintData(Sample sample, string whatData)
{
var data = whatData switch
{
"name" => sample.name,
"phone" => sample.phone,
"address" => sample.address
_ => throw new ArgumentOutOfRangeException(nameof(whatData)),
};
Console.WriteLine(data);
}
I'm not sure what you are trying to achieve. Perhaps this will help you:
private static void PrintSample(Sample getData, string whatData)
{
var property = getData.GetType().GetProperty(whatData);
string value = (string)property?.GetValue(getData) ?? "";
Console.WriteLine($"{value}");
}
What PO really needs is
private static void PrintSamples(List<Sample> samples)
{
foreach (var sample in samples)
Console.WriteLine($"name : {sample.name} phone: {sample.phone} address: {sample.address} ");
}
and code
var list = new List<Sample>();
foreach (var line in lines)
{
......
}
PrintSamples(list);
it is radicolous to use
PrintSample(getData, "name");
instead of just
PrintSample(getData.name)
You can do this using reflection. However, it's known to be relatively slow.
public static void PrintSample(object getData, string whatData)
{
Console.WriteLine( $"{getData.GetType().GetProperty(whatData).GetValue(getData, null)}");
}
I have this code which reads from my json file an array of words
public static string[] GetProfanity()
{
var json = string.Empty;
using (var fs = File.OpenRead("profanity.json"))
using (var sr = new StreamReader(fs, new UTF8Encoding(false)))
json = sr.ReadToEnd();
var profanityJson = JsonConvert.DeserializeObject<ProfanityJson>(json);
return profanityJson.badwords;
}
This is the json
{
"badwords" : ["bad", "stupid"]
}
And i try to access this here
public static bool ProfanityCheck(string inputString)
{
string[] badWords = GetProfanity();
string checkString = inputString.ToLower();
if (badWords.Any(checkString.Contains))
return true;
return false;
}
As requested I access the ProfanityCheck method here
[Command("echo")]
[Description("says whatever the user gives")]
public async Task Echo(CommandContext ctx, [RemainingText] string echoText)
{
bool hasProfanity = ProfanityFilter.ProfanityCheck(echoText);
if(hasProfanity)
{
var errMsg = ProfanityFilter.ErrorMessage();
var errSent = await ctx.Channel.SendMessageAsync(embed: errMsg).ConfigureAwait(false);
Thread.Sleep(3000);
await ctx.Channel.DeleteMessageAsync(errSent).ConfigureAwait(false);
await ctx.Channel.DeleteMessageAsync(ctx.Message).ConfigureAwait(false);
return;
}
await ctx.Channel.SendMessageAsync(echoText).ConfigureAwait(false);
}
and the struct I Deserialize it as
public struct ProfanityJson
{
[JsonProperty("badwords")]
public string[] badwords { get; private set; }
}
but when i attempt to search for this any bad words in a string I pass, nothing happens, no errors in the console, no output otherwise. I have it set up so that it sends me an error message when profanity is found, but in its current state it does nothing when profanity is passed
Your code seems to be correct... I would write the GetProfanity() in another way (and I wouldn't surely reread it every time a word is passed to to ProfanityCheck) but this is tangential to your problem. I've written a minimum testable example:
public class ProfanityJson
{
public string[] badwords { get; set; }
}
public static class ProfanityChecker
{
public static string[] GetProfanity()
{
var json = string.Empty;
using (var fs = File.OpenRead("profanity.json"))
using (var sr = new StreamReader(fs, new UTF8Encoding(false)))
json = sr.ReadToEnd();
var profanityJson = JsonConvert.DeserializeObject<ProfanityJson>(json);
return profanityJson.badwords;
}
public static string[] GetProfanity2()
{
using (var sr = new StreamReader("profanity.json"))
using (var jtr = new JsonTextReader(sr))
{
var ser = new JsonSerializer();
var profanityJson = ser.Deserialize<ProfanityJson>(jtr);
return profanityJson.badwords;
}
}
public static bool ProfanityCheck(string inputString)
{
string[] badWords = GetProfanity2();
Trace.WriteLine($"Loaded {badWords.Length} bad words");
string checkString = inputString.ToLower();
if (badWords.Any(checkString.Contains))
return true;
return false;
}
}
static void Main(string[] args)
{
Console.WriteLine(ProfanityChecker.ProfanityCheck("badder"));
}
So the only idea I have is that you are using a "stale" version of profanity.json. I've added a little loggin in the ProfanityCheck() method. It will go to the Output pane in Visual Studio.
(Would be a mess as a comment)
You could have your class like this:
public class ProfanityJson
{
[JsonProperty("badwords")]
public string[] Badwords { get; set; }
}
Is it like so? Json is case sensitive.
This question already has answers here:
Help someone new to C# variables
(5 answers)
Closed 2 years ago.
In my current application while I have been able to implement the required logic that I need I am really stuck when trying to take off the content from the main method and using it from a different method .
My code is as below,
class Program
{
const string path = #"filePath";
static void Main(string[] args)
{
setUpValues();
}
private static void setUpValues()
{
var Content = JsonConvert.DeserializeObject<deploy>(File.ReadAllText(path));
List<Variable> variables = Content.Variables.ToList();
Scopes Scope = Content.ScopeValues;
string Version = null;
List<string> ListOfSelectedItems= new List<string>();
List<string> TempListOfSelectedItems = new List<string>();
List<string> Channels = new List<string>();
foreach (var item in variables)
{
if (item.Name.Equals("version"))
{
Version = item.Value;
}
if (item.Name.Equals("Selected"))
{
TempListOfSelectedItems.Add(item.Value);
}
}
Console.WriteLine("Version " + Version);
Console.WriteLine();
string SelectedItems= TempListOfSelectedItems[0];
ListOfSelectedItems = SelectedItems.Split(',').ToList();
Console.WriteLine();
Console.WriteLine("Selected Modules");
Console.WriteLine();
foreach (var item in ListOfSelectedItems)
{
Console.WriteLine(item);
}
foreach (var item in Scope.Channels)
{
Channels.Add(item.Name);
}
}
}
I want to be able to access the variable string Version , the List of ListOfSelectedItems and the List of channels from outside this method .. I want to use these in another as well . So how can I make these globally accessible ?
Would really appreciate your help on this as I have been stuck here
In order to use variables outside a method, you should declare them as fields of a class. Like this:
class Program
{
const string path = #"filePath";
static deploy Content;
static string Version;
static List<string> ListOfSelectedItems;
static List<string> TempListOfSelectedItems;
static List<string> Channels;
// and others
static void Main(string[] args)
{
setUpValues();
}
private static void setUpValues()
{
Content = JsonConvert.DeserializeObject<deploy>(File.ReadAllText(path));
List<Variable> variables = Content.Variables.ToList();
Scopes Scope = Content.ScopeValues;
Version = null;
ListOfSelectedItems = new List<string>();
TempListOfSelectedItems = new List<string>();
Channels = new List<string>();
foreach (var item in variables)
{
if (item.Name.Equals("version"))
{
Version = item.Value;
}
if (item.Name.Equals("Selected"))
{
TempListOfSelectedItems.Add(item.Value);
}
}
Console.WriteLine("Version " + Version);
Console.WriteLine();
string SelectedItems = TempListOfSelectedItems[0];
ListOfSelectedItems = SelectedItems.Split(',').ToList();
Console.WriteLine();
Console.WriteLine("Selected Modules");
Console.WriteLine();
foreach (var item in ListOfSelectedItems)
{
Console.WriteLine(item);
}
foreach (var item in Scope.Channels)
{
Channels.Add(item.Name);
}
}
}
You have to declare those fields as static because they are used in a static method. After the setUpValues finishes running, you can use those fields inside the Main method as well.
Also, this is not related to the question, but the general code convention in C# is to start methods' names with an uppercase letter (so SetUpValues instead of setUpValues) and to start the local variables' names with a lowercase letter (selectedItems instead of SelectedItems). Obviously, it's ultimately up to you how to name things and which code convention to use.
Create a class with properties that you want to access from other places. Instantiate this class in setUpValues and return this.
public class TestClass
{
public TestClass()
{
this.ListOfSelectedItems = new List<string>();
}
public string Version { get; set; }
public List<string> ListOfSelectedItems { get; set; }
}
And then modify your Main method as:
var myObj = setUpValues();
And then Modify setUpValues to return this:
private static TestClass setUpValues()
{
var Content = JsonConvert.DeserializeObject<deploy>(File.ReadAllText(path));
List<Variable> variables = Content.Variables.ToList();
Scopes Scope = Content.ScopeValues;
string Version = null;
List<string> ListOfSelectedItems = new List<string>();
List<string> TempListOfSelectedItems = new List<string>();
List<string> Channels = new List<string>();
foreach (var item in variables)
{
if (item.Name.Equals("version"))
{
Version = item.Value;
}
if (item.Name.Equals("Selected"))
{
TempListOfSelectedItems.Add(item.Value);
}
}
var retObj = new TestClass();
Console.WriteLine("Version " + Version);
Console.WriteLine();
retObj.Version = Version;
string SelectedItems = TempListOfSelectedItems[0];
ListOfSelectedItems = SelectedItems.Split(',').ToList();
Console.WriteLine();
Console.WriteLine("Selected Modules");
Console.WriteLine();
foreach (var item in ListOfSelectedItems)
{
Console.WriteLine(item);
retObj.ListOfSelectedItems.Add(item);
}
foreach (var item in Scope.Channels)
{
Channels.Add(item.Name);
}
return retObj;
}
How to read a CSV file from SFTP and use CSVHelper to parse the content without saving CSV locally?
Is this possible, or do we have to save it locally, parse and delete the file?
I am using SSH.Net and CSVHelper.
It needs to rely on Stream-processing of file:
public async Task ProcessRemoteFilesAsync()
{
var credentials = new Credentials("host", "username", "password");
var filePaths = new List<string>();
// initializing filePaths ..
var tasks = filePaths
.Select(f => ParseRemoteFileAsync(credentials, f))
.ToArray();
var results = await Task.WhenAll(tasks).ConfigureAwait(false);
// traverse through results..
}
public async Task<FileContent> ParseRemoteFileAsync(Credentials credentials, string filePath)
{
using (var sftp = new SftpClient(credentials.host, credentials.username, credentials.password))
{
sftp.Connect();
try
{
using (var remoteFileStream = sftp.OpenRead(filePath))
{
using (var reader = new StreamReader(remoteFileStream))
{
using (var csv = new CsvReader(reader))
{
/*
// Example of CSV parsing:
var records = new List<Foo>();
csv.Read();
csv.ReadHeader();
while (csv.Read())
{
var record = new Foo
{
Id = csv.GetField<int>("Id"),
Name = csv.GetField("Name")
};
records.Add(record);
}
*/
}
}
}
}
finally {
sftp.Disconnect();
}
}
}
Modified version that uses pool of SftpClient
See C# Object Pooling Pattern implementation.
Implementation of pool borrowed from How to: Create an Object Pool by Using a ConcurrentBag:
/// <summary>
/// Implementation borrowed from [How to: Create an Object Pool by Using a
/// ConcurrentBag](https://learn.microsoft.com/en-us/dotnet/standard/collections/thread-safe/how-to-create-an-object-pool).
/// </summary>
/// <typeparam name="T"></typeparam>
public class ObjectPool<T> : IDisposable
where T : IDisposable
{
private readonly Func<T> _objectGenerator;
private readonly ConcurrentBag<T> _objects;
public ObjectPool(Func<T> objectGenerator)
{
_objectGenerator = objectGenerator ?? throw new ArgumentNullException(nameof(objectGenerator));
_objects = new ConcurrentBag<T>();
}
public void Dispose()
{
while (_objects.TryTake(out var item))
{
item.Dispose();
}
}
public T GetObject()
{
return _objects.TryTake(out var item) ? item : _objectGenerator();
}
public void PutObject(T item)
{
_objects.Add(item);
}
}
The simplest Pool-based implementation (it doesn't care about exception processing, retry-policies):
internal class SftpclientTest
{
private readonly ObjectPool<SftpClient> _objectPool;
public SftpclientTest(Credentials credentials)
{
_objectPool = new ObjectPool<SftpClient>(() =>
{
var client = new SftpClient(credentials.host, credentials.username, credentials.password);
client.Connect();
return client;
});
}
public void GetDirectoryList()
{
var client = _objectPool.GetObject();
try
{
// client.ListDirectory() ..
}
finally
{
if (client.IsConnected)
{
_objectPool.PutObject(client);
}
}
}
public async Task ProcessRemoteFilesAsync()
{
var filePaths = new List<string>();
// initializing filePaths ..
var tasks = filePaths
.Select(f => ParseRemoteFileAsync(f))
.ToArray();
var results = await Task.WhenAll(tasks).ConfigureAwait(false);
// traverse through results..
}
public Task<FileContent> ParseRemoteFileAsync(string filePath)
{
var client = _objectPool.GetObject();
try
{
using (var remoteFileStream = client.OpenRead(filePath))
{
using (var reader = new StreamReader(remoteFileStream))
{
using (var csv = new CsvReader(reader))
{
// ..
}
}
return Task.FromResult(new FileContent());
}
}
finally
{
if (client.IsConnected)
{
_objectPool.PutObject(client);
}
}
}
}
I have the following code:
public int LoadFilesAndSaveInDatabase(string filesPath)
{
var calls = new ConcurrentStack<GdsCallDto>();
var filesInDirectory = this._directoryProxy.GetFiles(filesPath);
if (filesInDirectory.Any())
{
Parallel.ForEach(filesInDirectory, file =>
{
var lines = this._fileProxy.ReadAllLines(file, Encoding.Unicode);
if (lines.Any())
{
// Reads the file and setup a new DTO.
var deserializedCall = this._fileManager.DeserializeFileContent(lines, Path.GetFileName(file));
// Insert the DTO in the database.
this._gdsCallsData.InsertOrUpdateGdsCall(deserializedCall);
// We keep track of the dto to count the number of restored items.
calls.Push(deserializedCall);
}
});
}
return calls.Count;
}
And I have the following unit test:
[TestMethod]
public void ShouldLoadFilesAndSaveInDatabase()
{
// Arrange
var path = RandomGenerator.GetRandomString(56);
var encoding = Encoding.Unicode;
var fileNameEnvironment = RandomGenerator.GetRandomString();
var fileNameModule = RandomGenerator.GetRandomString();
var fileNameRecordLocator = RandomGenerator.GetRandomString(6);
var fileNameTimestamp = RandomGenerator.GetRandomDateTime().ToString("O").Replace(':', 'o');
// We simulate the presence of 4 files.
var files = new List<string>
{
RandomGenerator.GetRandomString(255),
RandomGenerator.GetRandomString(255),
RandomGenerator.GetRandomString(255),
RandomGenerator.GetRandomString(255)
}.ToArray();
var expectedResult = 4;
this._directoryProxy.Expect(d => d.GetFiles(path))
.Return(files);
this._fileProxy.Expect(f => f.ReadAllLines(path, encoding))
.Return(files).Repeat.Times(files.Length);
// Act
var result = this._databaseReloadManager.LoadFilesAndSaveInDatabase(path);
// Assert
Assert.AreEqual(result, expectedResult);
this._directoryProxy.AssertWasCalled(d => d.GetFiles(path));
this._fileProxy.AssertWasCalled(f => f.ReadAllLines(path, Encoding.Unicode));
}
The problem is on the following line:
var lines = this._fileProxy.ReadAllLines(file, Encoding.Unicode);
Even though I set an expectation and a return value, when I run the unit test, it always returns null.
I am using Rhino.Mocks, it works perfectly fine elsewhere but not there.
I had a look at some discussions here but none of them helped. Can it be due to the use of Parallel.ForEach? Is there a way of doing such a mock?
If you need any other info, please let me know.
I don't think there is a problem with the Parallelization. It seems your problem is related to the proxy instance setup with Rhino Mock.
Ensure what you pass into the parameters of the ReadAllLines are the same as you call them when it run through the production code.
this._fileProxy.Expect(f => f.ReadAllLines(path, encoding))
.Return(files).Repeat.Times(files.Length);
For instance if you setup on different path and the value of that path diefferent when test executes you may see NULL in return. But again hard to tell without seeing the full setup/constructor in the code. Also check the random generators see what has been used in each time.
The below is I sort of put together and working for me.
Working means I don't get NULL for:
var lines = this._fileProxy.ReadAllLines(file, Encoding.Unicode);
//some dummy code so I can compile
public interface IProxyDir
{
IEnumerable<string> GetFiles(string path);
}
public class GdsCallDto
{
}
public class Proxy : IProxyDir
{
public IEnumerable<string> GetFiles(string path)
{
throw new NotImplementedException();
}
}
public interface IFileDir
{
IEnumerable<string> ReadAllLines(string path, Encoding encoding);
}
public class FileProxy : IFileDir
{
public IEnumerable<string> ReadAllLines(string path, Encoding encoding)
{
throw new NotImplementedException();
}
}
public interface IFileMgr
{
string DeserializeFileContent(IEnumerable<string> lines, string content);
}
public class FileMgr : IFileMgr
{
public string DeserializeFileContent(IEnumerable<string> lines, string content)
{
throw new NotImplementedException();
}
}
//system under test
public class Sut
{
private IProxyDir _directoryProxy;
private IFileDir _fileProxy;
private IFileMgr _fileManager;
public Sut(IProxyDir proxyDir, IFileDir fileProxy, IFileMgr mgr)
{
_fileManager = mgr;
_directoryProxy = proxyDir;
_fileProxy = fileProxy;
}
public int LoadFilesAndSaveInDatabase(string filesPath)
{
var calls = new ConcurrentStack<GdsCallDto>();
var filesInDirectory = this._directoryProxy.GetFiles(filesPath);
if (filesInDirectory.Any())
{
Parallel.ForEach(filesInDirectory, file =>
{
var lines = this._fileProxy.ReadAllLines("ssss", Encoding.Unicode);
if (lines.Any())
{
// Reads the file and setup a new DTO.
var deserializedCall = this._fileManager.DeserializeFileContent(lines, Path.GetFileName("file"));
// Insert the DTO in the database.
//this._gdsCallsData.InsertOrUpdateGdsCall(deserializedCall);
// We keep track of the dto to count the number of restored items.
//calls.Push(deserializedCall);
}
});
}
return 1;
}
}
Sample Unit Test
[TestClass]
public class UnitTest1
{
private IProxyDir _directoryProxy;
private IFileDir _fileProxy;
private IFileMgr _fileMgr;
private Sut _sut;
public UnitTest1()
{
_directoryProxy = MockRepository.GenerateMock<IProxyDir>();
_fileProxy = MockRepository.GenerateMock<IFileDir>();
_fileMgr = MockRepository.GenerateMock<IFileMgr>();
}
[TestMethod]
public void ShouldLoadFilesAndSaveInDatabase()
{
// Arrange
var path = RandomGenerator.GetRandomString(56);
var encoding = Encoding.Unicode;
var fileNameEnvironment = RandomGenerator.GetRandomString(5);
var fileNameModule = RandomGenerator.GetRandomString(5);
var fileNameRecordLocator = RandomGenerator.GetRandomString(6);
var fileNameTimestamp = RandomGenerator.GetRandomDateTime().ToString("O").Replace(':', 'o');
// We simulate the presence of 4 files.
var files = new List<string>
{
RandomGenerator.GetRandomString(255),
RandomGenerator.GetRandomString(255),
RandomGenerator.GetRandomString(255),
RandomGenerator.GetRandomString(255)
}.ToArray();
var expectedResult = 4;
this._directoryProxy.Expect(d => d.GetFiles(path))
.Return(files);
this._fileProxy.Expect(f => f.ReadAllLines(path, encoding))
.Return(files).Repeat.Times(files.Length);
_sut = new Sut(_directoryProxy, _fileProxy, _fileMgr);
// Act
var result = this._sut.LoadFilesAndSaveInDatabase(path);
// Assert
Assert.AreEqual(result, expectedResult);
this._directoryProxy.AssertWasCalled(d => d.GetFiles(path));
this._fileProxy.AssertWasCalled(f => f.ReadAllLines(path, Encoding.Unicode));
}
}
internal class RandomGenerator
{
public static string GetRandomString(int number)
{
return "ssss";
}
public static DateTime GetRandomDateTime()
{
return new DateTime();
}
}
I could get rid of this issue, probably caused by the use of random values. I am now calling the method IgnoreArguments() on my expectation:
this._fileProxy.Expect(f => f.ReadAllLines(path, encoding))
.Return(files).Repeat.Times(files.Length).IgnoreArguments();
It does the trick (i.e. the unit test is running successfully), but I do not know if it is very elegant.