I am attempting to write separate collections to seperate csv in a Parallel.Foreach loop. When I run this, I get the following error:
The process cannot access the file 'C:\temp\OCRProcess\2011\49\OCRProcess-2011-49.csv' because it is being used by another process.
When I look at all the worker processes, they are opening different files, none of them are using the same file location. From what I can tell, each thread (worker process) is not conflicting.
public void GenerateCSVFile(List<CSVFile> csvFiles)
{
Parallel.ForEach(csvFiles, (csvfile, state, index) =>
{
var fileLocation = _storageLocation + csvfile.Type + s + csvfile.Year + s + csvfile.Day + s + csvfile.Type + "-" +
csvfile.Year + "-" + csvfile.Day + ".csv";
if (!File.Exists(fileLocation))
{
File.Create(fileLocation);
}
using (var streamWriter = new StreamWriter(fileLocation))
{
foreach (var csvRecord in csvfile.CSVRecords)
{
streamWriter.WriteLine(csvRecord.Object_Id + "," + csvRecord.DocumentName + "," + csvRecord.BCI_DCN + "," + csvRecord.CreationDate);
}
}
});
}
Here is the CSVFile and CSVRecord classes just incase.
public sealed class CSVRecord
{
public String BCI_DCN { get; set; }
public String Object_Id { get; set; }
public String DocumentName { get; set; }
public String CreationDate { get; set; }
}
public sealed class CSVFile
{
public List<CSVRecord> CSVRecords { get; set; }
public String Year { get; set; }
public String Type { get; set; }
public String Day { get; set; }
public CSVFile()
{
CSVRecords = new List<CSVRecord>();
}
}
The issues is due to File.Create(fileLocation), which returns a FileStream and keeps the file open. When the StreamWriter attempted to open this, it was already open and caused the error.
To correct the problem, removed the following IF statement:
if (!File.Exists(fileLocation))
{
File.Create(fileLocation);
}
And updated the USING statement as followed. By adding the TRUE parameter, it allows StreamWriter to append to the file if it exists, otherwise create it.
using (var streamWriter = new StreamWriter(fileLocation, true))
{
foreach (var csvRecord in csvfile.CSVRecords)
{
streamWriter.WriteLine(csvRecord.Object_Id + "," + csvRecord.DocumentName + "," + csvRecord.BCI_DCN + "," + csvRecord.CreationDate);
}
}
Related
Here's an example of what the data looks like on each csv.
inventory.csv
ID, payment, Receipt#
90006, VISA, jk73kl_
90006, VISA, null
inventory.csv
INFO, VIEWS, DATE
9006_Produce_Banana_jk73kl__, 0, 10-13-20
9006_Produce_Banana, 0, 10-13-20
I am having trouble with my where, I need to search both forms and if there is a matching receipt number pull them out and combine the two rows.
using System;
using System.Collections.Generic;
using System.Linq;
namespace ConsoleApp1
{
class Program
{
static void Main()
{
string[] ourDataSheet = System.IO.File.ReadAllLines(#"../../OurData.csv");
string[] theirDataSheet = System.IO.File.ReadAllLines(#"../../TheirData.csv");
string ourDataHeaders = ourDataSheet[0];
string theirDataHeaders = theirDataSheet[0];
IEnumerable<string> TrackingQuery =
from ourData in ourDataSheet
let ourDataFields = ourData.Split(',')
from theirData in theirDataSheet
let theirDataFields = theirData.Split(',')
where theirDataFields[0].EndsWith(ourDataFields[2])
select ourDataFields[0] + "," + ourDataFields[1] + "," + ourDataFields[2] + "," + theirDataFields[0] + "," + theirDataFields[0] + "," + theirDataFields[1] + "," + theirDataFields[2] + "," + theirDataFields[3];
OutPutResults(TrackingQuery);
Console.ReadKey();
}
static void OutPutResults(IEnumerable<string> query)
{
foreach(string item in query)
{
Console.WriteLine(item);
Console.Read();
}
}
}
}
First things first, I would recommend that you use a third-party package like CsvHelper and do something like the following
public class Sale
{
public int ID { get; set; }
public string Payment { get; set; }
[Name("Recipt#")]
public string Recipt { get; set; }
}
public class Inventory
{
public string INFO { get; set; }
public string VIEWS { get; set; }
public string DATE { get; set; }
}
IEnumerable<Sale> sales;
IEnumerable<Inventory> inventory;
using (var reader = new StreamReader(#"../../OurData.csv"))
using (var csv = new CsvReader(reader, CultureInfo.InvariantCulture))
{
sales = csv.GetRecords<Sale>();
}
using (var reader = new StreamReader(#"../../TheirData.csv"))
using (var csv = new CsvReader(reader, CultureInfo.InvariantCulture))
{
inventory = csv.GetRecords<Inventory>();
}
var results = sales.Where(s => inventory.Any(i => i.INFO.EndsWith(s.Recipt)));
I'm getting an error saying:
The number of row value expressions in the INSERT statement exceeds the maximum allowed number of 1000 row values.
I'm getting JSON-data from an API-call. And when I'm trying to insert this in my SQL Server database I'm getting the error. I'm trying to insert 3000 rows. How can I solve this issue? The call result from the APi is stored in the variable "body", and then deserialized into the variable "json".
This is my code:
using (var response = await client.SendAsync(request))
{
response.EnsureSuccessStatusCode();
var body = await response.Content.ReadAsStringAsync();
var json = JsonConvert.DeserializeObject<Root>(body);
var Api = json.api;
string SqlString = "INSERT INTO land.leagues(" +
"league_id" +
",name" +
",type" +
",country" +
",country_code" +
",season" +
",season_start" +
",season_end" +
",logo" +
",flag" +
",standings" +
",is_current" +
",coverage_standings" +
",coverage_players" +
",coverage_topScorers" +
",coverage_predictions" +
",coverage_odds" +
",coverage_fixtures_events" +
",coverage_fixtures_lineups" +
",coverage_fixtures_statistics" +
",coverage_fixtures_playersStatistics" +
",created" +
") VALUES";
foreach (var a in Api.leagues)
{
SqlString += "(" +
"'" + a.league_id +
"','" + a.name.Replace("'","`") +
"','" + a.type +
"','" + a.country +
"','" + a.country_code +
"','" + a.season +
"','" + a.season_start +
"','" + a.season_end +
"','" + a.logo +
"','" + a.flag +
"','" + a.standings +
"','" + a.is_current +
"','" + a.coverage.standings +
"','" + a.coverage.players +
"','" + a.coverage.topScorers +
"','" + a.coverage.predictions +
"','" + a.coverage.odds +
"','" + a.coverage.fixtures.events +
"','" + a.coverage.fixtures.lineups +
"','" + a.coverage.fixtures.statistics +
"','" + a.coverage.fixtures.players_statistics +
"','" + DateTime.Now +
"'),";
}
SqlConnection con = new SqlConnection(#"_ConnectionString_");
SqlCommand cmd;
con.Open();
cmd = new SqlCommand("TRUNCATE TABLE land.leagues " + SqlString.Remove(SqlString.Length - 1), con);
cmd.ExecuteNonQuery();
}
These are all my classes:
public class Fixtures
{
public bool events { get; set; }
public bool lineups { get; set; }
public bool statistics { get; set; }
public bool players_statistics { get; set; }
}
public class Coverage
{
public bool standings { get; set; }
public Fixtures fixtures { get; set; }
public bool players { get; set; }
public bool topScorers { get; set; }
public bool predictions { get; set; }
public bool odds { get; set; }
}
public class League
{
public int league_id { get; set; }
public string name { get; set; }
public string type { get; set; }
public string country { get; set; }
public string country_code { get; set; }
public int season { get; set; }
public string season_start { get; set; }
public string season_end { get; set; }
public string logo { get; set; }
public string flag { get; set; }
public int standings { get; set; }
public int is_current { get; set; }
public Coverage coverage { get; set; }
}
public class Api
{
public int results { get; set; }
public List<League> leagues { get; set; }
}
public class Root
{
public Api api { get; set; }
}
You should insert each row one by one.
https://www.red-gate.com/simple-talk/sql/performance/comparing-multiple-rows-insert-vs-single-row-insert-with-three-data-load-methods/
Gist - 23 columns means inserting more than 2 at a time isn't worth it. Time wise, single row inserts is almost as fast as multiple row inserts. There is a case where stringifying your parameters can give you a 50% boost, but IMO not worth it. If you reduce your columns to 2 or even 7 it might be worth it.
Take those estimates with a grain of salt - the performance of the box may affect the relative benefits.
Of course keeping your row inserts to one at a time makes it easy to parameterize your query and keep things clear while eliminating the potential of sql injection. Not having to compile the sql everytime, while doing most of the parameter work on the client can help a lot also.
The maximum number of rows you can insert that way is 1000 so the error is fair enough. I would simply divide my rows into thousands and make a new INSERT statement for each batch of rows. Hope that makes sense :)
I am writing a code throug which I can serialize a JSON file with multiple data like shorttext,Geocoordinates, and image. But after generating it will show in a big line. But I want to organize it with new line order.But throug my code I cannot generated it. The Code I use here is-
public class GeoCoordinates
{
public double Longitude { get; set; }
public double Latitude { get; set; }
}
public class POIData
{
public string Shorttext { get; set; }
public GeoCoordinates GeoCoordinates { get; set; }
public List<string> Images { get; set; }
}
Now My Class where I use this classes is
public GeoCoordinates GeosFromString(string path)
{
string[] lines = File.ReadAllLines(path);
GeoCoordinates gc = new GeoCoordinates();
gc.Latitude = Double.Parse(lines[0].Substring(4));
gc.Longitude = Double.Parse(lines[1].Substring(4));
return gc;
}
public void ToJsonForLocation(string name)
{
var startPath = Application.StartupPath;
string folderName = Path.Combine(startPath, "Text_ImageWithHash");
System.IO.Directory.CreateDirectory(folderName);
string fileName = name + ".json";
var path = Path.Combine(folderName, fileName);
var Jpeg_File = new DirectoryInfo(startPath + #"\Image\" + name).GetFiles("*.jpg");
POIData Poi=new POIData();
Poi.Shorttext = File.ReadAllText(startPath + #"\Short Text\" + name + ".txt");
Poi.GeoCoordinates=GeosFromString(startPath + #"\Latitude Longitude\" + name + ".txt");
Poi.Images=new List<string> { Jpeg_File[0].Name};
string json = JsonConvert.SerializeObject(Poi);
File.WriteAllText(path , json);
}
Try to replace this code:
string json = JsonConvert.SerializeObject(Poi);
With this:
string json = JsonConvert.SerializeObject(Poi, Formatting.Indented);
Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 9 years ago.
Improve this question
How can I read the values inside H_N1_N2,H_N1_N3,H_N1_N4 that calls from H_N1_N1.
My objective is to put all the values into List that contains another object inside.
The requirement is to eliminate the current process which is storing the values to temp table in database before validation. After validated, it goes to the physical table.
Here's my entities:
public class H_N1_N1
{
public string N101_EntityIdentifierCode { get; set; }
public string N102_Name { get; set; }
public string N103_IdentificationCodeQualifier { get; set; }
public string N104_IdentificationCode { get; set; }
public H_N1_N1()
{
ListH_N1_N2 = new List<H_N1_N2>();
ListH_N1_N3 = new List<H_N1_N3>();
ListH_N1_N4 = new List<H_N1_N4>();
}
public List<H_N1_N2> ListH_N1_N2 { get; set; }
public List<H_N1_N3> ListH_N1_N3 { get; set; }
public List<H_N1_N4> ListH_N1_N4 { get; set; }
}
public class H_N1_N2
{
public string N201_Name01 { get; set; }
public string N202_Name02 { get; set; }
}
public class H_N1_N3
{
public string N301_AddressInformation01 { get; set; }
public string N302_AddressInformation02 { get; set; }
}
public class H_N1_N4
{
public string N401_CityName { get; set; }
public string N402_StateProvinceCode { get; set; }
public string N403_PostalCode { get; set; }
public string N404_CountryCode { get; set; }
}
Here's how I populate my object:
var N1_N1 = dal.GetModelDetails(11);
for (int i = 0; i < Convert.ToInt16(N1_N1.MaxLoop); i++)
{
H_N1_N1 oH_N1_N1 = new H_N1_N1();
stElements = ResizeArray(stElements.Count(), stElements, "H_N1_N1");
oH_N1_N1.N101_EntityIdentifierCode = stElements[1];
oH_N1_N1.N102_Name = stElements[2];
oH_N1_N1.N103_IdentificationCodeQualifier = stElements[3];
oH_N1_N1.N104_IdentificationCode = stElements[4];
objH_N1_N1.Add(oH_N1_N1);
var N1_N2 = dal.GetModelDetails(12);
H_N1_N2 oH_N1_N2 = new H_N1_N2();
stElements = ResizeArray(stElements.Count(), stElements, "H_N1_N2");
oH_N1_N2.N201_Name01 = stElements[1];
oH_N1_N2.N202_Name02 = stElements[2];
oH_N1_N1.ListH_N1_N2.Add(oH_N1_N2);
var N1_N3 = dal.GetModelDetails(14);
H_N1_N3 oH_N1_N3 = new H_N1_N3();
stElements = ResizeArray(stElements.Count(), stElements, "H_N1_N3");
oH_N1_N3.N301_AddressInformation01 = stElements[1];
oH_N1_N3.N302_AddressInformation02 = stElements[2];
oH_N1_N1.ListH_N1_N3.Add(oH_N1_N3);
var N1_N4 = dal.GetModelDetails(16);
H_N1_N4 oH_N1_N4 = new H_N1_N4();
stElements = ResizeArray(stElements.Count(), stElements, "H_N1_N4");
oH_N1_N4.N401_CityName = stElements[1];
oH_N1_N4.N402_StateProvinceCode = stElements[2];
oH_N1_N4.N403_PostalCode = stElements[3];
oH_N1_N4.N404_CountryCode = stElements[4];
}
There's the part where I cannot read the object:
foreach (var oH_N1_N1 in objH_N1_N1)
{
MessageBox.Show(
// Print N1
oH_N1_N1.N101_EntityIdentifierCode
+ "\n" + oH_N1_N1.N102_Name
+ "\n" + oH_N1_N1.N103_IdentificationCodeQualifier
+ "\n" + oH_N1_N1.N104_IdentificationCode
// Print N2
oH_N1_N1.ListH_N1_N2.N201_Name01 //ERROR HERE
+"\n" + oH_N1_N1.ListH_N1_N2.N201_Name02 //ERROR HERE
// Print N3
oH_N1_N1.ListH_N1_N3.N301_AddressInformation01 //ERROR HERE
+"\n" + oH_N1_N1.ListH_N1_N3.N301_AddressInformation02 //ERROR HERE
// Print N4
oH_N1_N1.ListH_N1_N4.N401_CityName //ERROR HERE
+"\n" + oH_N1_N1.ListH_N1_N4.N402_StateProvinceCode //ERROR HERE
+"\n" + oH_N1_N1.ListH_N1_N4.N403_PostalCode //ERROR HERE
+"\n" + oH_N1_N1.ListH_N1_N4.N404_CountryCode //ERROR HERE
);
}
This is what i am trying to do:
N1_somedata
N1_somedata
N1_somedata
N1_somedata
N2_somedata
N2_somedata
N3_somedata
N3_somedata
N4_somedata
N4_somedata
N4_somedata
N4_somedata
N1_somedata
N1_somedata
N1_somedata
N1_somedata
N2_somedata
N2_somedata
N3_somedata
N3_somedata
N4_somedata
N4_somedata
N4_somedata
N4_somedata
N1_somedata
N1_somedata
N1_somedata
N1_somedata
N2_somedata
N2_somedata
N3_somedata
N3_somedata
N4_somedata
N4_somedata
N4_somedata
N4_somedata
Thanks in advance!
OP, this code makes me want to weep for little children, but the reason you're getting errors is because you're placing 4 separate variables in MessageBox.Show() call and not tying them together.
Update
Based on your comment,
when I try to type oH_N1_N1.ListH_N1_N2, there is no N201_Name01 and N201_Name02
That's because oH_N1_N1.ListH_N1_N2 is a List<H_N1_N2> property. You cannot access properties of H_N1_N2 that way. You have to access via the list, for example using the indexer:
oH_N1_N1.ListH_N1_N2[0].N201_Name01
You can also do foreach to get all of the elements...
string crazyNames = string.Empty;
foreach(var crazyName in oH_N1_N1.ListH_N1_N2)
{
crazyNames += crazyName.N201_Name01 + " " + N201_Name02 // etc.
}
Notice the [0] above which is the first element in the list. Now, the intellisense will show you available properties of the stored object, which is an instance of H_N1_N2 and will contain the property N201_Name01 and so on.
Original issue / answer:
Add + signs there and it'll work.
For the love of humanity please use some different naming conventions!
foreach (var oH_N1_N1 in objH_N1_N1)
{
MessageBox.Show(
// Print N1
oH_N1_N1.N101_EntityIdentifierCode
+ "\n" + oH_N1_N1.N102_Name
+ "\n" + oH_N1_N1.N103_IdentificationCodeQualifier
+ "\n" + oH_N1_N1.N104_IdentificationCode
+ // concatenate next object to first one
// Print N2
oH_N1_N1.ListH_N1_N2.N201_Name01 //ERROR HERE
+"\n" + oH_N1_N1.ListH_N1_N2.N201_Name02 //ERROR HERE
+ // concatenate next object to first+second one
// Print N3
oH_N1_N1.ListH_N1_N3.N301_AddressInformation01 //ERROR HERE
+"\n" + oH_N1_N1.ListH_N1_N3.N301_AddressInformation02 //ERROR HERE
+ // concatenate last object to first+second+third one
// Print N4
oH_N1_N1.ListH_N1_N4.N401_CityName //ERROR HERE
+"\n" + oH_N1_N1.ListH_N1_N4.N402_StateProvinceCode //ERROR HERE
+"\n" + oH_N1_N1.ListH_N1_N4.N403_PostalCode //ERROR HERE
+"\n" + oH_N1_N1.ListH_N1_N4.N404_CountryCode //ERROR HERE
);
}
I am mapping JSON data to my own classes in C# and then trying to display various data where I need it however I'm having a small issue where I can't seem to get at data when I believe I should be able to. Obviously something is wrong and I was hoping someone could point me in the right direction:
Example snippet of JSON:
"items"
{
"0"
{
"name" "Riki's Dagger"
"prefab" "default_item"
"item_name" "#DOTA_Item_Rikis_Dagger"
"item_type_name" "#DOTA_WearableType_Daggers"
"model_player" "models/heroes/rikimaru/rikimaru_weapon.mdl"
"used_by_heroes"
{
"npc_dota_hero_riki" "1"
}
}
"1"
{
"name" "Anti-Mage's Glaive"
"prefab" "default_item"
"item_description" "#DOTA_Item_Desc_AntiMages_Glaives"
"item_name" "#DOTA_Item_AntiMages_Glaive"
"item_type_name" "#DOTA_WearableType_Glaive"
"model_player" "models/heroes/antimage/antimage_weapon.mdl"
"used_by_heroes"
{
"npc_dota_hero_antimage" "1"
}
}
I am using the following to serialize this to my own classes as follows:
DotaItemsGameResult itemGameResult
= JsonConvert.DeserializeObject<DotaItemsGameResult>(rawItemDetailsJson);
This is working OK as I have used this elsewhere.
The trouble comes here:
string dataToDisplay = "";
foreach (DotaItemsGameItem item in itemGameResult.Items_Game.Items.Item_Index)
{
dataToDisplay += "<h3> Item: " + item.Name + "</h3>";
dataToDisplay += "<p> Quality: " + item.Item_Quality + "</p>";
dataToDisplay += "<p> Attributes: " + item.Used_By_Heroes + "</p>";
}
return dataToDisplay;
My code seems to know about itemGameResult.Items_Game.Items but not when I want to go further into .Items_Index.
I have the following classes:
public class DotaItemsGameResult
{
public DotaItemsGameResultProperties Items_Game { get; set; }
}
public class DotaItemsGameResultProperties
{
public List<DotaItemsGameRarities> Rarities { get; set; }
public List<DotaItemsGameItems> Items { get; set; }
}
public class DotaItemsGameItems
{
public List<DotaItemsGameItem> Item_Index { get; set; }
}
public class DotaItemsGameItem
{
public string Name { get; set; }
public string Hidden { get; set; }
public string Item_Class { get; set; }
public string Item_Name { get; set; }
public string Item_Slot { get; set; }
public string Item_Quality { get; set; }
public string Min_Ilevel { get; set; }
public string Max_Ilevel { get; set; }
public string Item_Description { get; set; }
public string Item_Type_Name { get; set; }
public List<DotaItemsGameUsedByHero> Used_By_Heroes { get; set; }
}
Basically, my aim is to be able to get at the Used_By_Heroes data in my loop but I can't.
In my for each, itemGameResult.Items_Game.Items is fine, but itemGameResult.Items_Game.Items.Item_Index is not and I can't see why.
I can't access the the Item_Index when I use dot notation, there's no Item_Index where I would expect there to be. If I put it in anyway and try to compile, I see this:
Error 1 'System.Collections.Generic.List' does not contain a definition for 'Item_Index' and no extension method 'Item_Index' accepting a first argument of type 'System.Collections.Generic.List' could be found (are you missing a using directive or an assembly reference?).
Any help would be fantastic.
Problem is that the Items property of the class DotaItemsGameResultProperties is a collection (List<DotaItemsGameItems>) so it doesn't have a property Item_Index. Not sure what do you want to achieve, but one possibility to fix this is to iterate on this collection also:
string dataToDisplay = "";
foreach (DotaItemsGameItems items in itemGameResult.Items_Game.Items)
{
foreach (DotaItemsGameItem item in items.Item_Index)
{
dataToDisplay += "<h3> Item: " + item.Name + "</h3>";
dataToDisplay += "<p> Quality: " + item.Item_Quality + "</p>";
dataToDisplay += "<p> Attributes: " + item.Used_By_Heroes + "</p>";
}
}
return dataToDisplay;
This is to illustrate the structure of your data, you can do this in a more concise way:
string dataToDisplay = "";
var allItems = itemGameResult.Items_Game.Items.SelectMany(i => i.Item_Index);
foreach (DotaItemsGameItem item in allItems)
{
dataToDisplay += "<h3> Item: " + item.Name + "</h3>";
dataToDisplay += "<p> Quality: " + item.Item_Quality + "</p>";
dataToDisplay += "<p> Attributes: " + item.Used_By_Heroes + "</p>";
}
return dataToDisplay;