I have the following method:
public byte[] WriteCsvWithHeaderToMemory<T>(IEnumerable<T> records) where T : class
{
using (var memoryStream = new MemoryStream())
using (var streamWriter = new StreamWriter(memoryStream))
using (var csvWriter = new CsvWriter(streamWriter))
{
csvWriter.WriteRecords<T>(records);
return memoryStream.ToArray();
}
}
Which is being called with a list of objects - eventually from a database, but since something is not working I'm just populating a static collection. The objects being passed are as follows:
using CsvHelper.Configuration;
namespace Application.Models.ViewModels
{
public class Model
{
[CsvField(Name = "Field 1", Ignore = false)]
public string Field1 { get; set; }
[CsvField(Name = "Statistic 1", Ignore = false)]
public int Stat1{ get; set; }
[CsvField(Name = "Statistic 2", Ignore = false)]
public int Stat2{ get; set; }
[CsvField(Name = "Statistic 3", Ignore = false)]
public int Stat3{ get; set; }
[CsvField(Name = "Statistic 4", Ignore = false)]
public int Stat4{ get; set; }
}
}
What I'm trying to do is write a collection to a csv for download in an MVC application. Every time I try to write to the method though, the MemoryStream is coming back with zero length and nothing being passed to it. I've used this before, but for some reason it's just not working - I'm somewhat confused. Can anyone point out to me what I've done wrong here?
Cheers
You already have a using block which is great. That will flush your writer for you. You can just change your code slightly for it to work.
using (var memoryStream = new MemoryStream())
{
using (var streamWriter = new StreamWriter(memoryStream))
using (var csvWriter = new CsvWriter(streamWriter))
{
csvWriter.WriteRecords<T>(records);
} // StreamWriter gets flushed here.
return memoryStream.ToArray();
}
If you turn AutoFlush on, you need to be careful. This will flush after every write. If your stream is a network stream and over the wire, it will be very slow.
Put csvWriter.Flush(); before you return to flush the writer/stream.
EDIT: Per Jack's response. It should be the stream that gets flushed, not the csvWriter. streamWriter.Flush();. Leaving original solution, but adding this correction.
EDIT 2: My preferred answer is: https://stackoverflow.com/a/22997765/1795053 Let the using statements do the heavy lifting for you
Putting all these together (and the comments for corrections), including resetting the memory stream position, the final solution for me was;
using (MemoryStream ms = new MemoryStream())
{
using (TextWriter tw = new StreamWriter(ms))
using (CsvWriter csv = new CsvWriter(tw, CultureInfo.InvariantCulture))
{
csv.WriteRecords(errors); // Converts error records to CSV
tw.Flush(); // flush the buffered text to stream
ms.Seek(0, SeekOrigin.Begin); // reset stream position
Attachment a = new Attachment(ms, "errors.csv"); // Create attachment from the stream
// I sent an email here with the csv attached.
}
}
In case the helps someone else!
There is no flush in csvWriter, the flush is in the streamWriter. When called
csvWriter.Dispose();
it will flush the stream.
Another approach is to set
streamWriter.AutoFlush = true;
which will automatically flush the stream every time.
Here is working example:
void Main()
{
var records = new List<dynamic>{
new { Id = 1, Name = "one" },
new { Id = 2, Name = "two" },
};
Console.WriteLine(records.ToCsv());
}
public static class Extensions {
public static string ToCsv<T>(this IEnumerable<T> collection)
{
using (var memoryStream = new MemoryStream())
{
using (var streamWriter = new StreamWriter(memoryStream))
using (var csvWriter = new CsvWriter(streamWriter))
{
csvWriter.WriteRecords(collection);
} // StreamWriter gets flushed here.
return Encoding.ASCII.GetString(memoryStream.ToArray());
}
}
}
Based on this answer.
using CsvHelper;
public class TwentyFoursStock
{
[Name("sellerSku")]
public string ProductSellerSku { get; set; }
[Name("shippingPoint")]
public string ProductShippingPoint { get; set; }
}
using (var writer = new StreamWriter("file.csv"))
using (var csv = new CsvWriter(writer, CultureInfo.InvariantCulture))
{
csv.WriteRecords(TwentyFoursStock);
}
Related
When a user uses my program, it will generate many thousands of objects over the course of a couple of hours. These cannot accumulate in RAM, so I want to write them to a single file as they are generated. Then, in a different program, the objects must all be deserialized.
When I try to serialize different objects of the same class to the same XML file and then try to deserialize later, I get:
System.InvalidOperationException: 'There is an error in XML document (1, 206).'
Inner Exception
XmlException: There are multiple root elements. Line 1, position 206.
Here is an example of a .NET 6.0 console app that recapitulates this problem:
using System.Xml.Serialization;
using System.IO;
using System.Xml;
namespace ConsoleApp1
{
internal class Program
{
static void Main(string[] args)
{
// Serialize a person
using (FileStream stream = new FileStream("people.xml", FileMode.Create))
{
Person jacob = new Person { Name = "Jacob", Age = 33, Alive = true };
XmlSerializer serializer = new XmlSerializer(typeof(Person));
serializer.Serialize(stream, jacob);
}
// Serialize another person to the same file using the "clean XML" method
// https://stackoverflow.com/questions/1772004/how-can-i-make-the-xmlserializer-only-serialize-plain-xml
using (StreamWriter stream = new StreamWriter("people.xml", true))
{
Person rebecca = new Person { Name = "Rebecca", Age = 45, Alive = true };
stream.Write(SerializeToString(rebecca));
}
// Deserialize the people
List<Person> people = new List<Person>();
using (FileStream stream = new FileStream("people.xml", FileMode.Open))
{
XmlSerializer deserializer = new XmlSerializer(typeof(Person));
while (stream.Position < stream.Length)
{
people.Add((Person)deserializer.Deserialize(stream));
}
}
// See the people
foreach (Person person in people)
Console.WriteLine($"Hello. I am {person.Name}. I am {person.Age} and it is {person.Alive} that I am alive.");
}
// Serialize To Clean XML
public static string SerializeToString<T>(T value)
{
var emptyNamespaces = new XmlSerializerNamespaces(new[] { XmlQualifiedName.Empty });
var serializer = new XmlSerializer(value.GetType());
var settings = new XmlWriterSettings();
settings.Indent = true;
settings.OmitXmlDeclaration = true;
using (var stream = new StringWriter())
using (var writer = XmlWriter.Create(stream, settings))
{
serializer.Serialize(writer, value, emptyNamespaces);
return stream.ToString();
}
}
}
public class Person
{
public string Name { get; set; }
public int Age { get; set; }
public bool Alive { get; set; }
public Person()
{
Name = "";
}
}
}
Probably not the perfect answer, but could you de-serialize to a list, append your new person, and then re-serialize?
public static List<Person> ReadPeople(string file)
{
var people = new List<Person>();
if (File.Exists(file))
{
var serializer = new XmlSerializer(typeof(List<Person>));
using (var stream = new FileStream(file, FileMode.Open))
{
people = (List<Person>)serializer.Deserialize(stream);
}
}
return people;
}
public static void SavePerson(string file, Person person)
{
var people = ReadPeople(file);
people.Add(person);
var serializer = new XmlSerializer(typeof(List<Person>));
using (var stream = new FileStream(file, FileMode.Create))
{
serializer.Serialize(stream, people);
}
}
Additionally, on the "FileMode.Create", is the "FileMode.Append" option. The problem with that is it will append lists next to each other, rather than nesting the "people"
NB - I've split this into two functions to give the flexibility of being able to load the file easily
Hi I have an app that uses a json file to monitor run intervals of emails. I use it as a simple way to store the times I run an email notification. But I've run into an issue where the user can run concurrent instances of this app. What happens then is that two instances can read that json file and if they both read that an email is not sent, both instances will send an email leading to duplicate emails.
This is how the code is (sampled down for replicability)
public class SummaryEmailStatus
{
public DateTime DateSent { get; set; }
public string ToolType { get; set; }
public bool IsSent { get; set; }
}
public static class JsonUtil
{
public static Dictionary<string, SummaryEmailStatus> EmailStatusKVP = new Dictionary<string, SummaryEmailStatus>();
public static bool CreateEmailItemJasonFile(string jsonFileName)
{
var summCfgEmailStatus = new SummaryEmailStatus
{
DateSent = DateTime.Parse("2022-01-01"),
ToolType = "CIM",
IsSent = false
};
EmailStatusKVP.Add(SummaryEmailJsonObjs.ConfigError, summCfgEmailStatus);
string json = JsonConvert.SerializeObject(EmailStatusKVP, Formatting.Indented);
File.WriteAllText(jsonFileName, json);
}
public static Dictionary<string, SummaryEmailStatus> ReadEmailItemJsonFromFile(string jsonFileName)
{
string json = File.ReadAllText(jsonFileName);
EmailStatusKVP = JsonConvert.DeserializeObject<Dictionary<string, SummaryEmailStatus>>(json);
return EmailStatusKVP;
}
public static void WriteSummEmailStatusJsonToFile(string summaryEmailType, SummaryEmailStatus emailItem)
{
//string json = JsonConvert.SerializeObject(emailItem, Formatting.Indented);
EmailStatusKVP[summaryEmailType] = emailItem;
string json = JsonConvert.SerializeObject(EmailStatusKVP, Formatting.Indented);
File.WriteAllText(ParserConfigFilesConstants.SummaryEmailJsonFilePath, json);
}
}
The issue is I am using File.WritallText and ReadAllText. Is there a way to do what I am doing but in a way that locks the file each time the CreateEmailItemJasonFile or ReadEmailItemJsonFromFile or WriteSummEmailStatusJsonToFile is called?
I want only one instance for the console application to use this file. If the other instance tries to use it, it should get some "being used by another program" exception.
I saw this solution How to lock a file with C#? but with how new I am to C# I am not sure how to use it for my own needs.
I also thought about using a lock object around my File.Write and File.Read sections but I was under the impression that would only work if its another thread within the console application instance:
lock (fileReadLock)
{
string json = File.ReadAllText(jsonFileName);
}
I fixed it by using FileStream:
For reading I used:
FileStream fileStream = new FileStream(jsonFileName, FileMode.Open, FileAccess.Read, FileShare.None);
using (StreamReader reader = new StreamReader(fileStream))
{
json = reader.ReadToEnd();
}
And for Writing I used:
FileStream fs = new FileStream(ParserConfigFilesConstants.SummaryEmailJsonFilePath, FileMode.Create, FileAccess.Write, FileShare.None);
using (StreamWriter writer = new StreamWriter(fs))
{
writer.WriteLine(json);
writer.Flush();
}
This allows me to lock the file out whenever my app is using the file.
I'm trying to test my custom serialization and deserializaton logic with memory stream by writing the test cases but unable to test is because of this error while reading the stream during deserializaton.
EndOfStreamException Unable to read beyond the end of the stream
Am I using memory stream correctly or should I use some other stream? sharing the code:
[TestClass]
public class SerialTest
{
[TestMethod]
public void SerializationTestUsingStream()
{
Employee emp = new Employee(20);
MemoryStream stream = new MemoryStream();
this.SerializeEmployee(stream, emp);
StreamReader reader = new StreamReader(stream);
string text = reader.ReadToEnd(); // shows Empty string ""
var newEmp = this.DeserializeEmployee(stream);
emp.Should().Equals(newEmp);
}
private void SerializeEmployee(Stream stream, Employee collection)
{
using (BinaryWriter writer = new BinaryWriter(stream, Encoding.UTF8, true))
{
writer.Write(collection.age);
}
}
private Employee DeserializeEmployee(Stream stream)
{
using (var reader = new BinaryReader(stream, Encoding.UTF8, true))
{
int age = reader.ReadInt32(); // Exception Comes here while reading from the stream
return new Employee(age);
}
}
internal class Employee
{
public Employee(int age)
{
this.age = age;
}
public int age { get; set; }
}
}
You need to reset the position of the stream before read it.
stream.Position = 0L;
I am using the following code to serialize some data and save it to file:
MemoryStream stream = new MemoryStream();
DataContractJsonSerializer serializer = new DataContractJsonSerializer((typeof(Item)));
Item item = ((Item)list.SelectedItems[0].Tag);
serializer.WriteObject(stream, item);
var filepath = Program.appDataPath + list.SelectedItems[0].Group.Name + ".group";
stream.Position = 0;
using (FileStream fileStream = new FileStream(filepath, FileMode.Create))
{
stream.WriteTo(fileStream);
}
And later on, I'm trying to read back that data from file and insert it into ListView:
private void OpenFiles()
{
// DEBUG ONLY:
// Read into memorystream and filestream.
Console.WriteLine("Attempeting to open note.");
bool canLoad = false;
foreach (string file in Directory.GetFiles(Program.appDataPath))
{
if (file.EndsWith(".group"))
{
MemoryStream stream = new MemoryStream();
DataContractJsonSerializer serializer =
new DataContractJsonSerializer(
typeof(
List<Item>
)
);
using (FileStream fileStream =
new FileStream(
file,
FileMode.Open)
)
{
fileStream.CopyTo(stream);
}
stream.Position = 0;
//List<Withdrawal> tempWithList = new List<Withdrawal>();
foreach (Item item in (List<Item>)serializer.ReadObject(stream))
{
Console.WriteLine(item.Title + " " + item.Group.Name);
Item.Items.Add(item);
}
//Console.WriteLine("Got file \{file}");
//if (file.EndsWith(".group"))
//{
// Console.WriteLine("File is a group.");
// MemoryStream stream = new MemoryStream();
// DataContractJsonSerializer serializer = new DataContractJsonSerializer(typeof(List<Item>));
// using (FileStream fileStream = new FileStream(file, FileMode.Open))
// {
// fileStream.CopyTo(stream);
// }
// Console.WriteLine("Got stream");
// stream.Position = 0;
// try
// {
// Item.Items = (List<Item>)serializer.ReadObject(stream);
// Console.WriteLine("WTF?");
// }
// catch(Exception exception)
// {
// Console.WriteLine(exception.Message);
// }
// Console.WriteLine(Item.Items.Count);
// canLoad = true;
//}
//else Console.WriteLine("File is not a group.");
}
if (canLoad)
{
//list.Items.Clear();
foreach (Item item in Item.Items)
{
ListViewGroup group = new ListViewGroup(item.Group.Name);
list.Groups.Add(group);
list.Items.Add(
new ListViewItem(
item.Title,
group)
);
Console.WriteLine(item.Title + " " + item.Group.Name);
}
}
}
}
Now, the above exact code works in an older program (few months old), but it's not working in a new program. I have no idea why. I have set breakpoints EVERYWHERE and it has proven to be kind of pointless in this case.
One thing I did learn from setting a breakpoint is that even though the stream contains the data expected, the very next second, when it gets added to list, it is NULL. There is nothing in the list. I've run out of ideas, and Google wasn't of much help.
Group.cs:
using System;
using System.Collections.Generic;
using System.Linq;
using System.Runtime.Serialization;
using System.Text;
using System.Threading.Tasks;
namespace Notes
{
[DataContract]
public class Group
{
[DataMember]
public string Name { get; set; }
}
}
Item.cs:
using System;
using System.Collections.Generic;
using System.Linq;
using System.Runtime.Serialization;
using System.Text;
using System.Threading.Tasks;
namespace Notes
{
[DataContract]
[Serializable]
public class Item : Note
{
[DataMember]
public static List<Item> Items = new List<Item>();
[DataContract]
public enum ItemType
{
Group,
Note
}
[DataMember]
public ItemType Type { get; set; }
[DataMember]
public int Index { get; set; }
}
}
Note.cs:
using System;
using System.Collections.Generic;
using System.Linq;
using System.Runtime.Serialization;
using System.Text;
using System.Threading.Tasks;
namespace Notes
{
[DataContract]
public class Note
{
[DataMember]
public string Title { get; set; }
[DataMember]
public string Content { get; set; }
[DataMember]
public Group Group;
[DataContract]
public enum NoteImportance
{
Important,
Neutral,
NotImportant
}
[DataMember]
public NoteImportance Importance { get; set; }
[DataMember]
public bool Protected { get; set; }
}
}
How can I deserialize these objects/read from file and get them into a List or ListView? I've already done this, but for some reason it's not working anymore.
Any help would be appreciated.
When you create a .group file, you serialize a single Item:
DataContractJsonSerializer serializer = new DataContractJsonSerializer(typeof(Item));
// And later
serializer.WriteObject(stream, item);
But when you deserialize the contents of a .group file, you try to deserialize a List<Item>:
DataContractJsonSerializer serializer = new DataContractJsonSerializer(typeof(List<Item>));
// And later
foreach (Item item in (List<Item>)serializer.ReadObject(stream))
{
Item.Items.Add(item);
}
Those types don't match. But in order to deserialize the data you previously serialized, they need to match - or at least, the deserialized type cannot be a collection if the serialized type was, because collections are serialized as JSON arrays while other classes are serialized as JSON objects (name/value pairs).
Since it looks like each .group file has a single item, and there are many .group files in the directory you are scanning, you probably just want to do
var serializer = new DataContractJsonSerializer(typeof(Item));
// And later
var item = (Item)serializer.ReadObject(stream);
if (item != null)
Item.Items.Add(item);
I would like to save and load my History list filled with History Entry objects. I am trying to do this through Isolated Storage, so that when the user opens and closes the app none of their browsing history is lost. It is saved which can be loaded once the app is clicked. I have had a look around and saw this question on stackoverflow, and I have tried to follow it but came across so errors. Isolated Storage & Saving Multiple Objects.
Here is the code
The HistoryEntry class
public string URL { get; set; }
public string timestamp { get; set; }
public string date { get; set; }
The MainPage code:
using System.Xml.Serialization;
using System.Collections.ObjectModel;
using System.IO;
List<HistoryEntry> urls = new List<HistoryEntry>();
public HistoryEntry selectedHistory;
public MainPage()
{
InitializeComponent();
Deserialize<>(urls, ???);
}
void Browser_Navigated(object sender, System.Windows.Navigation.NavigationEventArgs e)
{
HistoryEntry urlObj = new HistoryEntry();
urlObj.URL = url;
urlObj.timestamp = DateTime.Now.ToString("HH:mm");
urlObj.date = url.Remove(url.LastIndexOf('.'));
urls.Add(urlObj);
textBox1.Text = url;
listBox.ItemsSource = null;
listBox.ItemsSource = urls;
Serialize(urlObj, urls);
}
private static void Serialize(string fileName, object source)
{
var userStore = IsolatedStorageFile.GetUserStoreForApplication();
using (var stream = new IsolatedStorageFileStream(fileName, FileMode.Create, userStore))
{
XmlSerializer serializer = new XmlSerializer(source.GetType());
serializer.Serialize(stream, source);
}
}
public static void Deserialize<T>(ObservableCollection<T> list, string filename)
{
list = new ObservableCollection<T>();
var userStore = IsolatedStorageFile.GetUserStoreForApplication();
if (userStore.FileExists(filename))
{
using (var stream = new IsolatedStorageFileStream(filename, FileMode.Open, userStore))
{
XmlSerializer serializer = new XmlSerializer(list.GetType());
var items = (ObservableCollection<T>)serializer.Deserialize(stream);
foreach (T item in items)
{
list.Add(item);
}
}
}
}
Serialize has some invalid arguments which is the same with when De serialize is called. What are the appropriate values to be sent to the method, and will this successfully save and load the history objects.
Thank you in advance :)
If you need any more details please comment and I will be happy to explain in further detail :)
Did you try this way using MemoryStream which it did work for Win 8 :
To Serialize:
MemoryStream sessionData = new MemoryStream();
DataContractSerializer serializer = new
DataContractSerializer(typeof(ObservableCollection<NewsByTag>));
serializer.WriteObject(sessionData, data);
StorageFile file = await ApplicationData.Current.LocalFolder
.CreateFileAsync(sFileName);
using (Stream fileStream = await file.OpenStreamForWriteAsync())
{
sessionData.Seek(0, SeekOrigin.Begin);
await sessionData.CopyToAsync(fileStream);
await fileStream.FlushAsync();
}
To Deserialize it back:
StorageFile file = await ApplicationData.Current.LocalFolder.
GetFileAsync(sFileName);
using (IInputStream inStream = await file.OpenSequentialReadAsync())
{
DataContractSerializer serializer =
new DataContractSerializer(typeof(ObservableCollection<NewsByTag>));
var data = (ObservableCollection<NewsByTag>)serializer
.ReadObject(inStream.AsStreamForRead());
}
Hope it helps!