Develop Reference

Two streams of writing to file

When the writing starts from one file to the resulting file, the entry passes, and from the second file to this result file is blocked. So in two streams record can not be available in the same time. Error: a process can not access a file because this file is being used by another process. How I can resolve this problem. I tried diffrent ways. For example, If you write directly not to a file from two streams, but add string to some array, and then at the end go through the array and write everything to a file. It must be without synschronized.
class Program
{
public static void WriteOneThread()
{
List<char> listOfChars = new List<char>();
FileInfo file = new FileInfo(#"C:\test\task.txt");
FileStream stream = file.Open(FileMode.Open, FileAccess.ReadWrite);
StreamReader reader = new StreamReader(stream);
string str = reader.ReadToEnd();
Console.WriteLine(str);
char[] arr = str.ToCharArray();
for (int i = 0; i < arr.Length; i++)
{
listOfChars.Add(arr[i]);
}
Console.WriteLine(new string('-', 20));
FileInfo file2 = new FileInfo(#"C:\test\Result.txt");
try
{
foreach (var value in listOfChars)
{
StreamWriter writer = file2.CreateText();
writer.WriteLine(value);
writer.Close();
}
}
catch (Exception e)
{
Console.WriteLine(e.Message);
}
}
public static void WriteSecondThread()
{
List<char> listOfChars = new List<char>();
FileInfo file = new FileInfo(#"C:\test\pretask.txt");
FileStream stream = file.Open(FileMode.Open, FileAccess.ReadWrite);
StreamReader reader = new StreamReader(stream);
string str = reader.ReadToEnd();
Console.WriteLine(str);
char[] arr = str.ToCharArray();
for (int i = 0; i < arr.Length; i++)
{
listOfChars.Add(arr[i]);
}
Console.WriteLine(new string('-', 20));
FileInfo file2 = new FileInfo(#"C:\test\Result.txt");
try
{
foreach (var value in listOfChars)
{
StreamWriter writer = file2.CreateText();
writer.WriteLine(value);
writer.Close();
}
}
catch (Exception e)
{
Console.WriteLine(e.Message);
}
}
static void Main(string[] args)
{
Thread threadOne = new Thread(WriteOneThread);
Thread threadSecond = new Thread(WriteSecondThread);
threadOne.Start();
Thread.Sleep(10000);
threadSecond.Start();
threadOne.Join();
threadSecond.Join();
Console.WriteLine("Запись в файл осуществлена");
Console.ReadKey();
}
}

C# StreamWriter won't work after calling StreamReader

I'm trying to make betting program in C#, storing the user's data in a txt file. I have no problem reading the data from it. However, I can't manage to overwrite it.
From what I've tested, if I call the StreamWriter part alone the overwriting happens just fine. When I put the same code after the StreamReader part, the code will reach the Console.WriteLine("reached"); line and ignore everything after it (username is never written in the console). No error is detected and compilation won't stop either.
Here's the class code:
class Dinero
{
private List<string> data;
private string path = #"C:\Users\yy\Documents\Visual Studio 2015\Projects\ErikaBot\ErikaBot\img\bank_data.txt";
...
some other methods here
...
public void thing(string username, int money)
{
FileStream fs = new FileStream(path, FileMode.OpenOrCreate, FileAccess.ReadWrite, FileShare.None);
data = new List<string>();
using (StreamReader sr = new StreamReader(fs))
{
string a = sr.ReadLine();
for (int i = 0; a != null; i++)
{
if (a != username)
{
data.Add(a);
}
else i++;
a = sr.ReadLine();
}
}
string b = Convert.ToString(money);
Console.WriteLine("reached");
using (StreamWriter tw = new StreamWriter(fs))
{
Console.WriteLine(username);
if (data != null)
{
for (int i = 0; i < data.Count; i++)
{
tw.WriteLine(data.ElementAt(i));
}
}
string money2 = Convert.ToString(money);
tw.WriteLine(username);
tw.WriteLine(money2);
}
}
}

By disposing StreamReader you also dispose the FileStream.
Either repeat the filestream initialisation before the using statement for StreamWriter or put the latter in the using statement for StreamReader.

C# ZipArchive losing data

I'm trying to copy the contents of one Excel file to another Excel file while replacing a string inside of the file on the copy. It's working for the most part, but the file is losing 27 kb of data. Any suggestions?
public void ReplaceString(string what, string with, string path) {
List < string > doneContents = new List < string > ();
List < string > doneNames = new List < string > ();
using(ZipArchive archive = ZipFile.Open(_path, ZipArchiveMode.Read)) {
int count = archive.Entries.Count;
for (int i = 0; i < count; i++) {
ZipArchiveEntry entry = archive.Entries[i];
using(var entryStream = entry.Open())
using(StreamReader reader = new StreamReader(entryStream)) {
string txt = reader.ReadToEnd();
if (txt.Contains(what)) {
txt = txt.Replace(what, with);
}
doneContents.Add(txt);
string name = entry.FullName;
doneNames.Add(name);
}
}
}
using(MemoryStream zipStream = new MemoryStream()) {
using(ZipArchive newArchive = new ZipArchive(zipStream, ZipArchiveMode.Create, true, Encoding.UTF8)) {
for (int i = 0; i < doneContents.Count; i++) {
int spot = i;
ZipArchiveEntry entry = newArchive.CreateEntry(doneNames[spot]);
using(var entryStream = entry.Open())
using(var sw = new StreamWriter(entryStream)) {
sw.Write(doneContents[spot]);
}
}
}
using(var fileStream = new FileStream(path, FileMode.Create)) {
zipStream.Seek(0, SeekOrigin.Begin);
zipStream.CopyTo(fileStream);
}
}
}
I've used Microsoft's DocumentFormat.OpenXML and Excel Interop, however, they are both lacking in a few main components that I need.
Update:
using(var fileStream = new FileStream(path, FileMode.Create)) {
var wrapper = new StreamWriter(fileStream);
wrapper.AutoFlush = true;
zipStream.Seek(0, SeekOrigin.Begin);
zipStream.CopyTo(wrapper.BaseStream);
wrapper.Flush();
wrapper.Close();
}

Try the process without changing the string and see if the file size is the same. If so then it would seem that your copy is working correctly, however as Marc B suggested, with compression, even a small change can result in a larger change in the overall size.

Serializing a ConcurrentBag of XAML

I have, in my code, a ConcurrentBag<Point3DCollection>.
I'm trying to figure out how to serialize them. Of course I could iterate through or package it with a provider model class, but I wonder if it's already been done.
The Point3DCollections themselves are potentially quite large and could stand to be compressed to speed up reading and writing to and from the disk, but the response times I need for this are largely in the user interface scale. In other words, I prefer a binary formatting over a XAML-text formatting, for performance reasons. (There is a nice XAML-text serializer which is part of the Helix 3D CodeProject, but it's slower than I'd like.)
Is this a use case where I'm left rolling out my own serializer, or is there something out there that's already packaged for this kind of data?

Here are some extensions methods that handle string and binary serialization of Point3DCollection bags. As I said in my comment, I don't think there is a best way of doing this in all cases, so you might want to try both. Also note they're using Stream parameter as input so you can chain these with calls to GZipStream of DeflateStream.
public static class Point3DExtensions
{
public static void StringSerialize(this ConcurrentBag<Point3DCollection> bag, Stream stream)
{
if (bag == null)
throw new ArgumentNullException("bag");
if (stream == null)
throw new ArgumentNullException("stream");
StreamWriter writer = new StreamWriter(stream);
Point3DCollectionConverter converter = new Point3DCollectionConverter();
foreach (Point3DCollection coll in bag)
{
// we need to use the english locale as the converter needs that for parsing...
string line = (string)converter.ConvertTo(null, CultureInfo.GetCultureInfo("en-US"), coll, typeof(string));
writer.WriteLine(line);
}
writer.Flush();
}
public static void StringDeserialize(this ConcurrentBag<Point3DCollection> bag, Stream stream)
{
if (bag == null)
throw new ArgumentNullException("bag");
if (stream == null)
throw new ArgumentNullException("stream");
StreamReader reader = new StreamReader(stream);
Point3DCollectionConverter converter = new Point3DCollectionConverter();
do
{
string line = reader.ReadLine();
if (line == null)
break;
bag.Add((Point3DCollection)converter.ConvertFrom(line));
// NOTE: could also use this:
//bag.Add(Point3DCollection.Parse(line));
}
while (true);
}
public static void BinarySerialize(this ConcurrentBag<Point3DCollection> bag, Stream stream)
{
if (bag == null)
throw new ArgumentNullException("bag");
if (stream == null)
throw new ArgumentNullException("stream");
BinaryWriter writer = new BinaryWriter(stream);
writer.Write(bag.Count);
foreach (Point3DCollection coll in bag)
{
writer.Write(coll.Count);
foreach (Point3D point in coll)
{
writer.Write(point.X);
writer.Write(point.Y);
writer.Write(point.Z);
}
}
writer.Flush();
}
public static void BinaryDeserialize(this ConcurrentBag<Point3DCollection> bag, Stream stream)
{
if (bag == null)
throw new ArgumentNullException("bag");
if (stream == null)
throw new ArgumentNullException("stream");
BinaryReader reader = new BinaryReader(stream);
int count = reader.ReadInt32();
for (int i = 0; i < count; i++)
{
int pointCount = reader.ReadInt32();
Point3DCollection coll = new Point3DCollection(pointCount);
for (int j = 0; j < pointCount; j++)
{
coll.Add(new Point3D(reader.ReadDouble(), reader.ReadDouble(), reader.ReadDouble()));
}
bag.Add(coll);
}
}
}
And a little console app test program to play with:
static void Main(string[] args)
{
Random rand = new Random(Environment.TickCount);
ConcurrentBag<Point3DCollection> bag = new ConcurrentBag<Point3DCollection>();
for (int i = 0; i < 100; i++)
{
Point3DCollection coll = new Point3DCollection();
bag.Add(coll);
for (int j = rand.Next(10); j < rand.Next(100); j++)
{
Point3D point = new Point3D(rand.NextDouble(), rand.NextDouble(), rand.NextDouble());
coll.Add(point);
}
}
using (FileStream stream = new FileStream("test.bin", FileMode.Create))
{
bag.StringSerialize(stream); // or Binary
}
ConcurrentBag<Point3DCollection> newbag = new ConcurrentBag<Point3DCollection>();
using (FileStream stream = new FileStream("test.bin", FileMode.Open))
{
newbag.StringDeserialize(stream); // or Binary
foreach (Point3DCollection coll in newbag)
{
foreach (Point3D point in coll)
{
Console.WriteLine(point);
}
Console.WriteLine();
}
}
}
}

Compression could potentially take advantage of repeated coordinates. Serializers will often use references for repeat objects as well, although I'm not sure there are many set up to work with structs (like Point3D). Anyhow, here are some examples of how to serialize this. To use the standard formatters, you need to convert the data type to something most of them support: list/array. The code below uses Nuget packages NUnit and Json.NET.
using Newtonsoft.Json;
using Newtonsoft.Json.Bson;
using NUnit.Framework;
using System;
using System.Collections.Concurrent;
using System.Collections.Generic;
using System.Diagnostics;
using System.IO;
using System.IO.Compression;
using System.Linq;
using System.Runtime.Serialization.Formatters.Binary;
using System.Text;
using System.Windows.Media.Media3D;
namespace DemoPoint3DSerialize
{
[TestFixture]
class Tests
{
[Test]
public void DemoBinary()
{
// this shows how to convert them all to strings
var collection = CreateCollection();
var data = collection.Select(c => c.ToArray()).ToList(); // switch to serializable types
var formatter = new BinaryFormatter();
using (var ms = new MemoryStream())
{
formatter.Serialize(ms, data);
Trace.WriteLine("Binary of Array Size: " + ms.Position);
ms.Position = 0;
var dupe = (List<Point3D[]>)formatter.Deserialize(ms);
var result = new ConcurrentBag<Point3DCollection>(dupe.Select(r => new Point3DCollection(r)));
VerifyEquality(collection, result);
}
}
[Test]
public void DemoString()
{
// this shows how to convert them all to strings
var collection = CreateCollection();
IEnumerable<IList<Point3D>> tmp = collection;
var strings = collection.Select(c => c.ToString()).ToList();
Trace.WriteLine("String Size: " + strings.Sum(s => s.Length)); // eh, 2x for Unicode
var result = new ConcurrentBag<Point3DCollection>(strings.Select(r => Point3DCollection.Parse(r)));
VerifyEquality(collection, result);
}
[Test]
public void DemoDeflateString()
{
// this shows how to convert them all to strings
var collection = CreateCollection();
var formatter = new BinaryFormatter(); // not really helping much: could
var strings = collection.Select(c => c.ToString()).ToList();
using (var ms = new MemoryStream())
{
using (var def = new DeflateStream(ms, CompressionLevel.Optimal, true))
{
formatter.Serialize(def, strings);
}
Trace.WriteLine("Deflate Size: " + ms.Position);
ms.Position = 0;
using (var def = new DeflateStream(ms, CompressionMode.Decompress))
{
var stringsDupe = (IList<string>)formatter.Deserialize(def);
var result = new ConcurrentBag<Point3DCollection>(stringsDupe.Select(r => Point3DCollection.Parse(r)));
VerifyEquality(collection, result);
}
}
}
[Test]
public void DemoStraightJson()
{
// this uses Json.NET
var collection = CreateCollection();
var formatter = new JsonSerializer();
using (var ms = new MemoryStream())
{
using (var stream = new StreamWriter(ms, new UTF8Encoding(true), 2048, true))
using (var writer = new JsonTextWriter(stream))
{
formatter.Serialize(writer, collection);
}
Trace.WriteLine("JSON Size: " + ms.Position);
ms.Position = 0;
using (var stream = new StreamReader(ms))
using (var reader = new JsonTextReader(stream))
{
var result = formatter.Deserialize<List<Point3DCollection>>(reader);
VerifyEquality(collection, new ConcurrentBag<Point3DCollection>(result));
}
}
}
[Test]
public void DemoBsonOfArray()
{
// this uses Json.NET
var collection = CreateCollection();
var formatter = new JsonSerializer();
using (var ms = new MemoryStream())
{
using (var stream = new BinaryWriter(ms, new UTF8Encoding(true), true))
using (var writer = new BsonWriter(stream))
{
formatter.Serialize(writer, collection);
}
Trace.WriteLine("BSON Size: " + ms.Position);
ms.Position = 0;
using (var stream = new BinaryReader(ms))
using (var reader = new BsonReader(stream, true, DateTimeKind.Unspecified))
{
var result = formatter.Deserialize<List<Point3DCollection>>(reader); // doesn't seem to read out that concurrentBag
VerifyEquality(collection, new ConcurrentBag<Point3DCollection>(result));
}
}
}
private ConcurrentBag<Point3DCollection> CreateCollection()
{
var rand = new Random(42);
var bag = new ConcurrentBag<Point3DCollection>();
for (int i = 0; i < 10; i++)
{
var collection = new Point3DCollection();
for (int j = 0; j < i + 10; j++)
{
var point = new Point3D(rand.NextDouble(), rand.NextDouble(), rand.NextDouble());
collection.Add(point);
}
bag.Add(collection);
}
return bag;
}
private class CollectionComparer : IEqualityComparer<Point3DCollection>
{
public bool Equals(Point3DCollection x, Point3DCollection y)
{
return x.SequenceEqual(y);
}
public int GetHashCode(Point3DCollection obj)
{
return obj.GetHashCode();
}
}
private void VerifyEquality(ConcurrentBag<Point3DCollection> collection, ConcurrentBag<Point3DCollection> result)
{
var first = collection.OrderBy(c => c.Count);
var second = collection.OrderBy(c => c.Count);
first.SequenceEqual(second, new CollectionComparer());
}
}
}

Use Google's protobuf-net. protobuf-net is an open source .net implementation of Google's protocol buffer binary serialization format which can be used as a replacement for the BinaryFormatter serializer. It is probably going to be the fastest solution and easiest to implement.
Here is a link to the the main google wiki for protobuf-net. On the left, you'll find the downloads for all of the most updated binaries.
https://code.google.com/p/protobuf-net/
Here is a great article that you might want to look at first to get a feel for how it works.
http://wallaceturner.com/serialization-with-protobuf-net
Here is a link to a discussion on google's wiki about your specific problem. The answer is at the bottom of the page. That's where I got the code below and substituted with details from your post.
https://code.google.com/p/protobuf-net/issues/detail?id=354
I haven't used it myself but it looks like a very good solution to your stated needs. From what I gather, your code would end up some variation of this.
[ProtoContract]
public class MyClass {
public ConcurrentQueue<Point3DCollection> Points {get;set;}
[ProtoMember(1)]
private Point3DCollection[] Items
{
get { return Points.ToArray(); }
set { Items = new ConcurrentBag<Point3DCollection>(value); }
}
}
I wish you the best of luck. Take care.

For a large amount of data, why don't you consider Sqlite or any other small database system etc, which can store structured data in the file.
I have seen many 3d programs using database to store structure along with relations, which allow them to partially insert/update/delete data.
Benefit of Sqlite/database will be multithreaded serialization to improve speed, however you need to do little bit of work on sqlite to enable multi threaded sqlite connection, or else you can use LocalDB of SQL Express or even Sql Compact.
Also some of workload of loading data can be done through queries, which will be indexed by database nicely. And most of things can be done on background worker without interfering with User Interface.
Sqlite has limited multi-thread support, which can be explored here http://www.sqlite.org/threadsafe.html
Sql Compact is thread safe and requires installation that can be installed without admin priviledges. And you can use Entity framework as well.

Writing to MemoryStream with StreamWriter returns empty

I am not sure what I am doing wrong, have seen a lot of examples, but can't seem to get this working.
public static Stream Foo()
{
var memStream = new MemoryStream();
var streamWriter = new StreamWriter(memStream);
for (int i = 0; i < 6; i++)
streamWriter.WriteLine("TEST");
memStream.Seek(0, SeekOrigin.Begin);
return memStream;
}
I am doing a simple test on this method to try and get it to pass, but no matter what, my collection count is 0.
[Test]
public void TestStreamRowCount()
{
var stream = Foo();
using (var reader = new StreamReader(stream))
{
var collection = new List<string>();
string input;
while ((input = reader.ReadLine()) != null)
collection.Add(input);
Assert.AreEqual(6, collection.Count);
}
}
Note: I changed some syntax above without compiling in the Test method. What is more important is the first method which seems to be returning an empty stream (my reader.ReadLine() always reads once). Not sure what I am doing wrong. Thank you.

You are forgetting to flush your StreamWriter instance.
public static Stream Foo()
{
var memStream = new MemoryStream();
var streamWriter = new StreamWriter(memStream);
for (int i = 0; i < 6; i++)
streamWriter.WriteLine("TEST");
streamWriter.Flush(); <-- need this
memStream.Seek(0, SeekOrigin.Begin);
return memStream;
}
Also note that StreamWriter is supposed to be disposed of, since it implements IDisposable, but that in turn generates another problem, it will close the underlying MemoryStream as well.
Are you sure you want to return a MemoryStream here?
I would change the code to this:
public static byte[] Foo()
{
using (var memStream = new MemoryStream())
using (var streamWriter = new StreamWriter(memStream))
{
for (int i = 0; i < 6; i++)
streamWriter.WriteLine("TEST");
streamWriter.Flush();
return memStream.ToArray();
}
}
[Test]
public void TestStreamRowCount()
{
var bytes = Foo();
using (var stream = new MemoryStream(bytes))
using (var reader = new StreamReader(stream))
{
var collection = new List<string>();
string input;
while ((input = reader.ReadLine()) != null)
collection.Add(input);
Assert.AreEqual(6, collection.Count);
}
}

Since you are not using "using" or streamWriter.Flush() the writer did not commit changes to the stream. As result Stream itslef does not have data yet. In general you want to wrap manipulation with Stream and StremaWriter instances with using.
You also should consider returning new instance of MemoryStream:
using(var memStream = new MemoryStream())
{
....
return new MemoryStream(memStream.ToArray(), false /*writable*/);
}

Try flushing streamWriter after writing your lines.