I have problem with memory.
I want do add pictures to my excel. Picuters are on azure blob.
After 1k of added pictures my program slowing so much till outofmemory exception.
I will paste my code here:
string filePath = #"excel.xlsx";
int i = 0;
public void Generate(int worksheetnumber)
{
using (FileStream fileStream = File.Open(filePath, FileMode.Open))
{
var package = new ExcelPackage(fileStream);
ExcelWorksheet workSheet = package.Workbook.Worksheets[worksheetnumber];
workSheet.Column(2).Width = 21;
workSheet.Column(4).Width = 21;
for (int j = workSheet.Dimension.Start.Row + 1; j <= workSheet.Dimension.End.Row; j++)
{
InsertImage(workSheet, j, 1);
InsertImage(workSheet, j, 3);
}
string path = string.Format(#"{0}.xlsx", worksheetnumber);
Stream stream2 = File.Create(path);
package.SaveAs(stream2);
stream2.Close();
}
}
private void InsertImage(ExcelWorksheet workSheet, int j, int columnNumber)
{
if (!string.IsNullOrEmpty(workSheet.Cells[j, columnNumber].Text))
{
try
{
var splittedUrlHash = workSheet.Cells[j, columnNumber].Text.Split('/');
using (var stream = DownloadBlobFileToStream(splittedUrlHash[1], splittedUrlHash[0], SHOP))
{
Image image = Image.FromStream(stream);
var picture = workSheet.Drawings.AddPicture(string.Format("{0}{1}", j, columnNumber+1), image);
stream.Close();
picture.From.Column = columnNumber;
picture.From.Row = j - 1;
picture.To.Column = columnNumber;
picture.To.Row = j - 1;
picture.SetSize(150, 120);
workSheet.Row(j).Height = 100;
Debug.WriteLine("Dodano obrazek - {0}", i);
i++;
}
}
catch (System.ArgumentException e)
{
}
}
}
I want to admit, I have a link images in Columns 1 and 3. Images are locating in cols 2 and 4.
Is there any solution to handle this error in this case without losing images, or leaving empty cells
Related
I would like to know how to write to a .wav file, I have written the following code which supposedly writes data to the file. But when I try to play the sound file it says the file is corrupt / empty.
try
{
SaveFileDialog save = new SaveFileDialog();
save.Filter = "Wave File (*.wav)|*.wav;";
if (save.ShowDialog() != System.Windows.Forms.DialogResult.OK)
return;
using (FileStream FS = new FileStream(save.FileName, FileMode.Open, FileAccess.Write))
{
BinaryWriter wr = new BinaryWriter(FS);
int subchunk1Size = 16;
short audioFormat = 1;
short bitsPerSample = 64;
short numChannels = 2;
int sampleRate = Convert.ToInt32(samplingRateBox.Text);
int byteRate = sampleRate * numChannels * (bitsPerSample / 8);
short blockAlign = (short)(numChannels * (bitsPerSample / 8));
int numSamples = Convert.ToInt32(numberOfsamplesBox.Text);
int subChunk2Size = numSamples * numChannels * (bitsPerSample / 8);
int chunkSize = 4 + (8 + subchunk1Size) + (8 + subChunk2Size);
wr.Write(getBytes("RIFF"));
wr.Write(chunkSize);
wr.Write(getBytes("WAVE"));
wr.Write(getBytes("fmt"));
wr.Write((byte)32);
wr.Write(subchunk1Size);
wr.Write(audioFormat);
wr.Write(numChannels);
wr.Write(sampleRate);
wr.Write(byteRate);
wr.Write(blockAlign);
wr.Write(bitsPerSample);
wr.Write(getBytes("data"));
wr.Write(subChunk2Size);
double[] primArray = new double[samples.Length];
byte[] byteArray = new byte[samples.Length * 8];
for (int i = 0; i < samples.Length; i++)
{
primArray[i] = Convert.ToDouble(samples[i].Real);
}
byteArray = doubleToBytes(primArray);
for (int i = 0; i < samples.Length; i++)
{
wr.Write(byteArray[i]);
}
for (int i = 0; i < samples.Length; i++)
{
primArray[i] = Convert.ToDouble(samples[i].Imaginary);
}
byteArray = doubleToBytes(primArray);
for (int i = 0; i < samples.Length; i++)
{
wr.Write(byteArray[i]);
}
wr.Close();
wr.Dispose();
}
}
catch (Exception e)
{
MessageBox.Show(e.Message);
}
As you can see I have tried converting from Complex to double, plus the header.
The sampling rate and number of samples all come from textboxes. Plus I am assuming the bit depth is that of a double.
I have also tried using this method, however the length of the audio file is 0.
This used the NAudio library.
NAudio.Wave.WaveFileWriter waveWriter = null;
WaveIn wi = new WaveIn();
double[] primArray = new double[samples.Length];
for (int i = 0; i < samples.Length; i++)
{
primArray[i] = Convert.ToDouble(samples[i].Real);
}
SaveFileDialog save = new SaveFileDialog();
save.Filter = "Wave File (*.wav)|*.wav;";
if (save.ShowDialog() != System.Windows.Forms.DialogResult.OK)
return;
wi = new NAudio.Wave.WaveIn();
wi.WaveFormat = new WaveFormat(samplingRate,1);
waveWriter = new WaveFileWriter(save.FileName, wi.WaveFormat);
byte[] byteArray = new byte[samples.Length*8];
byteArray = doubleToBytes(primArray);
waveWriter.Write(byteArray, 0, byteArray.Length);
I am currently exploring neural networks and machine learning and I implemented a basic neural network in c#. Now I wanted to test my back propagation training algorithm with the MNIST database. Although I am having serious trouble reading the files correctly.
Spoiler the code is currently very badly optimised for performance. My aim currently is to grasp the subject and get a structured view how things work before I start throwing out my data structures for faster ones.
To train the network I want to feed it a custom TrainingSet data structure:
[Serializable]
public class TrainingSet
{
public Dictionary<List<double>, List<double>> data = new Dictionary<List<double>, List<double>>();
}
Keys will be my input data (784 pixels per entry(image) which will represent the greyscale values in range from 0 to 1). Values will be my output data (10 entries representing the digits from 0-9 with all entries on 0 except the exspected one at 1)
Now I want to read the MNIST database according to this contract. I am currentl on my 2nd try which is inspired by this blogpost: https://jamesmccaffrey.wordpress.com/2013/11/23/reading-the-mnist-data-set-with-c/ . Sadly it is still producing the same nonsense as my first try scattering the pixels in a strange pattern:
My current reading algorithm:
public static TrainingSet GenerateTrainingSet(FileInfo imagesFile, FileInfo labelsFile)
{
MnistImageView imageView = new MnistImageView();
imageView.Show();
TrainingSet trainingSet = new TrainingSet();
List<List<double>> labels = new List<List<double>>();
List<List<double>> images = new List<List<double>>();
using (BinaryReader brLabels = new BinaryReader(new FileStream(labelsFile.FullName, FileMode.Open)))
{
using (BinaryReader brImages = new BinaryReader(new FileStream(imagesFile.FullName, FileMode.Open)))
{
int magic1 = brImages.ReadBigInt32(); //Reading as BigEndian
int numImages = brImages.ReadBigInt32();
int numRows = brImages.ReadBigInt32();
int numCols = brImages.ReadBigInt32();
int magic2 = brLabels.ReadBigInt32();
int numLabels = brLabels.ReadBigInt32();
byte[] pixels = new byte[numRows * numCols];
// each image
for (int imageCounter = 0; imageCounter < numImages; imageCounter++)
{
List<double> imageInput = new List<double>();
List<double> exspectedOutput = new List<double>();
for (int i = 0; i < 10; i++) //generate empty exspected output
exspectedOutput.Add(0);
//read image
for (int p = 0; p < pixels.Length; p++)
{
byte b = brImages.ReadByte();
pixels[p] = b;
imageInput.Add(b / 255.0f); //scale in 0 to 1 range
}
//read label
byte lbl = brLabels.ReadByte();
exspectedOutput[lbl] = 1; //modify exspected output
labels.Add(exspectedOutput);
images.Add(imageInput);
//Debug view showing parsed image.......................
Bitmap image = new Bitmap(numCols, numRows);
for (int y = 0; y < numRows; y++)
{
for (int x = 0; x < numCols; x++)
{
image.SetPixel(x, y, Color.FromArgb(255 - pixels[x * y], 255 - pixels[x * y], 255 - pixels[x * y])); //invert colors to have 0,0,0 be white as specified by mnist
}
}
imageView.SetImage(image);
imageView.Refresh();
//.......................................................
}
brImages.Close();
brLabels.Close();
}
}
for (int i = 0; i < images.Count; i++)
{
trainingSet.data.Add(images[i], labels[i]);
}
return trainingSet;
}
All images produce a pattern as shown above. It's never the exact same pattern but always seems to have the pixels "pulled" down to the right corner.
That is how I did it:
public static class MnistReader
{
private const string TrainImages = "mnist/train-images.idx3-ubyte";
private const string TrainLabels = "mnist/train-labels.idx1-ubyte";
private const string TestImages = "mnist/t10k-images.idx3-ubyte";
private const string TestLabels = "mnist/t10k-labels.idx1-ubyte";
public static IEnumerable<Image> ReadTrainingData()
{
foreach (var item in Read(TrainImages, TrainLabels))
{
yield return item;
}
}
public static IEnumerable<Image> ReadTestData()
{
foreach (var item in Read(TestImages, TestLabels))
{
yield return item;
}
}
private static IEnumerable<Image> Read(string imagesPath, string labelsPath)
{
BinaryReader labels = new BinaryReader(new FileStream(labelsPath, FileMode.Open));
BinaryReader images = new BinaryReader(new FileStream(imagesPath, FileMode.Open));
int magicNumber = images.ReadBigInt32();
int numberOfImages = images.ReadBigInt32();
int width = images.ReadBigInt32();
int height = images.ReadBigInt32();
int magicLabel = labels.ReadBigInt32();
int numberOfLabels = labels.ReadBigInt32();
for (int i = 0; i < numberOfImages; i++)
{
var bytes = images.ReadBytes(width * height);
var arr = new byte[height, width];
arr.ForEach((j,k) => arr[j, k] = bytes[j * height + k]);
yield return new Image()
{
Data = arr,
Label = labels.ReadByte()
};
}
}
}
Image class:
public class Image
{
public byte Label { get; set; }
public byte[,] Data { get; set; }
}
Some extension methods:
public static class Extensions
{
public static int ReadBigInt32(this BinaryReader br)
{
var bytes = br.ReadBytes(sizeof(Int32));
if (BitConverter.IsLittleEndian) Array.Reverse(bytes);
return BitConverter.ToInt32(bytes, 0);
}
public static void ForEach<T>(this T[,] source, Action<int, int> action)
{
for (int w = 0; w < source.GetLength(0); w++)
{
for (int h = 0; h < source.GetLength(1); h++)
{
action(w, h);
}
}
}
}
Usage:
foreach (var image in MnistReader.ReadTrainingData())
{
//use image here
}
or
foreach (var image in MnistReader.ReadTestData())
{
//use image here
}
Why not use a nuget package:
MNIST.IO Just a datareader (disclaimer: my package)
Accord.DataSets Contains classes to download and parse machine learning datasets such as MNIST, News20, Iris. This package is part of the Accord.NET Framework.
I've put the .AutoSizeColumn right before the write Method
int numberOfColumns = sheet.GetRow(rowcount - 1).PhysicalNumberOfCells;
for (int i = 0; i <= numberOfColumns; i++)
{
sheet.AutoSizeColumn(i);
GC.Collect();
}
using (var fileData = new FileStream(#"C:\Temp\Contatti.xlsx", FileMode.Create))
{
wb.Write(fileData);
}
this is an example of the result
The problem also migh be, that PhysicalNumberOfCells can return 1, even if you have a cell lets say in 'Z' column. There is LastCellNum property,you i instead of PhysicalNumberOfCells:
int lastColumNum = sheet.GetRow(0).LastCellNum;
for (int i = 0; i <= lastColumNum; i++)
{
sheet.AutoSizeColumn(i);
GC.Collect();
}
using (var fileData = new FileStream(#"D:\Contatti.xlsx", FileMode.Create))
{
wb.Write(fileData);
}
I need a help with iText in C#. I'm trying to extract a table from a PDF file and save this into a new CSV file, keeping the values in the correct places. For this, I thought the solution was to create a two-dimensional array to organize the data.
Extracting all information from PDF with iText, I saw it was possible to get some numerical data that seemed to be the position of a piece of text on the page and I organized my array based these indexes. It didn’t work, the text was completely dispersed in various different cells. Now, I want to know what this values means, because they don't follow a "correct" order and I want to know if is possible to organize the future table with this.
I'm using ";" as delimiter cell.
For testing, I'm using this PDF
http://www.americana.sp.gov.br/americanaV5/download/contasPublicas/Despesa_Categoria_Economica_2014.pdf
Here's my code:
protected void Button2_Click(object sender, EventArgs e)
{
try
{
TextBox2.Text = "";
byte[] conteudo = download(TextBox1.Text);
if (conteudo != null)
{
PdfReader leitorp = new PdfReader(conteudo);
ITextExtractionStrategy estrategia = new SimpleTextExtractionStrategy();
List<Celula> celulas = new List<Celula>();
int i, j;
for (i = 1; i <= leitorp.NumberOfPages; i++)
{
//Total and crude extraction of all information from text in PDF via iText, separate lines in an array of strings.
string[] linhas = (Encoding.UTF8.GetString(Encoding.Convert(Encoding.Default, Encoding.UTF8, leitorp.GetPageContent(i)))).Split('\n');
for (j = 1; j < linhas.Length; j++)
{
if (linhas[j].Length > 2)
{
if (linhas[j].Substring(0, 2).Equals("BT"))
{
string[] campos = linhas[j].Split(' ');
Celula umacelula = new Celula();
umacelula.coluna = float.Parse(campos[1]);
umacelula.linha = float.Parse(campos[2]);
linhadodebug = j;
int t1 = linhas[j].IndexOf('(');
int t2 = linhas[j].LastIndexOf(')');
umacelula.conteudo = System.Text.RegularExpressions.Regex.Replace((linhas[j].Substring(linhas[j].IndexOf('(') + 1, (linhas[j].LastIndexOf(')') - 1 - linhas[j].IndexOf('(')))), #"\s\s+", "");
celulas.Add(umacelula);
}
}
}
}
leitorp.Close();
string[] totallinhas = new string[celulas.Count];
string[] totalcolunas = new string[celulas.Count];
for (i = 0; i < celulas.Count; i++)
{
totallinhas[i] = celulas[i].linha.ToString();
totalcolunas[i] = celulas[i].coluna.ToString();
}
totallinhas = totallinhas.Distinct().ToArray();
totalcolunas = totalcolunas.Distinct().ToArray();
Array.Sort(totallinhas);
Array.Reverse(totallinhas);
Array.Sort(totalcolunas);
Array.Reverse(totalcolunas);
string[,] matriz = new string[totallinhas.Length + 1, totalcolunas.Length + 1];
for (i = 1; i < totallinhas.Length; i++)
{
matriz[i, 0] = totallinhas[i - 1].ToString();
}
for (i = 1; i < totalcolunas.Length; i++)
{
matriz[0, i] = totalcolunas[i - 1].ToString();
}
int z;
for (i = 0; i < celulas.Count(); i++)
{
for (j = 1; j < matriz.GetLength(0); j++)
{
for (z = 1; z < matriz.GetLength(1); z++)
{
if ((celulas[i].linha.ToString().Equals(matriz[j, 0])) && (celulas[i].coluna.ToString().Equals(matriz[0, z])))
{
matriz[j, z] = celulas[i].conteudo.ToString();
}
}
}
}
StringWriter texto = new StringWriter();
for (i = 0; i < matriz.GetLength(0); i++)
{
for (j = 0; j < matriz.GetLength(1); j++)
{
texto.Write(matriz[i, j] + ";");
}
texto.WriteLine();
}
Response.ContentType = "text/plain";
Response.AddHeader("content-disposition", "attachment;filename=" + string.Format("teste-{0}.csv", string.Format("{0:ddMMyyyy}", DateTime.Today)));
Response.Clear();
using (StreamWriter writer = new StreamWriter(Response.OutputStream, Encoding.UTF8))
{
writer.Write(texto.ToString());
}
Response.End();
}
}
catch (Exception E)
{
TextBox2.Text = "Erro Button2_Click: " + E.Message + " # " + linhadodebug.ToString();
}
}
And here, the struct of celula (cell) and method to download the file:
public struct Celula
{
public float coluna;
public float linha;
public string conteudo;
public Celula(float coluna, float linha, string conteudo)
{
this.coluna = coluna;
this.linha = linha;
this.conteudo = conteudo;
}
public Celula(Celula celula)
{
this.coluna = celula.coluna;
this.linha = celula.linha;
this.conteudo = celula.conteudo;
}
}
protected byte[] download(string url)
{
try
{
WebRequest endereco = HttpWebRequest.Create(url);
Stream leitor = endereco.GetResponse().GetResponseStream();
MemoryStream memoria = new MemoryStream();
byte[] conteudo = null;
int count = 0;
do
{
byte[] buffer = new byte[1024];
count = leitor.Read(buffer, 0, 1024);
memoria.Write(buffer, 0, count);
}
while (leitor.CanRead && count > 0);
// Converte da memória direto para bytes
conteudo = memoria.ToArray();
if (conteudo != null)
{
return conteudo;
}
else
{
TextBox2.Text = "Error: download null.";
return null;
}
}
catch (Exception E)
{
TextBox2.Text = "Error download: " + E.Message;
return null;
}
}
This is a non-profit project. I hope you can help me. Thank you!
private void write(object sender, EventArgs e)
{
FileStream outputFileStream = new FileStream("test.txt", FileMode.Create, FileAccess.Write);
StreamWriter writer = new StreamWriter(outputFileStream);
Random r = new Random();
for (int i = 0; i < dataGridView1.RowCount; i++)
{
for (int j = 0; j < dataGridView1.ColumnCount; j++)
{
double d = 0;
Double.TryParse(Convert.ToString(dataGridView1.Rows[i].Cells[j].Value), out d);
writer.Write(d + "\t");
}
writer.Write("\n");
}
writer.Close();
outputFileStream.Close();
}
So this is the method to write the text file. It works fine because I have opened it successfully with an Excel. I even tried to copy and paste it and it would work. Now the problem is.....
private void read(object sender, EventArgs e)
{
char TAB = '\t';
char NEWLINE = '\n';
FileStream inputFileStream = new FileStream("test.txt", FileMode.Open, FileAccess.Read);
StreamReader reader = new StreamReader(inputFileStream);
string line;
string[] fields;
for (int i = 0; i < dataGridView1.RowCount; i++)
{
for (int j = 0; j < dataGridView1.ColumnCount; j++)
{
line = reader.ReadLine();
fields = line.Split(TAB, NEWLINE);
dataGridView1.Rows[i].Cells[j].Value = fields;
}
}
inputFileStream.Close();
reader.Close();
}
However when I read the file back onto a DataGridView it does not work properly. Now this is the exact text file that I wrote on my code. Instead what happens is that it is displayed on only 1 row. How do I get back the amount of columns and rows from what the user entered? I prefer keeping it a text file.
I have used default properties for my dataGridView1
RowCount returns displayed rows count. So here it returns 1 because you are displaying one row and it's empty:
for (int i = 0; i < dataGridView1.RowCount; i++)
Instead of this, you should create new rows and add it to the Rows collection like this:
// Use File.ReadAllLines, it's easier
string[] lines = File.ReadAllLines("test.txt");
foreach(line in lines)
{
var text = line.Split('\t','\n');
dataGridView1.Rows.Add(text);
}
System.IO.StreamReader file = new System.IO.StreamReader("yourfile.txt");
string[] columnnames = file.ReadLine().Split(' ');
DataTable dt = new DataTable();
foreach (string c in columnnames)
{
dt.Columns.Add(c);
}
string newline;
while ((newline = file.ReadLine()) != null)
{
DataRow dr = dt.NewRow();
string[] values = newline.Split(' ');
for (int i = 0; i < values.Length; i++)
{
dr[i] = values[i];
}
dt.Rows.Add(dr);
}
file.Close();
dataGridView1.DataSource = dt;
try this one with your own delimiters i.e. \t and \n. And First try to search your problems on internet before posting queries and try to solve them by yourself. I am not giving full code.