Develop Reference

C# out of memory with file

I'm getting a OutOfMemory exception when running the following code, it happens on the File.ReadLines line, it processes most files fine until it hits larger files.
It's consistantly using tons of memory and cpu during the whole process though.
The file it crashed on is only 156,000KB, which is 156mb
static void Main(string[] args)
{
Console.CursorVisible = false;
Console.ForegroundColor = ConsoleColor.Green;
Console.WriteLine();
Console.WriteLine(" [" + DateTime.Now.ToShortTimeString() + "]" + " Connected to the Cassandra Database");
Console.WriteLine();
Console.ForegroundColor = ConsoleColor.White;
string filepath = #"C:\Users\admin\Desktop\wecrack lists";
DirectoryInfo directory = new DirectoryInfo(filepath);
int fileCount = 0;
var client = new MongoClient("mongodb://localhost:27017");
var database = client.GetDatabase("cracking");
var collection = database.GetCollection<Password>("passwords");
foreach (var file in directory.GetFiles("*"))
{
fileCount++;
Console.WriteLine(" [" + DateTime.Now.ToShortTimeString() + "]" + " Working through file: {" + file + "} {" + fileCount + "/" + directory.GetFiles("*").Count() + "}");
List<Password> entitys = new List<Password>();
foreach (string line in File.ReadLines(filepath + #"\" + file.ToString()))
{
entitys.Add(new Password { password = line });
}
collection.InsertManyAsync(entitys);
}
Console.WriteLine();
Console.WriteLine(" [" + DateTime.Now.ToShortTimeString() + "]" + " Finished inserting records, press any key to get the count.");
Console.ReadKey(true);
while (true)
{
Console.ReadKey(true);
}
}

Try batching your updates. That way you won't have all that data in memory at the same time. It may also help you not totally lock up your database.
...
foreach (var file in directory.GetFiles("*"))
{
fileCount++;
Console.WriteLine(" [" + DateTime.Now.ToShortTimeString() + "]" + " Working through file: {" + file + "} {" + fileCount + "/" + directory.GetFiles("*").Count() + "}");
System.IO.StreamReader file = new System.IO.StreamReader(filepath + #"\" + file.ToString());
while(!file.EndOfStream)
{
int passwordBatchCount = 0;
List<Password> entitysBatch = new List<Password>();
while ((string line = file.ReadLine()) != null && passwordBatchCount < BATCH_SIZE)
{
entitysBatch.Add(new Password { password = line });
passwordBatchCount++;
}
collection.InsertManyAsync(entitysBatch);
}
file.Close();
}
}
...

File being used by another process after using File.AppendAllText()

The process cannot access the file 'file path' because it is being used by another process.
i have found these 2 question
File being used by another process after using File.Create()
and
Does File.AppendAllText close the file after the operation
this is an API that i have and need to save every request that comes in and the result that goes out,
there might be more than one request that a give time
my code
public static void SaveTheRequestAndResponse(string type, SearchRequest searchRequest = null, dynamic result = null)
{
var FilePath = AppDomain.CurrentDomain.BaseDirectory + #"SearchRequest";
bool exists = Directory.Exists(FilePath);
if (!exists)
{
var stream = Directory.CreateDirectory(FilePath);
}
if (type == "request")
{
string Space = ", ";
StringBuilder request = new StringBuilder();
request.Append("Search Id : " + searchRequest.ID);
request.Append(Space + "Company Name : " + searchRequest.CompanyName);
request.Append(Space + "Country Code : " + searchRequest.CountryCode);
request.Append(Space + "Search Type : " + searchRequest.SeacrhType);
request.Append(Space + "Request Time : " + DateTime.Now + Environment.NewLine);
var DataToBeSave = request.ToString();
System.IO.File.AppendAllText(FilePath + #"\" + "FileNAme" + DateTime.Now.ToString("dd-MM-yyyy") + ".txt", DataToBeSave + Environment.NewLine);
}
else
{
string Space = ", ";
StringBuilder SearchResult = new StringBuilder();
SearchResult.Append("The result for Request" + Space);
SearchResult.Append("Search Id : " + searchRequest.ID + Space);
SearchResult.Append("States Code : " + result.StatusCode + Space);
SearchResult.Append("Result Time : " + DateTime.Now + Environment.NewLine);
var DataToBeSave = SearchResult.ToString();
System.IO.File.AppendAllText(FilePath + #"\" + "FileNAme" + DateTime.Now.ToString("dd-MM-yyyy") + ".txt", DataToBeSave + Environment.NewLine);
}
}
my understanding is that the File.AppendAllText will close after the Operation so why do i get the this error

my code is having an race condition, and this is because the API is being call by more than one user at each given time, even that
System.IO.File.AppendAllText(FilePath + #"\" + "FileNAme" + DateTime.Now.ToString("dd-MM-yyyy") + ".txt", DataToBeSave + Environment.NewLine);
will close after the Operation, it still need time do its work and only one connection can be open at each give time so the thread need to be lock and that can be done by
private static Object thisLock = new Object();
lock (thisLock)
{
System.IO.File.AppendAllText(FilePath + #"\" + "DandB" + DateTime.Now.ToString("dd-MM-yyyy") + ".txt", DataToBeSave + Environment.NewLine);
}
Thanks to Abydal

C# print list issue

I'm doing a WCF service with GUI as client, however I have a problem with printing list of current items added. I have a code to add new entries to the list:
public bool Add_Data(Data sample)
{
container.Add(sample);
Console.WriteLine("New record added!");
return true;
}
And it's working, however when I'm trying to view added records with first try it works, however if I want to view it again list is adding same element. To show you how it works:
I'm adding new entry and I "print" list:
IMAGE CLICK [works how it should]
However I want to see it again, so I'm pressing same button in my form, and here is what happens:IMAGE CLICK as you can see, we have our list + additional same record, if I will press button again, I will have 3 same records.
Here is my "show records" code:
public string Show_Data()
{
Console.WriteLine("Printing records");
foreach (Data record in container)
{
string final_result = ("\nID: "+ + record.ID + " " + "product: " + record.product + " " + "category: " + record.category + " " + "price: " + record.price + " " + "quantity: " + record.quantity + " " + "\n ");
result += final_result;
}
return result;
}
Let me know if you know how to solve it.

You need to look into variable scope. You have result declared outside of the Show_Data() method. Each time the method is called you are doing result += final_result; which is adding to result. Try the code below and you will get different results.
public string Show_Data()
{
Console.WriteLine("Printing records");
var output = string.Empty;
foreach (Data record in container)
{
string final_result = ("\nID: "+ + record.ID + " " + "product: " + record.product + " " + "category: " + record.category + " " + "price: " + record.price + " " + "quantity: " + record.quantity + " " + "\n ");
output += final_result;
}
return output;
}
Also, I would consider using a string builder and string format as well.
public string Show_Data()
{
Console.WriteLine("Printing records");
var output = new StringBuilder();
foreach (Data record in container)
{
string final_result = string.Format("ID: {0} product: {1} category: {2} price: {3} quantity: {4}", record.ID, record.product, record.category, record.price, record.quantity);
// if using C# 6
// string final_result = string.Format($"ID: {record.ID} product: {record.product} category: {record.category} price: {record.price} quantity: {record.quantity)}";
output.AppendLine(final_result);
}
return output.ToString();
}

How can I check for a stream before initializing one in a loop in C#

I've got the following bit of code that generates a list I'm trying to process. I need to create multiple files as I go through the list.
Currently I'm getting the following error on the calls to genStream outside of the IF, (in and out of the FOREACH loop), and I'm not sure why:
The name genStream' does not exist in the current context
I'm trying to figure out how I can close the one stream, and open another in the IF statement. I tried to put a .close() before creating the new stream, and it gave me an error for using it before initializing the stream.
Here is my bit of code:
/// <summary>
/// Creates a file for each Genre, and writes movie info to each for the cooresponding movies
/// </summary>
/// <param name="cPath">Path to create HTML files in</param>
/// <param name="mList">List of Movies to generate Genre and Movie info from</param>
public static void WriteGenreHTML(string cPath, List<Movie> mList)
{
int lineID = 0;
string tmpGen = null;
string strHeader, strMovie, strGenre, tmpGenre = null;
// Gets a list of unique Genres from the MovieList
var distinctGenres = from m in mList
from genre in m.Genres
group genre by genre into genres
select genres.First();
// Gets a list of Movies with the associated Genres
var moviesWithGenre = from g in distinctGenres
from m in mList
where m.Genres.Contains(g)
orderby g, m.Title
select new { Genre = g, Movie = m };
// Traverses list of movies creating new HTML Genre files, and writing movie info to the HTML genre files
foreach (var m in moviesWithGenre)
{
// Creates new HTML file if new Genre is detected
if (m.Genre != tmpGen)
{
tmpGen = m.Genre;
// initiates streamwriter for catalog output file
FileStream fs = new FileStream(cPath + Path.DirectorySeparatorChar + m.Genre, FileMode.Create);
StreamWriter genStream = new StreamWriter(fs);
// Generates header info for new file, and new Genre
strHeader = "<style type=\"text/css\">\r\n" + "<!--\r\n" + "tr#odd {\r\n" + " background-color:#e2e2e2;\r\n" + " vertical-align:top;\r\n" + "}\r\n" + "\r\n" + "tr#even {\r\n" + " vertical-align:top;\r\n" + "}\r\n" + "div#title {\r\n" + " font-size:16px;\r\n" + " font-weight:bold;\r\n" + "}\r\n" + "\r\n" + "div#mpaa {\r\n" + " font-size:10px;\r\n" + "}\r\n" + "\r\n" + "div#genre {\r\n" + " font-size:12px;\r\n" + " font-style:italic;\r\n" + "}\r\n" + "\r\n" + "div#plot {\r\n" + " height: 63px;\r\n" + " font-size:12px;\r\n" + " overflow:hidden;\r\n" + "}\r\n" + "\r\n" + "div#genre_heading {\r\n" + " height: 50px;\r\n" + " font-size: 24px;\r\n" + " font-weight: bold;\r\n" + " text-align: center;\r\n" + " text-decoration: underline;\r\n" + "}\r\n" + "-->\r\n" + "</style>\r\n" + "\r\n" + "<html>\r\n" + " <body>\r\n" + " <table>\r\n";
strHeader += " <tr>\r\n" + " <td colspan=2>\r\n" + " <div id=\"genre_heading\">" + m.Genre + "</div>\r\n" + " </td>\r\n" + " </tr>\r\n" + "\r\n";
// Writes header HTML to stream
genStream.WriteLine(strHeader);
Console.WriteLine();
Console.WriteLine("Now Processing " + m.Genre);
}
// Generates the HTML for the Movie
strMovie = lineID == 0 ? " <tr id=\"odd\" style=\"page-break-inside:avoid\">\r\n" : " <tr id=\"even\" style=\"page-break-inside:avoid\">\r\n";
strMovie += " <td>\r\n" + " <img src=\".\\images\\" + m.Movie.ImageFile + "\" width=\"75\" height=\"110\">\r\n" + " </td>\r\n" + " <td>\r\n" + " <div id=\"title\">" + m.Movie.Title + "</div>\r\n" + " <div id=\"mpaa\">" + m.Movie.Certification + " " + m.Movie.MPAA + "</div>\r\n" + " <div id=\"genre\">" + strGenre + "</div>\r\n" + " <div id=\"plot\">" + m.Movie.Plot + "</div>\r\n" + " </td>\r\n" + " </tr>\r\n";
// Writes the HTML to the stream
genStream.WriteLine(strMovie);
lineID = lineID == 0 ? 1 : 0;
}
string closingHTML = " </table>\r\n" + " </body>\r\n" + "</html>";
genStream.WriteLine(closingHTML);
genStream.Close();
}
Side note. I'd GREATLY appreciate it if someone could point me towards something I can use to convert the HTML files into PDFs. I tried EO, and it has a nasty "watermark", and chokes on files over a couple MB (mine are 5-10mb+). I have WkHTMLToSharp, but I am not sure how to use it, and can't find any documentation on how to intitialize/use it.
Thanks as always!

Declare the genStream outside of foreach loop, and then initialize it as needed within the if statement:
StreamWriter genStream;
foreach (var m in moviesWithGenre)
{
// Creates new HTML file if new Genre is detected
if (m.Genre != tmpGen)
{
tmpGen = m.Genre;
// initiates streamwriter for catalog output file
FileStream fs = new FileStream(cPath + Path.DirectorySeparatorChar + m.Genre, FileMode.Create);
// Set genStream to the FileStream
genStream = new StreamWriter(fs);

How to convert a HTML File to PDF using WkHTMLToSharp / wkhtmltopdf with Images in C#

I am generating HTML files on the fly, and I would like to create a PDF from the final file. I am using the following to generate the HTML file:
public static void WriteHTML(string cFile, List<Movie> mList)
{
int lineID = 0;
string strHeader, strMovie, strGenre, tmpGenre = null;
string strPDF = null;
// initiates streamwriter for catalog output file
FileStream fs = new FileStream(cFile, FileMode.Create);
StreamWriter catalog = new StreamWriter(fs);
strHeader = "<style type=\"text/css\">\r\n" + "<!--\r\n" + "tr#odd {\r\n" + " background-color:#e2e2e2;\r\n" + " vertical-align:top;\r\n" + "}\r\n" + "\r\n" + "tr#even {\r\n" + " vertical-align:top;\r\n" + "}\r\n" + "div#title {\r\n" + " font-size:16px;\r\n" + " font-weight:bold;\r\n" + "}\r\n" + "\r\n" + "div#mpaa {\r\n" + " font-size:10px;\r\n" + "}\r\n" + "\r\n" + "div#genre {\r\n" + " font-size:12px;\r\n" + " font-style:italic;\r\n" + "}\r\n" + "\r\n" + "div#plot {\r\n" + " height: 63px;\r\n" + " font-size:12px;\r\n" + " overflow:hidden;\r\n" + "}\r\n" + "-->\r\n" + "</style>\r\n" + "\r\n" + "<html>\r\n" + " <body>\r\n" + " <table>\r\n";
catalog.WriteLine(strHeader);
strPDF = strHeader;
foreach (Movie m in mList)
{
tmpGenre = null;
strMovie = lineID == 0 ? " <tr id=\"odd\" style=\"page-break-inside:avoid\">\r\n" : " <tr id=\"even\" style=\"page-break-inside:avoid\">\r\n";
catalog.WriteLine(strMovie);
strPDF += strMovie;
foreach (string genre in m.Genres)
tmpGenre += ", " + genre + "";
strGenre = tmpGenre != null ? tmpGenre.Substring(2) : null;
strMovie = " <td>\r\n" + " <img src=\".\\images\\" + m.ImageFile + "\" width=\"75\" height=\"110\">\r\n" + " </td>\r\n" + " <td>\r\n" + " <div id=\"title\">" + m.Title + "</div>\r\n" + " <div id=\"mpaa\">" + m.Certification + " " + m.MPAA + "</div>\r\n" + " <div id=\"genre\">" + strGenre + "</div>\r\n" + " <div id=\"plot\">" + m.Plot + "</div>\r\n" + " </td>\r\n" + " </tr>\r\n";
catalog.WriteLine(strMovie);
strPDF += strMovie;
lineID = lineID == 0 ? 1 : 0;
}
string closingHTML = " </table>\r\n" + " </body>\r\n" + "</html>";
catalog.WriteLine(closingHTML);
strPDF += closingHTML;
WritePDF(strPDF, cFile + ".PDF");
catalog.Close();
}
Once completed, I want to call the following function to generate the PDF file:
public static void WritePDF(string cFile, string pdfFile)
{
WkHtmlToPdfConverter w = new WkHtmlToPdfConverter();
byte[] strHTML = w.Convert(cFile);
File.WriteAllBytes(pdfFile, strHTML);
w.Dispose();
}
I've discovered that the .Convert function will convert HTML code to PDF, not a file. Secondly, when I pass in the HTML code directly, the images are not appearing in the PDF. I know there is an issue with .GIF files, but these are all .JPG files.
I've read a lot about how good wkhtmltopdf is, and the guy who wrote WkHTMLToSharp posted his project all over SO, but I've been disappointed by the lack of documentation for it.
I WANT to be able to pass in a file to convert, change the margins (I know this is possible, I just need to figure out the correct settings), have it convert images correctly, and most importantly, to not break up my items across multiple pages (support "page-break-inside:avoid" or something similar).
I'd love to see how others are using this!

I have coded an example about how to create a PDF from HTML. I just updated it to also print images.
https://github.com/hmadrigal/playground-dotnet/tree/master/MsDotNet.PdfGeneration
(In my blog post I explain most of the project https://hmadrigal.wordpress.com/2015/10/16/creating-pdf-reports-from-html-using-dotliquid-markup-for-templates-and-wkhtmltoxsharp-for-printing-pdf/ )
Pretty much you have two options:
1: Using file:// and the fullpath to the file.
<img alt="profile" src="{{ employee.PorfileFileName | Prepend: "Assets\ProfileImage\" | ToLocalPath }}" />
2: Using URL Data (https://en.wikipedia.org/wiki/Data_URI_scheme)
<img alt="profile" src="data:image/png;base64,{{ employee.PorfileFileName | Prepend: "Assets\ProfileImage\" | ToLocalPath | ToBase64 }}" />
Cheers,
Herb

Use WkHtmlToXSharp.
Download the latest DLL from Github
public static string ConvertHTMLtoPDF(string htmlFullPath, string pageSize, string orientation)
{
string pdfUrl = htmlFullPath.Replace(".html", ".pdf");
try
{
#region USING WkHtmlToXSharp.dll
//IHtmlToPdfConverter converter = new WkHtmlToPdfConverter();
IHtmlToPdfConverter converter = new MultiplexingConverter();
converter.GlobalSettings.Margin.Top = "0cm";
converter.GlobalSettings.Margin.Bottom = "0cm";
converter.GlobalSettings.Margin.Left = "0cm";
converter.GlobalSettings.Margin.Right = "0cm";
converter.GlobalSettings.Orientation = (PdfOrientation)Enum.Parse(typeof(PdfOrientation), orientation);
if (!string.IsNullOrEmpty(pageSize))
converter.GlobalSettings.Size.PageSize = (PdfPageSize)Enum.Parse(typeof(PdfPageSize), pageSize);
converter.ObjectSettings.Page = htmlFullPath;
converter.ObjectSettings.Web.EnablePlugins = true;
converter.ObjectSettings.Web.EnableJavascript = true;
converter.ObjectSettings.Web.Background = true;
converter.ObjectSettings.Web.LoadImages = true;
converter.ObjectSettings.Load.LoadErrorHandling = LoadErrorHandlingType.ignore;
Byte[] bufferPDF = converter.Convert();
System.IO.File.WriteAllBytes(pdfUrl, bufferPDF);
converter.Dispose();
#endregion
}
catch (Exception ex)
{
throw new Exception(ex.Message, ex);
}
return pdfUrl;
}

You can use Spire.Pdf to do so.
This component could convert html to pdf.
PdfDocument pdfdoc = new PdfDocument();
pdfdoc.LoadFromHTML(fileFullName, true, true, true);
//String url = "http://www.e-iceblue.com/";
//pdfdoc.LoadFromHTML(url, false, true, true);
pdfdoc.SaveToFile("FromHTML.pdf");

We're also using wkhtmltopdf and are able to render images correctly. However, by default the rendering of images is disabled.
You have to specify those options on your converter instance:
var wk = _GetConverter()
wk.GlobalSettings.Margin.Top = "20mm";
wk.GlobalSettings.Margin.Bottom = "10mm";
wk.GlobalSettings.Margin.Left = "10mm";
wk.GlobalSettings.Margin.Right = "10mm";
wk.GlobalSettings.Size.PaperSize = PdfPaperSize.A4;
wk.ObjectSettings.Web.PrintMediaType = true;
wk.ObjectSettings.Web.LoadImages = true;
wk.ObjectSettings.Web.EnablePlugins = false;
wk.ObjectSettings.Web.EnableJavascript = true;
result = wk.Convert(htmlContent);