Develop Reference

Writing Parquet files using Parquet.NET works with local file, but results in empty file in blob storage

We are using parquet.net to write parquet files. I've set up a simple schema containing 3 columns, and 2 rows:
// Set up the file structure
var UserKey = new Parquet.Data.DataColumn(
new DataField<Int32>("UserKey"),
new Int32[] { 1234, 12345}
);
var AADID = new Parquet.Data.DataColumn(
new DataField<string>("AADID"),
new string[] { Guid.NewGuid().ToString(), Guid.NewGuid().ToString() }
);
var UserLocale = new Parquet.Data.DataColumn(
new DataField<string>("UserLocale"),
new string[] { "en-US", "en-US" }
);
var schema = new Schema(UserKey.Field, AADID.Field, UserLocale.Field
);
When using a FileStream to write to a local file, a file is created, and when the code finishes, I can see two rows in the file (which is 1 kb after):
using (Stream fileStream = System.IO.File.OpenWrite("C:\\Temp\\Users.parquet")) {
using (var parquetWriter = new ParquetWriter(schema, fileStream)) {
// Creare a new row group in the file
using (ParquetRowGroupWriter groupWriter = parquetWriter.CreateRowGroup()) {
groupWriter.WriteColumn(UserKey);
groupWriter.WriteColumn(AADID);
groupWriter.WriteColumn(UserLocale);
}
}
}
Yet, when I attempt to use the same to write to our blob storage, that only generates an empty file, and the data is missing:
// Open reference to Blob Container
CloudAppendBlob blob = OpenBlobFile(blobEndPoint, fileName);
using (MemoryStream stream = new MemoryStream()) {
blob.CreateOrReplaceAsync();
using (var parquetWriter = new ParquetWriter(schema, stream)) {
// Creare a new row group in the file
using (ParquetRowGroupWriter groupWriter = parquetWriter.CreateRowGroup()) {
groupWriter.WriteColumn(UserKey);
groupWriter.WriteColumn(AADID);
groupWriter.WriteColumn(UserLocale);
}
// Set stream position to 0
stream.Position = 0;
blob.AppendBlockAsync(stream);
return true;
}
...
public static CloudAppendBlob OpenBlobFile (string blobEndPoint, string fileName) {
CloudBlobContainer container = new CloudBlobContainer(new System.Uri(blobEndPoint));
CloudAppendBlob blob = container.GetAppendBlobReference(fileName);
return blob;
}
Reading the documentation, I would think my implementation of the blob.AppendBlocAsync should do the trick, but yet I end up with an empty file. Would anyone have suggestions as to why this is and how I can resolve it so I actually end up with data in the file?
Thanks in advance.

The explanation for the file ending up empty is the line:
blob.AppendBlockAsync(stream);
Note how the function called has the Async suffix. This means it expects whatever is calling it to wait. I turned the function the code was in into an Async one, and had Visual Studio suggest the following change to the line:
_ = await blob.AppendBlockAsync(stream);
I'm not entirely certain what _ represents, and hovering my mouse over it doesn't reveal much more, other than it being a long data type, but the code now works as intended.

How to edit bookmarks in a Word template using DocumentFormat.OpenXml and save it as a new PDF file?

I'm really having trouble in editing bookmarks in a Word template using Document.Format.OpenXML and then saving it to a new PDF file.
I cannot use Microsoft.Word.Interop as it gives a COM error on the server.
My code is this:
public static void CreateWordDoc(string templatePath, string destinationPath, Dictionary<string, dynamic> dictionary)
{
byte[] byteArray = File.ReadAllBytes(templatePath);
using (MemoryStream stream = new MemoryStream())
{
stream.Write(byteArray, 0, (int)byteArray.Length);
using (WordprocessingDocument wordDoc = WordprocessingDocument.Open(stream, true))
{
var bookmarks = (from bm in wordDoc.MainDocumentPart.Document.Body.Descendants<BookmarkStart>()
select bm).ToList();
foreach (BookmarkStart mark in bookmarks)
{
if (mark.Name != "Table" && mark.Name != "_GoBack")
{
UpdateBookmark(dictionary, mark);//Not doing anything
}
else if (mark.Name != "Table")
{
// CreateTable(dictionary, wordDoc, mark);
}
}
File.WriteAllBytes("D:\\RohitDocs\\newfile_rohitsingh.docx", stream.ToArray());
wordDoc.Close();
}
// Save the file with the new name
}
}
private static void UpdateBookmark(Dictionary<string, dynamic> dictionary, BookmarkStart mark)
{
string name = mark.Name;
string value = dictionary[name];
Run run = new Run(new Text(value));
RunProperties props = new RunProperties();
props.AppendChild(new FontSize() { Val = "20" });
run.RunProperties = props;
var paragraph = new DocumentFormat.OpenXml.Wordprocessing.Paragraph(run);
mark.Parent.InsertAfterSelf(paragraph);
paragraph.PreviousSibling().Remove();
mark.Remove();
}
I was trying to replace bookmarks with my text but the UpdateBookmark method doesn't work. I'm writing stream and saving it because I thought if bookmarks are replaced then I can save it to another file.

I think you want to make sure that when you reference mark.Parent that you are getting the correct instance that you are expecting.
Once you get a reference to the correct Paragraph element where your content should go, use the following code to add/swap the run.
// assuming you have a reference to a paragraph called "p"
p.AppendChild<Run>(new Run(new Text(content)) { RunProperties = props });
// and here is some code to remove a run
p.RemoveChild<Run>(run);
To answers the second part of your question, when I did a similar project a few years ago we used iTextSharp to create PDFs from Docx. It worked very well and the API was easy to grok. We even added password encryption and embedded watermarks to the PDFs.

How to write to excel many times using one object of EPPLUS in C#

Refer to some EPPLUS sample code, there are just creating one epplus object for one activity.
ex
using (ExcelPackage package = new ExcelPackage(newFile))
{...
// activity
}
it means that after activity is finished, object is disposed automatically.
and for the next, object will be created again for doing activity again.
And i want to create just one EPPLUS object for many times of activity, i want to create one EPPLUS object can be used many times, not using "using" statement.
this is my code
public partial class FMain : Form
{
...
ExcelPackage pack;
FileInfo InfoPathFile;
public StringPathFile = ""
...
public FMain()
{
...
}
private void NewDialog_FileOk(object sender, CancelEventArgs e)
{
if(pack != null)
pack.Dispose();
StringPathFile = NewDialog.FileName;
InfoPathFile = new FileInfo(StringPathFile);
pack = new ExcelPackage(InfoPathFile);
...
}
private void SaveData(float[] Sens, string tt, string dd)
{
var ExSheet = pack.Workbook.Worksheets["Data"];
ExSheet.Cells["A" + rowExcel].Value = Numb;
ExSheet.Cells["B" + rowExcel].Value = Sens[0];
ExSheet.Cells["C" + rowExcel].Value = Sens[1];
ExSheet.Cells["D" + rowExcel].Value = Sens[2];
ExSheet.Cells["E" + rowExcel].Value = Sens[3];
ExSheet.Cells["F" + rowExcel].Value = tt;
ExSheet.Cells["G" + rowExcel].Value = dd;
//pack.SaveAs(InfoPathFile);
pack.Save();
}
I want to write to excel many times, using just one EPPLUS object, i dont want to create epplus object every time i do an activity. Using my code, i can just write once to excel file, and second writing process is failed.
Can i do that?

The problem you are having is calling the Save() will automatically close the package so the next time you write to it it will generate an error. EPPlus isnt really meant to do "incremental" saves like that - its more designed to sit on a server, have the client tell it to generate a file all at once, and send it to the client.
I think the best bet would be to keep a copy of it in memory and incrementally write the file. You could do something like this via MemoryStream. So create class-level MemoryStreamvar and use that to hold the work-in-progress Excel Package. This hopefully demonstrates that concept:
[TestMethod]
public void Multi_Save_Test()
{
//http://stackoverflow.com/questions/28007087/how-to-write-to-excel-many-times-using-one-object-of-epplus-in-c-sharp
var existingFile = new FileInfo(#"c:\temp\temp.xlsx");
if (existingFile.Exists)
existingFile.Delete();
//Use memstream and create the package but WITHOUT the FI so it is a memory stream as well
//Avoid using and call manual dispose
var holdingstream = new MemoryStream();
var pack = new ExcelPackage();
var ExSheet = pack.Workbook.Worksheets.Add("Data");
ExSheet.Cells["A1"].Value = "wer";
ExSheet.Cells["B1"].Value = "sdf";
//Do an incremental save to the file and copy the stream before closing - ORDER COUNTS!
pack.SaveAs(existingFile);
holdingstream.SetLength(0);
pack.Stream.Position = 0;
pack.Stream.CopyTo(holdingstream);
//*********************************************************
//reopen the holding stream, make a change, and resave it
pack.Load(holdingstream);
ExSheet = pack.Workbook.Worksheets["Data"];
ExSheet.Cells["A2"].Value = "wer";
ExSheet.Cells["B2"].Value = "sdf";
//Another incremental change
pack.SaveAs(existingFile);
holdingstream.SetLength(0);
pack.Stream.Position = 0;
pack.Stream.CopyTo(holdingstream);
//*********************************************************
//reopen the holding stream, make a change, and resave it
pack.Load(holdingstream);
ExSheet = pack.Workbook.Worksheets["Data"];
ExSheet.Cells["A3"].Value = "wer";
ExSheet.Cells["B3"].Value = "sdf";
//Another incremental change
pack.SaveAs(existingFile);
holdingstream.SetLength(0);
pack.Stream.Position = 0;
pack.Stream.CopyTo(holdingstream);
//*********************************************************
//reopen the holding stream, make a change, and do a FINAL save
pack.Load(holdingstream);
ExSheet = pack.Workbook.Worksheets["Data"];
ExSheet.Cells["A4"].Value = "wer";
ExSheet.Cells["B4"].Value = "sdf";
//All done so only need to save it to the file
pack.SaveAs(existingFile);
//cleanup
pack.Dispose();
holdingstream.Dispose();
}

Change object's color inside existiong PDF with iTextSharp

Major part of my job is automation of engineering process, so I have to create simple program, that compares 2 different version of 1 drawn element, by overlapping drawings, in order to review differences. Drawings represent single sheet PDF files.
I'm using .Net Framework and C# 4.5;
iTextSharp library for editing PDF files;
Initially, I'm getting 2 files, read them and create the third one, that contains the result;
var file1 = "file1.pdf";
var file2 = "file2.pdf";
var result = "result.pdf";
using (Stream f1Stream = new FileStream(file1, FileMode.Open))
using (Stream f2Stream = new FileStream(file2, FileMode.Open))
using (Stream resultStream = new FileStream(result, FileMode.Create, FileAccess.ReadWrite))
using (PdfReader f2Reader = new PdfReader(f2Stream))
using (PdfReader f1Reader = new PdfReader(f1Stream))
{
PdfStamper pdfStamper = new PdfStamper(f1Reader, resultStream);
PdfContentByte pdfContentByte = pdfStamper.GetOverContent(1);
var page = pdfStamper.GetImportedPage(f2Reader, 1);
pdfContentByte.AddTemplate(page,2,2);
pdfStamper.Close();
}
The code above makes just that, but a few sequential questions are arising
I want to change the color of elements in the result file i.e. elements that come from the 1st drawing in green and the others from 2nd one - in red color. Maybe I have to change the color of entities in initial 2 PDFs and then to merge;
Initial files have layers, and because they are two sequential revision of the same construction element and differences between them are very few, they have identical layers. And I want to have " layerFoo " and " layerFoo# " in the result PDF. Maybe I have to rename all the layers in one the the 2 initial PDFs and then to merge them.
Аll suggestions are welcomed including usage of another library :)
--> Edit1
Big thanks to Chris Haas! You are absolutely right for token type and string value! iTextRUPS is great helping tool for understanding the structure of PDF files.
Following code is taken from the post that you pointed me out.
The following statement:
stream.SetData(System.Text.Encoding.ASCII.GetBytes(String.Join("\n", newBuf.ToArray())));
updates the stream of the file and then with
using (var fs = new FileStream(file2, FileMode.Create, FileAccess.Write, FileShare.None))
{
var stamper = new PdfStamper(reader, fs);
reader.SetPageContent(1,reader.GetPageContent(1));
stamper.Close();
}
the new file is created with updated stream.
I made 1 simple test file with only 2 lines, change their color and save back to a new file.
No problem!
After that, I tried the same simple operation with real file, that represents real drawing of construction element, the result file was less than half of the original and was broken.
What comes to mind is the updated stream is saved to the new file but the other information inside other containers is not saved, it's just the stream.
Because I stuck with that, I continue to the next step of investigation -> layers
I wrote this code in order to get available layers in a PDF file. I will try to insert more records into layers dictionary to see what will happen.
var resourcesReference = page.Get(PdfName.RESOURCES) as PdfIndirectReference;
var resources = PdfReader.GetPdfObject(resourcesReference) as PdfDictionary;
var propertiesObjhectReferences = resources.Get(PdfName.PROPERTIES);
var properties = PdfReader.GetPdfObject(propertiesObjhectReferences) as PdfDictionary;
foreach (var property in properties.Keys)
{
var layerReference = properties.Get(property);
var layerObject = PdfReader.GetPdfObject(layerReference) as PdfDictionary;
foreach (var key in layerObject.Keys)
{
if (key.ToString()!=PdfName.TYPE.ToString())
{
var layerName = layerObject.GetAsString(key).ToUnicodeString();
}
}
}
If I come back to my main goal from the top of the post, I tends to insert the stream and layers from first file into second in order to obtain result file, that contains objects from the previous 2, painted in different colors + layers from both.
Feel free to suggest me another, more simpler and beautiful solution! I will be happy if you revise my code and correct it! Thank You very much!
EDIT 2
I will simplify the work because the lack of time, just change the color of entities inside one PDF and put it on the background on the other.
const string Pdf = "file1.pdf";
var reader = new PdfReader(Pdf);
var page = reader.GetPageN(1);
var objectReference = page.Get(PdfName.CONTENTS) as PdfIndirectReference;
var stream = (PRStream)PdfReader.GetPdfObject(objectReference);
var streamBytes = PdfReader.GetStreamBytes(stream);
var tokenizer = new PRTokeniser(new RandomAccessFileOrArray(streamBytes));
var newBuf = new List<string>();
while (tokenizer.NextToken())
{
var token = tokenizer.StringValue;
newBuf.Add(token);
if (tokenizer.TokenType == PRTokeniser.TokType.OTHER
&& newBuf[newBuf.Count - 1].Equals("S", StringComparison.CurrentCultureIgnoreCase))
{
newBuf.Insert(newBuf.Count - 1, "0");
newBuf.Insert(newBuf.Count - 1, "1");
newBuf.Insert(newBuf.Count - 1, "1");
newBuf.Insert(newBuf.Count - 1, "RG");
}
}
var resultStream = String.Join("\n", newBuf.ToArray());
stream.SetData(System.Text.Encoding.ASCII.GetBytes(resultStream));
var file2 = Pdf.Insert(Pdf.Length - 4, "Result");
using (var fs = new FileStream(file2, FileMode.Create, FileAccess.Write, FileShare.None))
{
var stamper = new PdfStamper(reader, fs);
reader.SetPageContent(1, reader.GetPageContent(1));
stamper.Close();
}
Result PDF is broken and iTextRUPS throws exception when try to get the stream data from the page.

how can I put a content in a mergefield in docx

I'm developing a web application with asp.net and I have a file called Template.docx that works like a template to generate other reports. Inside this Template.docx I have some MergeFields (Title, CustomerName, Content, Footer, etc) to replace for some dynamic content in C#.
I would like to know, how can I put a content in a mergefield in docx ?
I don't know if MergeFields is the right way to do this or if there is another way. If you can suggest me, I appreciate!
PS: I have openxml referenced in my web application.
Edits:
private MemoryStream LoadFileIntoStream(string fileName)
{
MemoryStream memoryStream = new MemoryStream();
using (FileStream fileStream = File.OpenRead(fileName))
{
memoryStream.SetLength(fileStream.Length);
fileStream.Read(memoryStream.GetBuffer(), 0, (int) fileStream.Length);
memoryStream.Flush();
fileStream.Close();
}
return memoryStream;
}
public MemoryStream GenerateWord()
{
string templateDoc = "C:\\temp\\template.docx";
string reportFileName = "C:\\temp\\result.docx";
var reportStream = LoadFileIntoStream(templateDoc);
// Copy a new file name from template file
//File.Copy(templateDoc, reportFileName, true);
// Open the new Package
Package pkg = Package.Open(reportStream, FileMode.Open, FileAccess.ReadWrite);
// Specify the URI of the part to be read
Uri uri = new Uri("/word/document.xml", UriKind.Relative);
PackagePart part = pkg.GetPart(uri);
XmlDocument xmlMainXMLDoc = new XmlDocument();
xmlMainXMLDoc.Load(part.GetStream(FileMode.Open, FileAccess.Read));
// replace some keys inside xml (it will come from database, it's just a test)
xmlMainXMLDoc.InnerXml = xmlMainXMLDoc.InnerXml.Replace("field_customer", "My Customer Name");
xmlMainXMLDoc.InnerXml = xmlMainXMLDoc.InnerXml.Replace("field_title", "Report of Documents");
xmlMainXMLDoc.InnerXml = xmlMainXMLDoc.InnerXml.Replace("field_content", "Content of Document");
// Open the stream to write document
StreamWriter partWrt = new StreamWriter(part.GetStream(FileMode.Open, FileAccess.Write));
//doc.Save(partWrt);
xmlMainXMLDoc.Save(partWrt);
partWrt.Flush();
partWrt.Close();
reportStream.Flush();
pkg.Close();
return reportStream;
}
PS: When I convert MemoryStream to a file, I got a corrupted file. Thanks!

I know this is an old post, but I could not get the accepted answer to work for me. The project linked would not even compile (which someone has already commented in that link). Also, it seems to use other Nuget packages like WPFToolkit.
So I'm adding my answer here in case someone finds it useful. This only uses the OpenXML SDK 2.5 and also the WindowsBase v4. This works on MS Word 2010 and later.
string sourceFile = #"C:\Template.docx";
string targetFile = #"C:\Result.docx";
File.Copy(sourceFile, targetFile, true);
using (WordprocessingDocument document = WordprocessingDocument.Open(targetFile, true))
{
// If your sourceFile is a different type (e.g., .DOTX), you will need to change the target type like so:
document.ChangeDocumentType(WordprocessingDocumentType.Document);
// Get the MainPart of the document
MainDocumentPart mainPart = document.MainDocumentPart;
var mergeFields = mainPart.RootElement.Descendants<FieldCode>();
var mergeFieldName = "SenderFullName";
var replacementText = "John Smith";
ReplaceMergeFieldWithText(mergeFields, mergeFieldName, replacementText);
// Save the document
mainPart.Document.Save();
}
private void ReplaceMergeFieldWithText(IEnumerable<FieldCode> fields, string mergeFieldName, string replacementText)
{
var field = fields
.Where(f => f.InnerText.Contains(mergeFieldName))
.FirstOrDefault();
if (field != null)
{
// Get the Run that contains our FieldCode
// Then get the parent container of this Run
Run rFldCode = (Run)field.Parent;
// Get the three (3) other Runs that make up our merge field
Run rBegin = rFldCode.PreviousSibling<Run>();
Run rSep = rFldCode.NextSibling<Run>();
Run rText = rSep.NextSibling<Run>();
Run rEnd = rText.NextSibling<Run>();
// Get the Run that holds the Text element for our merge field
// Get the Text element and replace the text content
Text t = rText.GetFirstChild<Text>();
t.Text = replacementText;
// Remove all the four (4) Runs for our merge field
rFldCode.Remove();
rBegin.Remove();
rSep.Remove();
rEnd.Remove();
}
}
What the code above does is basically this:
Identify the 4 Runs that make up the merge field named "SenderFullName".
Identify the Run that contains the Text element for our merge field.
Remove the 4 Runs.
Update the text property of the Text element for our merge field.
UPDATE
For anyone interested, here is a simple static class I used to help me with replacing merge fields.

Frank Fajardo's answer was 99% of the way there for me, but it is important to note that MERGEFIELDS can be SimpleFields or FieldCodes.
In the case of SimpleFields, the text runs displayed to the user in the document are children of the SimpleField.
In the case of FieldCodes, the text runs shown to the user are between the runs containing FieldChars with the Separate and the End FieldCharValues. Occasionally, several text containing runs exist between the Separate and End Elements.
The code below deals with these problems. Further details of how to get all the MERGEFIELDS from the document, including the header and footer is available in a GitHub repository at https://github.com/mcshaz/SimPlanner/blob/master/SP.DTOs/Utilities/OpenXmlExtensions.cs
private static Run CreateSimpleTextRun(string text)
{
Run returnVar = new Run();
RunProperties runProp = new RunProperties();
runProp.Append(new NoProof());
returnVar.Append(runProp);
returnVar.Append(new Text() { Text = text });
return returnVar;
}
private static void InsertMergeFieldText(OpenXmlElement field, string replacementText)
{
var sf = field as SimpleField;
if (sf != null)
{
var textChildren = sf.Descendants<Text>();
textChildren.First().Text = replacementText;
foreach (var others in textChildren.Skip(1))
{
others.Remove();
}
}
else
{
var runs = GetAssociatedRuns((FieldCode)field);
var rEnd = runs[runs.Count - 1];
foreach (var r in runs
.SkipWhile(r => !r.ContainsCharType(FieldCharValues.Separate))
.Skip(1)
.TakeWhile(r=>r!= rEnd))
{
r.Remove();
}
rEnd.InsertBeforeSelf(CreateSimpleTextRun(replacementText));
}
}
private static IList<Run> GetAssociatedRuns(FieldCode fieldCode)
{
Run rFieldCode = (Run)fieldCode.Parent;
Run rBegin = rFieldCode.PreviousSibling<Run>();
Run rCurrent = rFieldCode.NextSibling<Run>();
var runs = new List<Run>(new[] { rBegin, rCurrent });
while (!rCurrent.ContainsCharType(FieldCharValues.End))
{
rCurrent = rCurrent.NextSibling<Run>();
runs.Add(rCurrent);
};
return runs;
}
private static bool ContainsCharType(this Run run, FieldCharValues fieldCharType)
{
var fc = run.GetFirstChild<FieldChar>();
return fc == null
? false
: fc.FieldCharType.Value == fieldCharType;
}

You could try http://www.codeproject.com/KB/office/Fill_Mergefields.aspx which uses the Open XML SDK to do this.