Develop Reference

Lucene.Net (4.8) AutoComplete / AutoSuggestion

I'd like to implement a searchable index using Lucene.Net 4.8 that supplies a user with suggestions / autocomplete for single words & phrases.
The index has been created successfully; the suggestions are where I've stalled.
Version 4.8 seems to have introduced a substantial number of breaking changes, and none of the available samples I've found work.
Where I stand
For reference, LuceneVersion is this:
private readonly LuceneVersion LuceneVersion = LuceneVersion.LUCENE_48;
Solution 1
I've tried this, but can't get past reader.Terms:
public void TryAutoComplete()
{
var analyzer = new EnglishAnalyzer(LuceneVersion);
var config = new IndexWriterConfig(LuceneVersion, analyzer);
RAMDirectory dir = new RAMDirectory();
using (IndexWriter iw = new IndexWriter(dir, config))
{
Document d = new Document();
TextField f = new TextField("text","",Field.Store.YES);
d.Add(f);
f.SetStringValue("abc");
iw.AddDocument(d);
f.SetStringValue("colorado");
iw.AddDocument(d);
f.SetStringValue("coloring book");
iw.AddDocument(d);
iw.Commit();
using (IndexReader reader = iw.GetReader(false))
{
TermEnum terms = reader.Terms(new Term("text", "co"));
int maxSuggestsCpt = 0;
// will print:
// colorado
// coloring book
do
{
Console.WriteLine(terms.Term.Text);
maxSuggestsCpt++;
if (maxSuggestsCpt >= 5)
break;
}
while (terms.Next() && terms.Term.Text.StartsWith("co"));
}
}
}
reader.Terms no longer exists. Being new to Lucene, it's unclear how to refactor this.
Solution 2
Trying this, I'm thrown an error:
public void TryAutoComplete2()
{
using(var analyzer = new EnglishAnalyzer(LuceneVersion))
{
IndexWriterConfig config = new IndexWriterConfig(LuceneVersion, analyzer);
RAMDirectory dir = new RAMDirectory();
using(var iw = new IndexWriter(dir,config))
{
Document d = new Document()
{
new TextField("text", "this is a document with a some words",Field.Store.YES),
new Int32Field("id", 42, Field.Store.YES)
};
iw.AddDocument(d);
iw.Commit();
using (IndexReader reader = iw.GetReader(false))
using (SpellChecker speller = new SpellChecker(new RAMDirectory()))
{
//ERROR HERE!!!
speller.IndexDictionary(new LuceneDictionary(reader, "text"), config, false);
string[] suggestions = speller.SuggestSimilar("dcument", 5);
IndexSearcher searcher = new IndexSearcher(reader);
foreach (string suggestion in suggestions)
{
TopDocs docs = searcher.Search(new TermQuery(new Term("text", suggestion)), null, Int32.MaxValue);
foreach (var doc in docs.ScoreDocs)
{
System.Diagnostics.Debug.WriteLine(searcher.Doc(doc.Doc).Get("id"));
}
}
}
}
}
}
When debugging, speller.IndexDictionary(new LuceneDictionary(reader, "text"), config, false); throws a The object cannot be set twice! error, which I can't explain.
Any thoughts are welcome.
Clarification
I'd like to return a list of suggested terms for a given input, not the documents or their full content.
For example, if a document contains "Hello, my name is Clark. I'm from Atlanta," and I submit "Atl," then "Atlanta" should come back as a suggestion.

If I am understanding you correctly you may be over-complicating your index design a bit. If your goal is to use Lucene for auto-complete, you want to create an index of the terms you consider complete. Then simply query the index using a PrefixQuery using a partial word or phrase.
using Lucene.Net.Analysis;
using Lucene.Net.Analysis.En;
using Lucene.Net.Documents;
using Lucene.Net.Index;
using Lucene.Net.Search;
using Lucene.Net.Store;
using Lucene.Net.Util;
using System;
using System.Linq;
namespace LuceneDemoApp
{
class LuceneAutoCompleteIndex : IDisposable
{
const LuceneVersion Version = LuceneVersion.LUCENE_48;
RAMDirectory Directory;
Analyzer Analyzer;
IndexWriterConfig WriterConfig;
private void IndexDoc(IndexWriter writer, string term)
{
Document doc = new Document();
doc.Add(new StringField(FieldName, term, Field.Store.YES));
writer.AddDocument(doc);
}
public LuceneAutoCompleteIndex(string fieldName, int maxResults)
{
FieldName = fieldName;
MaxResults = maxResults;
Directory = new RAMDirectory();
Analyzer = new EnglishAnalyzer(Version);
WriterConfig = new IndexWriterConfig(Version, Analyzer);
WriterConfig.OpenMode = OpenMode.CREATE_OR_APPEND;
}
public string FieldName { get; }
public int MaxResults { get; set; }
public void Add(string term)
{
using (var writer = new IndexWriter(Directory, WriterConfig))
{
IndexDoc(writer, term);
}
}
public void AddRange(string[] terms)
{
using (var writer = new IndexWriter(Directory, WriterConfig))
{
foreach (string term in terms)
{
IndexDoc(writer, term);
}
}
}
public string[] WhereStartsWith(string term)
{
using (var reader = DirectoryReader.Open(Directory))
{
IndexSearcher searcher = new IndexSearcher(reader);
var query = new PrefixQuery(new Term(FieldName, term));
TopDocs foundDocs = searcher.Search(query, MaxResults);
var matches = foundDocs.ScoreDocs
.Select(scoreDoc => searcher.Doc(scoreDoc.Doc).Get(FieldName))
.ToArray();
return matches;
}
}
public void Dispose()
{
Directory.Dispose();
Analyzer.Dispose();
}
}
}
Running this:
var indexValues = new string[] { "apple fruit", "appricot", "ape", "avacado", "banana", "pear" };
var index = new LuceneAutoCompleteIndex("fn", 10);
index.AddRange(indexValues);
var matches = index.WhereStartsWith("app");
foreach (var match in matches)
{
Console.WriteLine(match);
}
You get this:
apple fruit
appricot

How to create Table Of Contents in iTextSharp

I need to create a Table Of Contents with page numbers, but I don't know how. Next format:
heading1 ----------------page number
subHeading1---------------page number
subHeading2---------------page number
heading2-----------------page number
I read a few articles and didn't understand. In particular, I mean this article, where "Named destinations" and "GoTo actions" I think it is useful for me, but I don't know how to use it in iTextSharp.
In my code, I have got a few "Chapter" and "Section", and I want to take it and create a TOC. I've understood that I need to use PdfPageEventHelper and OnChapter.

You've probably implemented this yourself by name, but I made a small example myself for the sake of completeness.
Please take a look at the CreateTOC example. It creates a PDF with some random text:
You can clearly see the titles and the content under the titles. After we have added all our content, we start a new page, and we add a table of contents:
The table of contents is composed by a series of key-value pairs, where the key is the title and the value is the page number. We create this list in a page event:
public class TOCEvent extends PdfPageEventHelper {
protected List<SimpleEntry<String, Integer>> toc = new ArrayList<>();
#Override
public void onGenericTag(PdfWriter writer, Document document, Rectangle rect, String text) {
toc.add(new SimpleEntry(text, writer.getPageNumber()));
}
public List getTOC() {
return toc;
}
}
We use this page event like this:
public void createPdf(String dest) throws IOException, DocumentException {
Document document = new Document();
PdfWriter writer = PdfWriter.getInstance(document, new FileOutputStream(dest));
TOCEvent event = new TOCEvent();
writer.setPageEvent(event);
document.open();
for (int i = 0; i < 10; i++) {
String title = "This is title " + i;
Chunk c = new Chunk(title, titleFont);
c.setGenericTag(title);
document.add(new Paragraph(c));
for (int j = 0; j < 50; j++) {
document.add(new Paragraph("Line " + j + " of title " + i));
}
}
document.newPage();
document.add(new Paragraph("Table of Contents", titleFont));
Chunk dottedLine = new Chunk(new DottedLineSeparator());
List<SimpleEntry<String, Integer>> entries = event.getTOC();
Paragraph p;
for (SimpleEntry<String, Integer> entry : entries) {
p = new Paragraph(entry.getKey());
p.add(dottedLine);
p.add(String.valueOf(entry.getValue()));
document.add(p);
}
document.close();
}
First we create an instance of the event and we declare it to the writer:
TOCEvent event = new TOCEvent();
writer.setPageEvent(event);
We mark the titles using setGenericTag():
String title = "This is title " + i;
Chunk c = new Chunk(title, titleFont);
c.setGenericTag(title);
document.add(new Paragraph(c));
Once we've finished adding the content, we get all the entries:
List<SimpleEntry<String, Integer>> entries = event.getTOC();
We loop over this list and compose a Paragraph for every entry:
for (SimpleEntry<String, Integer> entry : entries) {
p = new Paragraph(entry.getKey());
p.add(dottedLine);
p.add(String.valueOf(entry.getValue()));
document.add(p);
}
No one can argue that this was difficult. The event class takes less than 10 lines of code. Adding support for subheadings will add a handful of lines, but that shouldn't be difficult too. It's a matter of building a tree structure, and introducing some indentation where necessary.

Thanks for the example, i needed this in C# and with multicolumn, so i rewrote this example as below:
namespace GerarPDF
{
public class GerarPDF
{
public const String DEST = "results/example.pdf";
public GerarPDF()
{
FileInfo file = new FileInfo(String.Concat(AppDomain.CurrentDomain.BaseDirectory, #"/", DEST));
file.Directory.Create();
this.createPdf(file.FullName);
}
public void createPdf(String dest)
{
FileStream fs = new FileStream(dest, FileMode.OpenOrCreate, FileAccess.Write, FileShare.None);
Document document = new Document(PageSize.LETTER);
PdfWriter writer = PdfWriter.GetInstance(document, fs);
document.Open();
TOCEvent evento = new TOCEvent();
writer.PageEvent = evento;
for (int i = 0; i < 10; i++)
{
String title = "This is title " + i;
Chunk c = new Chunk(title, new Font());
c.SetGenericTag(title);
document.Add(new Paragraph(c));
for (int j = 0; j < 50; j++)
{
document.Add(new Paragraph("Line " + j + " of title " + i + " page: " + writer.PageNumber));
}
}
document.NewPage();
document.Add(new Paragraph("Table of Contents", new Font()));
Chunk dottedLine = new Chunk(new DottedLineSeparator());
List<PageIndex> entries = evento.getTOC();
MultiColumnText columns = new MultiColumnText();
columns.AddRegularColumns(72, 72 * 7.5f, 24, 2);
Paragraph p;
for (int i = 0; i < 10; i++)
{
foreach (PageIndex pageIndex in entries)
{
Chunk chunk = new Chunk(pageIndex.Text);
chunk.SetAction(PdfAction.GotoLocalPage(pageIndex.Name, false));
p = new Paragraph(chunk);
p.Add(dottedLine);
chunk = new Chunk(pageIndex.Page.ToString());
chunk.SetAction(PdfAction.GotoLocalPage(pageIndex.Name, false));
p.Add(chunk);
columns.AddElement(p);
}
}
document.Add(columns);
document.Close();
}
public class TOCEvent : PdfPageEventHelper
{
protected int counter = 0;
protected List<PageIndex> toc = new List<PageIndex>();
public override void OnGenericTag(PdfWriter writer, Document document, Rectangle rect, string text)
{
String name = "dest" + (counter++);
int page = writer.PageNumber;
toc.Add(new PageIndex() { Text = text, Name = name, Page = page });
writer.DirectContent.LocalDestination(name, new PdfDestination(PdfDestination.FITH, rect.GetTop(0)));
}
public List<PageIndex> getTOC()
{
return toc;
}
}
}
public class PageIndex
{
public string Text { get; set; }
public string Name { get; set; }
public int Page { get; set; }
}
}

this snipplet can it be recursive, the basic concept is:
List<PdfPCell> celdas = new List<PdfPCell>();
string urlSection=String.empty;
var estatus = new Phrase();
estatus.Leading = 25;
if (streams != null && streams.Any())
primero = streams.FirstOrDefault(x => x.Id == enlace.Id);
if (primero != null)
urlSection = primero.UrlSection;
//For the code and generate hyperlink:
Chunk espacioTab = new Chunk(" " + enlace.Name, baseFontBig );
espacioTab = Visor.Servicios.GeneracionPDF.PDFUtils.GenerarVinculo(" " + enlace.Name, urlSection, baseFontBig);
estatus.Add(espacioTab);
if (incluirPaginado)
{
if (primero != null)
actualPage = primero.TotalPages;
else
actualPage = 0;
///This is important, generate dots like "...." to chunk end
estatus.Add(new Chunk(new iTextSharp.text.pdf.draw.DottedLineSeparator()));
var linkPagina = new Chunk(actualPage.ToString());
linkPagina = Visor.Servicios.GeneracionPDF.PDFUtils.GenerarVinculo(actualPage.ToString(), urlSection, baseFontBig );
estatus.Add(linkPagina);
resultado.paginaFinal = actualPage;
}
//This is for add to your cell or table
PdfPCell rightCell = new PdfPCell()
{
Border = PdfPCell.NO_BORDER,
Colspan = 3,
PaddingLeft = espacioInicial.Length,
ExtraParagraphSpace = 10,
};
rightCell.AddElement(estatus);
celdas.Add(rightCell);
And create a new method, this create a hyperlinkand you can invoque when you want
/*Generar Vinculo (create hyperlink)**/
public static Chunk GenerarVinculo(String tituloMostrar, string urlDirecion, iTextSharp.text.Font fuente)
{
Chunk espacioTab = new Chunk();
try
{
if (String.IsNullOrEmpty(urlDirecion))
urlDirecion = "Indice de Contenido";
espacioTab = new Chunk(tituloMostrar, fuente);
var accion = PdfAction.GotoLocalPage(urlDirecion, false);
espacioTab.SetAction(accion);
}
catch (Exception error) { }
return espacioTab;
}
Hope helps someone

iTextSharp AcroForm - multi-field not copying

I have a pdf with buttons that take you out to web links. I used iTextSharp to split these into separate PDFs (1 per page) per outside requirements. ISSUE: Any button that has multiple positions, lost the actions.
QUESTION: Does anyone know how to update these actions? I can open the new file, but I'm not sure how to go about using the PdfStamper to add an AA to this Annotation
So when opening the original file, you could get to the Additional Action by doing this:
var r = new PdfReader(f.FullName);
var positionsOfThisButton = r.AcroFields.GetFieldPositions("14");
var field = r.AcroForm.GetField("14")
var targetObject = PdfReader.GetPdfObject(field.Ref);
var kids = targetObject.GetAsArray(PdfName.KIDS);
foreach (var k in kids){
var ko = (PdfDictionary)(k.IsIndirect() ? PdfReader.GetPdfObject(k) : k);
var aaObj = ko.Get(PdfName.AA);
//(aaObj is NULL in the new file)
var aa = (PdfDictionary)(aaObj.IsIndirect() ? PdfReader.GetPdfObject(aaObj) : aaObj);
var dObj = aa.Get(PdfName.D);
var d = (PdfDictionary)(dObj.IsIndirect() ? PdfReader.GetPdfObject(dObj) : dObj);
Debug.WriteLine("S:" + d.GetAsName(PdfName.S).ToString() );
//returns S:/Uri
Debug.WriteLine("URI:" + d.GetAsString(PdfName.URI).ToString() );
//returns URI:http://www.somesite.com/etc
}
Thanks for any help.
FYI ONLY - The following is how I split the files:
List<byte[]> Get(FileInfo f) {
List<byte[]> outputFiles = new List<byte[]>();
var reader = new PdfReader(f.FullName);
int n = reader.NumberOfPages;
reader.Close();
for (int i = n; i > 0; i--) {
reader = new PdfReader(f.FullName);
using (var document = new Document(reader.GetPageSizeWithRotation(1))) {
using (var outputStream = new MemoryStream()) {
using (var writer = new PdfCopy(document, outputStream)) {
writer.SetMergeFields();
writer.PdfVersion = '6';
document.Open();
writer.AddDocument(reader, new List<int> { i });
document.Close();
writer.Close();
}
outputFiles.Insert(0, outputStream.ToArray());
}
}
reader.Close();
}
return outputFiles;
}

Merge a list of pdfs and create new bookmarks (C#)

The project is in C# and use iTextSharp.
I have a dictionary with a title (string) and file content (byte array). I loop through this dictionary and merge all files together. What I need now is to add bookmarks to the start of the first page in each file, but I should not add any new pages or text to the final document. I have tried different solutions, but all seem to add a table of contents page, a new page before each page or some text at the start of the page.
None of the files have bookmarks originally.
I am looking for a bookmarks structure that looks something like this:
File1
File2
SomeCategory
File3
File4
I would very much appreciate it if anyone could point me in the right direction.
My function for merging the files looks like this:
/// <summary>
/// Merge PDF files, and stamp certificates. This is a modified version of the example in the link below.
/// See: http://www.codeproject.com/Articles/28283/Simple-NET-PDF-Merger for more information.
/// </summary>
/// <param name="sourceFiles">Files to be merged</param>
/// <returns>Byte array with the combined files.</returns>
public static byte[] MergeFiles(Dictionary<string, byte[]> sourceFiles)
{
var document = new Document();
var output = new MemoryStream();
try
{
// Initialize pdf writer
var writer = PdfWriter.GetInstance(document, output);
writer.PageEvent = new PdfPageEvents();
// Open document to write
document.Open();
var content = writer.DirectContent;
// Iterate through all pdf documents
foreach (var sourceFile in sourceFiles)
{
// Create pdf reader
var reader = new PdfReader(sourceFile.Value);
var numberOfPages = reader.NumberOfPages;
// Iterate through all pages
for (var currentPageIndex = 1; currentPageIndex <=
numberOfPages; currentPageIndex++)
{
// Determine page size for the current page
document.SetPageSize(
reader.GetPageSizeWithRotation(currentPageIndex));
// Create page
document.NewPage();
var importedPage =
writer.GetImportedPage(reader, currentPageIndex);
// Determine page orientation
var pageOrientation = reader.GetPageRotation(currentPageIndex);
if ((pageOrientation == 90) || (pageOrientation == 270))
{
content.AddTemplate(importedPage, 0, -1f, 1f, 0, 0,
reader.GetPageSizeWithRotation(currentPageIndex).Height);
}
else
{
content.AddTemplate(importedPage, 1f, 0, 0, 1f, 0, 0);
}
// Add stamp to certificates
if (sourceFile.Key.IsValidDocumentReference())
AddStamp(content, document, sourceFile.Key, currentPageIndex, numberOfPages);
}
}
}
catch (Exception exception)
{
throw new Exception("An unexpected exception occured during the merging process", exception);
}
finally
{
document.Close();
}
return output.GetBuffer();
}

Thanks to Bruno Lowagie who pointed me in the right direction, I was able to produce a solution to my problem.
This is my solution:
public static byte[] MergeFilesAndAddBookmarks(Dictionary<PrintDocument, byte[]> sourceFiles)
{
using (var ms = new MemoryStream())
{
using (var document = new Document())
{
using (var copy = new PdfCopy(document, ms))
{
//Order the files by chapternumber
var files = sourceFiles.GroupBy(f => f.Key.ChapterNumber);
document.Open();
var outlines = new List<Dictionary<string, object>>();
var pageIndex = 1;
foreach (var chapterGroup in files)
{
var map = new Dictionary<string, object>();
outlines.Add(map);
map.Add("Title", chapterGroup.First().Key.ChapterName);
var kids = new List<Dictionary<string, object>>();
map.Add("Kids", kids);
foreach (var sourceFile in chapterGroup)
{
using (var reader = new PdfReader(sourceFile.Value))
{
// add the pages
var n = reader.NumberOfPages;
for (var page = 0; page < n;)
{
if (page == 0)
{
var kid = new Dictionary<string, object>();
kids.Add(kid);
kid["Title"] = sourceFile.Key.Title;
kid["Action"] = "GoTo";
kid["Page"] = String.Format("{0} Fit", pageIndex);
}
copy.AddPage(copy.GetImportedPage(reader, ++page));
}
pageIndex += n;
reader.Close();
}
}
}
copy.Outlines = outlines;
document.Close();
copy.Close();
ms.Close();
}
}
return ms.ToArray();
}
}
}
public class PrintDocument
{
public string Title { get; set; }
public string ChapterName { get; set; }
public int ChapterNumber { get; set; }
}

OpenXml-SDK: How to insert Plaintext with CarriageReturn/Linefeed

Currently I´m having a Word-Template with an RTF-FormField in it, in this FormField I want to insert Plaintext that comes from a MSSQL-Database.
With this Method I´m setting the Content of any FormField in my Document(including my RTF-FormField)
public static void SetContentControlValue(this WordprocessingDocument doc, string name, string value)
{
var main = doc.MainDocumentPart;
var stdRuns = main.Document.Body
.Descendants<SdtRun>()
.Where(r => r.SdtProperties.GetFirstChild<Tag>().Val.Value.ToLower().Equals(name))
.ToList();
stdRuns.ForEach(c => c.Descendants<Text>().First().Text = value);
main.Document.Body
.Descendants<SdtBlock>()
.Where(r => r.SdtProperties.GetFirstChild<Tag>().Val.Value.ToLower().Equals(name))
.ToList()
.ForEach(c => c.Descendants<Text>().First().Text = value);
}
Unfortunately it swallows CarriageReturn/Linefeed
I tried to do it this way
var run = new SdtRun();
string[] newLineArray = { Environment.NewLine };
string[] textArray = value.Split(newLineArray, StringSplitOptions.None);
bool first = true;
foreach (var line in textArray)
{
if (!first)
{
run.Append(new Break());
}
first = false;
Text txt = new Text();
txt.Text = line;
run.Append(txt);
}
main.Document.Body.Append(run);
but unfortunately this breaks the WordDocument and I can´t open it anymore :-/
Anybody here maybe had the same Problem and has an Idea how I can insert the Text without losing the Plaintext-Formatting (CarriageReturns and Linefeeds)?

So finally I found a Solution by myself, you can use AltChunk to Insert Plaintext and not losing your formatting :-)
Here is the Code maybe it helps someone out who has the same Problem ;-)
public static void SetNotes(this WordprocessingDocument doc, string value)
{
MainDocumentPart main = doc.MainDocumentPart;
string altChunkId = "AltChunkId" + Guid.NewGuid().ToString().Replace("-", "");
var chunk = main.AddAlternativeFormatImportPart(AlternativeFormatImportPartType.TextPlain, altChunkId);
using (var mStream = new MemoryStream())
{
using (var writer = new StreamWriter(mStream))
{
writer.Write(value);
writer.Flush();
mStream.Position = 0;
chunk.FeedData(mStream);
}
}
var altChunk = new AltChunk();
altChunk.Id = altChunkId;
OpenXmlElement afterThat = null;
foreach (var para in main.Document.Body.Descendants<Paragraph>())
{
if (para.InnerText.Equals("Notes:"))
{
afterThat = para;
}
}
main.Document.Body.InsertAfter(altChunk, afterThat);
}