Develop Reference

Lucene query documents where name matches against collection

I have lucene documents with below structure
{
name : "A",
id :1
},
{
name : "B",
id :1
},
{
name : "C",
id :3
}
Now I have a collection like List which contains A, B. I wanted to select documents where name is A or B . So as per above lucene documents I should have documents A and B . I wanted to fetch these 2 documents with a single lucene call instead of multiple lucene calls for each document.
i tried with BooleanQuery and adding my search query in a loop but the search query did not return anything. if I hit lucene with single document it works and returns a single document.
Could anyone please suggest How I can retrieve all matching documents with a single query ?
I tried something like below
List<string> terms = new List<string>(){'A', 'B'};
var mainQuery = new BooleanQuery);
var parser = new QueryParser(Lucene.Net.Util.Version.LUCENE_30, "name", analyzer);
foreach (var term in terms)
{
var query = parser.Parse(term);
mainQuery.Add(query, Occur.MUST_NOT);
}
var hits = _searcher.Search(mainQuery, 1000);
Above query did not work and returns 0 result .

I am able to resolve this by my own.It's just a simple OR clause which is Occur.SHOULD
var booleanQuery = new BooleanQuery();
foreach (var term in terms)
{
var termQuery = new TermQuery(new Term("name", term ));
booleanQuery.Add(termQuery, Occur.SHOULD);
}

Lucene search creteria change the word

I use lucene for searching.
For each doc in index I have some field called "uniqueIdentifier" with type string.
When I want to find all items with "uniqueIdentifier" == "haaglanden", I use the next code:
var searcher = Examine.ExamineManager.Instance.SearchProviderCollection["RegionsSearcher"];
var searchCriteria = searcher.CreateSearchCriteria(BooleanOperation.And);
var temp = searchCriteria.RawQuery("+uniqueIdentifier:" + uniqueIdentifier);
In temp I see :
LuceneQuery: {+(+uniqueIdentifier:haagland)}
But "haagland" != "haaglanden".
And I can not find my docs.
How can I build query with "haaglanden"?

The cause was in analyzer.
Swiched Lucene.Net.Analysis.Nl.DutchAnalyzer to Lucene.Net.Analysis.Standard.StandardAnalyzer.

Lucene .NET searching

Hi i am trying to make autocomplete system using Lucene library to search over 170K records.
But there is a litle problem.
For example when i search for Candice Gra(...), it brings records like
Candice Jackson
Candice Hamilton
Candice Hayes
Bu not Candice Graham to make Lucene find Candice Graham i need to type Candice Graham exactly.
Here is the code that i'm building query.
Directory directory = FSDirectory.Open(new DirectoryInfo(context.Server.MapPath("
ISet<string> stopWordSet = new HashSet<string>(stopWords);
Analyzer analyzer = new StandardAnalyzer(Version.LUCENE_30, stopWordSet);
IndexReader indexReader = IndexReader.Open(directory, true);
Searcher indexSearch = new IndexSearcher(indexReader);
//Singe Field Search
var queryParser = new QueryParser(Version.LUCENE_30,
"Title",
analyzer);
string strQuery = string.Format("{0}", q);
var query = queryParser.Parse(strQuery);
If i build strQuery like this (* appended to the query)
string strQuery = string.Format("{0}*", q);
But using this way brings irrelevant records too.
For example if i search Candice Gra(...) again it returns records like
Grass
Gravity
Gray (etc.)
By the way i used KeywordAnalyzer and SimpleAnalyzer but these are not worked either.
Any ideas?

You should escape your spaces if you want them included in the search;
var query = queryParser.Parse(QueryParser.Escape(strQuery));

I think you need to put a AND keyword between these two words.
"Candice" AND "Gra"
http://lucene.apache.org/core/2_9_4/queryparsersyntax.html#AND

How to do regular expression search using lucene.Net

I m using lucene.Net version 3.0.3. I want to do regular expression search. I tried the following code:
// code
String SearchExpression = "[DM]ouglas";
const int hitsLimit = 1000000;
//state the file location of the index
string indexFileLocation = IndexLocation;
Lucene.Net.Store.Directory dir = Lucene.Net.Store.FSDirectory.Open(indexFileLocation);
//create an index searcher that will perform the search
Lucene.Net.Search.IndexSearcher searcher = new Lucene.Net.Search.IndexSearcher(dir);
var analyzer = new WhitespaceAnalyzer();
var parser = new MultiFieldQueryParser(Lucene.Net.Util.Version.LUCENE_30, new[] {
Field_Content, }, analyzer);
Term t = new Term(Field_Content, SearchExpression);
RegexQuery scriptQuery = new RegexQuery(t);
string s = string.Format("{0}", SearchExpression);
var query = parser.Parse(s);
BooleanQuery booleanQuery = new BooleanQuery();
booleanQuery.Add(query, Occur.MUST);
var hits = searcher.Search(booleanQuery, null, hitsLimit, Sort.RELEVANCE).ScoreDocs;
foreach (var hit in hits)
{
var hitDocument = searcher.Doc(hit.Doc);
string contentValue = hitDocument.Get(Field_Content);
}
// end of code
When I try to search with patten "Do*uglas", I get the results.
But if I search with the pattern "[DM]ouglas]" it is giving me the following error:
"Cannot parse '[DM]ouglas': Encountered " "]" "] "" at line 1, column 3. Was expecting one of: "TO" ... <RANGEIN_QUOTED> ... <RANGEIN_GOOP> ...".
I also tried doing simple search pattern like ".ouglas" which should give me results, as I have "Douglas" in my text content.
Does anyone know how to do regular expression search using lucene.Net version 3.0.3?

The StandardQueryParser does not support regular expressions at all. It is, instead, attempting to interpret that portion of the query as a range query.
I you wish to use regexes to search, you will need to construct a RegexQuery manually. Note, that RegexQuery performance tends to be poor. You might be able to improve it by switching from JavaUtilRegexCapabilities to JakartaRegexpCapabilities.

Why does this Lucene.Net query fail?

I am trying to convert my search functionality to allow for fuzzy searches involving multiple words. My existing search code looks like:
// Split the search into seperate queries per word, and combine them into one major query
var finalQuery = new BooleanQuery();
string[] terms = searchString.Split(new[] { " " }, StringSplitOptions.RemoveEmptyEntries);
foreach (string term in terms)
{
// Setup the fields to search
string[] searchfields = new string[]
{
// Various strings denoting the document fields available
};
var parser = new MultiFieldQueryParser(Lucene.Net.Util.Version.LUCENE_29, searchfields, new StandardAnalyzer(Lucene.Net.Util.Version.LUCENE_29));
finalQuery.Add(parser.Parse(term), BooleanClause.Occur.MUST);
}
// Perform the search
var directory = FSDirectory.Open(new DirectoryInfo(LuceneIndexBaseDirectory));
var searcher = new IndexSearcher(directory, true);
var hits = searcher.Search(finalQuery, MAX_RESULTS);
This works correctly, and if I have an entity with the name field of "My name is Andrew", and I perform a search for "Andrew Name", Lucene correctly finds the correct document. Now I want to enable fuzzy searching, so that "Anderw Name" is found correctly. I changed my method to use the following code:
const int MAX_RESULTS = 10000;
const float MIN_SIMILARITY = 0.5f;
const int PREFIX_LENGTH = 3;
if (string.IsNullOrWhiteSpace(searchString))
throw new ArgumentException("Provided search string is empty");
// Split the search into seperate queries per word, and combine them into one major query
var finalQuery = new BooleanQuery();
string[] terms = searchString.Split(new[] { " " }, StringSplitOptions.RemoveEmptyEntries);
foreach (string term in terms)
{
// Setup the fields to search
string[] searchfields = new string[]
{
// Strings denoting document field names here
};
// Create a subquery where the term must match at least one of the fields
var subquery = new BooleanQuery();
foreach (string field in searchfields)
{
var queryTerm = new Term(field, term);
var fuzzyQuery = new FuzzyQuery(queryTerm, MIN_SIMILARITY, PREFIX_LENGTH);
subquery.Add(fuzzyQuery, BooleanClause.Occur.SHOULD);
}
// Add the subquery to the final query, but make at least one subquery match must be found
finalQuery.Add(subquery, BooleanClause.Occur.MUST);
}
// Perform the search
var directory = FSDirectory.Open(new DirectoryInfo(LuceneIndexBaseDirectory));
var searcher = new IndexSearcher(directory, true);
var hits = searcher.Search(finalQuery, MAX_RESULTS);
Unfortunately, with this code if I submit the search query "Andrew Name" (same as before) I get zero results back.
The core idea is that all terms must be found in at least one document field, but each term can reside in different fields. Does anyone have any idea why my rewritten query fails?
Final Edit: Ok it turns out I was over complicating this by a LOT, and there was no need to change from my first approach. After reverting back to the first code snippet, I enabled fuzzy searching by changing
finalQuery.Add(parser.Parse(term), BooleanClause.Occur.MUST);
to
finalQuery.Add(parser.Parse(term.Replace("~", "") + "~"), BooleanClause.Occur.MUST);

Your code works for me if I rewrite the searchString to lower-case. I'm assuming that you're using the StandardAnalyzer when indexing, and it will generate lower-case terms.
You need to 1) pass your tokens through the same analyzer (to enable identical processing), 2) apply the same logic as the analyzer or 3) use an analyzer which matches the processing you do (WhitespaceAnalyzer).

You want this line:
var queryTerm = new Term(term);
to look like this:
var queryTerm = new Term(field, term);
Right now you're searching field term (which probably doesn't exist) for the empty string (which will never be found).