Lucene 2.4.0 API

Apache Lucene is a high-performance, full-featured text search engine library.

See:
Description

Core

org.apache.lucene	Top-level package.
org.apache.lucene.analysis	API and code to convert text into indexable/searchable tokens.
org.apache.lucene.analysis.standard	A fast grammar-based tokenizer constructed with JFlex.
org.apache.lucene.document	The logical representation of a Document for indexing and searching.
org.apache.lucene.index	Code to maintain and access indices.
org.apache.lucene.queryParser	A simple query parser implemented with JavaCC.
org.apache.lucene.search	Code to search indices.
org.apache.lucene.search.function	Programmatic control over documents scores.
org.apache.lucene.search.payloads	The payloads package provides Query mechanisms for finding and using payloads.
org.apache.lucene.search.spans	The calculus of spans.
org.apache.lucene.store	Binary i/o API, used for all index data.
org.apache.lucene.util	Some utility classes.
org.apache.lucene.util.cache

Demo

org.apache.lucene.demo
org.apache.lucene.demo.html

contrib: Analysis

org.apache.lucene.analysis.br	Analyzer for Brazilian.
org.apache.lucene.analysis.cjk	Analyzer for Chinese, Japanese and Korean.
org.apache.lucene.analysis.cn	Analyzer for Chinese.
org.apache.lucene.analysis.compound	A filter that decomposes compound words you find in many Germanic languages to the word parts.
org.apache.lucene.analysis.compound.hyphenation	The code for the compound word hyphenation is taken from the Apache FOP project.
org.apache.lucene.analysis.cz	Analyzer for Czech.
org.apache.lucene.analysis.de	Analyzer for German.
org.apache.lucene.analysis.el	Analyzer for Greek.
org.apache.lucene.analysis.fr	Analyzer for French.
org.apache.lucene.analysis.miscellaneous
org.apache.lucene.analysis.ngram
org.apache.lucene.analysis.nl	Analyzer for Dutch.
org.apache.lucene.analysis.payloads	Provides various convenience classes for creating payloads on Tokens.
org.apache.lucene.analysis.query
org.apache.lucene.analysis.ru	Analyzer for Russian.
org.apache.lucene.analysis.shingle
org.apache.lucene.analysis.sinks	Implementations of the SinkTokenizer that might be useful.
org.apache.lucene.analysis.th

contrib: Ant

org.apache.lucene.ant

Ant task to create Lucene indexes.

contrib: Benchmark

org.apache.lucene.benchmark
org.apache.lucene.benchmark.byTask	Benchmarking Lucene By Tasks.
org.apache.lucene.benchmark.byTask.feeds	Sources for benchmark inputs: documents and queries.
org.apache.lucene.benchmark.byTask.programmatic	Sample performance test written programatically - no algorithm file is needed here.
org.apache.lucene.benchmark.byTask.stats	Statistics maintained when running benchmark tasks.
org.apache.lucene.benchmark.byTask.tasks	Extendable benchmark tasks.
org.apache.lucene.benchmark.byTask.utils	Utilities used for the benchmark, and for the reports.
org.apache.lucene.benchmark.quality	Search Quality Benchmarking.
org.apache.lucene.benchmark.quality.trec	Utilities for Trec related quality benchmarking, feeding from Trec Topics and QRels inputs.
org.apache.lucene.benchmark.quality.utils	Miscellaneous utilities for search quality benchmarking: query parsing, submission reports.
org.apache.lucene.benchmark.standard
org.apache.lucene.benchmark.stats
org.apache.lucene.benchmark.utils

contrib: DB

com.sleepycat.db
org.apache.lucene.store.db
org.apache.lucene.store.je

contrib: Highlighter

org.apache.lucene.search.highlight

The highlight package contains classes to provide "keyword in context" features typically used to highlight search terms in the text of results pages.

contrib: Instantiated

org.apache.lucene.store.instantiated

InstantiatedIndex, alternative RAM store for small corpora.

contrib: Lucli

lucli

Lucene Command Line Interface

contrib: Memory

org.apache.lucene.index.memory

High-performance single-document main memory Apache Lucene fulltext search index.

contrib: Miscellaneous

org.apache.lucene.misc
org.apache.lucene.queryParser.analyzing
org.apache.lucene.queryParser.precedence

contrib: Queries

org.apache.lucene.search.similar

Document similarity query generators.

contrib: RegEx

org.apache.lucene.search.regex	Regular expression Query.
org.apache.regexp	This package exists to allow access to useful package protected data within Jakarta Regexp.

contrib: Snowball

org.apache.lucene.analysis.snowball

TokenFilter and Analyzer implementations that use Snowball stemmers.

contrib: SpellChecker

org.apache.lucene.search.spell

Suggest alternate spellings for words.

contrib: Surround Parser

org.apache.lucene.queryParser.surround.parser	This package contains the QueryParser.jj source file for the Surround parser.
org.apache.lucene.queryParser.surround.query	This package contains SrndQuery and its subclasses.

contrib: Swing

org.apache.lucene.swing.models

Decorators for JTable TableModel and JList ListModel encapsulating Lucene indexing and searching functionality.

contrib: Wikipedia

org.apache.lucene.wikipedia.analysis

contrib: WordNet

org.apache.lucene.wordnet

This package uses synonyms defined by WordNet to build a Lucene index storing them, which in turn can be used for query expansion.

contrib: XML Query Parser

org.apache.lucene.xmlparser
org.apache.lucene.xmlparser.builders

Other Packages

org.tartarus.snowball
org.tartarus.snowball.ext

Apache Lucene is a high-performance, full-featured text search engine library. Here's a simple example how to use Lucene for indexing and searching (using JUnit to check if the results are what we expect):

Analyzer analyzer = new StandardAnalyzer();

// Store the index in memory:
Directory directory = new RAMDirectory();
// To store an index on disk, use this instead:
//Directory directory = FSDirectory.getDirectory("/tmp/testindex");
IndexWriter iwriter = new IndexWriter(directory, analyzer, true);
iwriter.setMaxFieldLength(25000);
Document doc = new Document();
String text = "This is the text to be indexed.";
doc.add(new Field("fieldname", text, Field.Store.YES,
Field.Index.TOKENIZED));
iwriter.addDocument(doc);
iwriter.optimize();
iwriter.close();

// Now search the index:
IndexSearcher isearcher = new IndexSearcher(directory);
// Parse a simple query that searches for "text":
QueryParser parser = new QueryParser("fieldname", analyzer);
Query query = parser.parse("text");
Hits hits = isearcher.search(query);
assertEquals(1, hits.length());
// Iterate through the results:
for (int i = 0; i < hits.length(); i++) {
Document hitDoc = hits.doc(i);
assertEquals("This is the text to be indexed.", hitDoc.get("fieldname"));
}
isearcher.close();
directory.close();

The Lucene API is divided into several packages:

• org.apache.lucene.analysis defines an abstract Analyzer API for converting text from a java.io.Reader into a TokenStream, an enumeration of Tokens. A TokenStream is composed by applying TokenFilters to the output of a Tokenizer. A few simple implemenations are provided, including StopAnalyzer and the grammar-based StandardAnalyzer.
• org.apache.lucene.document provides a simple Document class. A document is simply a set of named Fields, whose values may be strings or instances of java.io.Reader.
• org.apache.lucene.index provides two primary classes: IndexWriter, which creates and adds documents to indices; and IndexReader, which accesses the data in the index.
• org.apache.lucene.search provides data structures to represent queries (TermQuery for individual words, PhraseQuery for phrases, and BooleanQuery for boolean combinations of queries) and the abstract Searcher which turns queries into Hits. IndexSearcher implements search over a single IndexReader.
• org.apache.lucene.queryParser uses JavaCC to implement a QueryParser.
• org.apache.lucene.store defines an abstract class for storing persistent data, the Directory, a collection of named files written by an IndexOutput and read by an IndexInput. Two implementations are provided, FSDirectory, which uses a file system directory to store files, and RAMDirectory which implements files as memory-resident data structures.
• org.apache.lucene.util contains a few handy data structures, e.g., BitVector and PriorityQueue.

To use Lucene, an application should:

1. Create Documents by adding Fields;
2. Create an IndexWriter and add documents to it with addDocument();
3. Call QueryParser.parse() to build a query from a string; and
4. Create an IndexSearcher and pass the query to its search() method.

Some simple examples of code which does this are:

• FileDocument.java contains code to create a Document for a file.
• IndexFiles.java creates an index for all the files contained in a directory.
• DeleteFiles.java deletes some of these files from the index.
• SearchFiles.java prompts for queries and searches an index.

To demonstrate these, try something like:

> java -cp lucene.jar:lucene-demo.jar org.apache.lucene.demo.IndexFiles rec.food.recipes/soups
adding rec.food.recipes/soups/abalone-chowder
[ ... ]

> java -cp lucene.jar:lucene-demo.jar org.apache.lucene.demo.SearchFiles
Query: chowder
Searching for: chowder
34 total matching documents
1. rec.food.recipes/soups/spam-chowder
[ ... thirty-four documents contain the word "chowder" ... ]

Query: "clam chowder" AND Manhattan
Searching for: +"clam chowder" +manhattan
2 total matching documents
1. rec.food.recipes/soups/clam-chowder
[ ... two documents contain the phrase "clam chowder" and the word "manhattan" ... ]
[ Note: "+" and "-" are canonical, but "AND", "OR" and "NOT" may be used. ]

The IndexHTML demo is more sophisticated. It incrementally maintains an index of HTML files, adding new files as they appear, deleting old files as they disappear and re-indexing files as they change.

> java -cp lucene.jar:lucene-demo.jar org.apache.lucene.demo.IndexHTML -create java/jdk1.1.6/docs/relnotes
adding java/jdk1.1.6/docs/relnotes/SMICopyright.html
[ ... create an index containing all the relnotes ]

> rm java/jdk1.1.6/docs/relnotes/smicopyright.html

> java -cp lucene.jar:lucene-demo.jar org.apache.lucene.demo.IndexHTML java/jdk1.1.6/docs/relnotes
deleting java/jdk1.1.6/docs/relnotes/SMICopyright.html