org.apache.lucene.index
Class LogMergePolicy

java.lang.Object
  org.apache.lucene.index.MergePolicy
      org.apache.lucene.index.LogMergePolicy

Direct Known Subclasses:

LogByteSizeMergePolicy, LogDocMergePolicy

public abstract class LogMergePolicy
extends MergePolicy

This class implements a MergePolicy that tries to merge segments into levels of exponentially increasing size, where each level has < mergeFactor segments in it. Whenever a given levle has mergeFactor segments or more in it, they will be merged.

This class is abstract and requires a subclass to define the size(org.apache.lucene.index.SegmentInfo) method which specifies how a segment's size is determined. LogDocMergePolicy is one subclass that measures size by document count in the segment. LogByteSizeMergePolicy is another subclass that measures size as the total byte size of the file(s) for the segment.

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.lucene.index.MergePolicy

MergePolicy.MergeAbortedException, MergePolicy.MergeException, MergePolicy.MergeSpecification, MergePolicy.OneMerge

Field Summary

static int	DEFAULT_MAX_MERGE_DOCS Default maximum segment size.
static int	DEFAULT_MERGE_FACTOR Default merge factor, which is how many segments are merged at a time
static double	LEVEL_LOG_SPAN Defines the allowed range of log(size) for each level.

Constructor Summary

LogMergePolicy()

Method Summary

void	close() Release all resources for the policy.
MergePolicy.MergeSpecification	findMerges(org.apache.lucene.index.SegmentInfos infos, IndexWriter writer) Checks if any merges are now necessary and returns a MergePolicy.MergeSpecification if so.
MergePolicy.MergeSpecification	findMergesForOptimize(org.apache.lucene.index.SegmentInfos infos, IndexWriter writer, int maxNumSegments, Set segmentsToOptimize) Returns the merges necessary to optimize the index.
MergePolicy.MergeSpecification	findMergesToExpungeDeletes(org.apache.lucene.index.SegmentInfos segmentInfos, IndexWriter writer) Finds merges necessary to expunge all deletes from the index.
int	getMaxMergeDocs() Returns the largest segment (measured by document count) that may be merged with other segments.
int	getMergeFactor() Returns the number of segments that are merged at once and also controls the total number of segments allowed to accumulate in the index.
boolean	getUseCompoundDocStore() Returns true if newly flushed and newly merge doc store segment files (term vectors and stored fields) are written in compound file format.
boolean	getUseCompoundFile() Returns true if newly flushed and newly merge segments are written in compound file format.
void	setMaxMergeDocs(int maxMergeDocs) Determines the largest segment (measured by document count) that may be merged with other segments.
void	setMergeFactor(int mergeFactor) Determines how often segment indices are merged by addDocument().
void	setUseCompoundDocStore(boolean useCompoundDocStore) Sets whether compound file format should be used for newly flushed and newly merged doc store segment files (term vectors and stored fields).
void	setUseCompoundFile(boolean useCompoundFile) Sets whether compound file format should be used for newly flushed and newly merged segments.
protected abstract long	size(org.apache.lucene.index.SegmentInfo info)
boolean	useCompoundDocStore(org.apache.lucene.index.SegmentInfos infos) Returns true if the doc store files should use the compound file format.
boolean	useCompoundFile(org.apache.lucene.index.SegmentInfos infos, org.apache.lucene.index.SegmentInfo info) Returns true if a newly flushed (not from merge) segment should use the compound file format.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

LEVEL_LOG_SPAN

public static final double LEVEL_LOG_SPAN

Defines the allowed range of log(size) for each level. A level is computed by taking the max segment log size, minuse LEVEL_LOG_SPAN, and finding all segments falling within that range.

See Also:

Constant Field Values

DEFAULT_MERGE_FACTOR

public static final int DEFAULT_MERGE_FACTOR

Default merge factor, which is how many segments are merged at a time

See Also:

Constant Field Values

DEFAULT_MAX_MERGE_DOCS

public static final int DEFAULT_MAX_MERGE_DOCS

Default maximum segment size. A segment of this size or larger will never be merged. @see setMaxMergeDocs

See Also:

Constant Field Values

Constructor Detail

LogMergePolicy

public LogMergePolicy()

Method Detail

getMergeFactor

public int getMergeFactor()

Returns the number of segments that are merged at once and also controls the total number of segments allowed to accumulate in the index.

setMergeFactor

public void setMergeFactor(int mergeFactor)

Determines how often segment indices are merged by addDocument(). With smaller values, less RAM is used while indexing, and searches on unoptimized indices are faster, but indexing speed is slower. With larger values, more RAM is used during indexing, and while searches on unoptimized indices are slower, indexing is faster. Thus larger values (> 10) are best for batch index creation, and smaller values (< 10) for indices that are interactively maintained.

useCompoundFile

public boolean useCompoundFile(org.apache.lucene.index.SegmentInfos infos,
                               org.apache.lucene.index.SegmentInfo info)

Description copied from class: MergePolicy

Returns true if a newly flushed (not from merge) segment should use the compound file format.

setUseCompoundFile

public void setUseCompoundFile(boolean useCompoundFile)

Sets whether compound file format should be used for newly flushed and newly merged segments.

getUseCompoundFile

public boolean getUseCompoundFile()

Returns true if newly flushed and newly merge segments are written in compound file format. @see #setUseCompoundFile

useCompoundDocStore

public boolean useCompoundDocStore(org.apache.lucene.index.SegmentInfos infos)

Description copied from class: MergePolicy

Returns true if the doc store files should use the compound file format.

setUseCompoundDocStore

public void setUseCompoundDocStore(boolean useCompoundDocStore)

Sets whether compound file format should be used for newly flushed and newly merged doc store segment files (term vectors and stored fields).

getUseCompoundDocStore

public boolean getUseCompoundDocStore()

Returns true if newly flushed and newly merge doc store segment files (term vectors and stored fields) are written in compound file format. @see #setUseCompoundDocStore

close

public void close()

Description copied from class: MergePolicy

Release all resources for the policy.

size

protected abstract long size(org.apache.lucene.index.SegmentInfo info)
                      throws IOException

Throws:

IOException

findMergesForOptimize

public MergePolicy.MergeSpecification findMergesForOptimize(org.apache.lucene.index.SegmentInfos infos,
                                                            IndexWriter writer,
                                                            int maxNumSegments,
                                                            Set segmentsToOptimize)
                                                     throws IOException

Returns the merges necessary to optimize the index. This merge policy defines "optimized" to mean only one segment in the index, where that segment has no deletions pending nor separate norms, and it is in compound file format if the current useCompoundFile setting is true. This method returns multiple merges (mergeFactor at a time) so the MergeScheduler in use may make use of concurrency.

Parameters:

infos - the total set of segments in the index

writer - IndexWriter instance

maxNumSegments - requested maximum number of segments in the index (currently this is always 1)

segmentsToOptimize - contains the specific SegmentInfo instances that must be merged away. This may be a subset of all SegmentInfos.

Throws:

IOException

findMergesToExpungeDeletes

public MergePolicy.MergeSpecification findMergesToExpungeDeletes(org.apache.lucene.index.SegmentInfos segmentInfos,
                                                                 IndexWriter writer)
                                                          throws CorruptIndexException,
                                                                 IOException

Finds merges necessary to expunge all deletes from the index. We simply merge adjacent segments that have deletes, up to mergeFactor at a time.

Parameters:

segmentInfos - the total set of segments in the index

writer - IndexWriter instance

Throws:

CorruptIndexException

IOException

findMerges

public MergePolicy.MergeSpecification findMerges(org.apache.lucene.index.SegmentInfos infos,
                                                 IndexWriter writer)
                                          throws IOException

Checks if any merges are now necessary and returns a MergePolicy.MergeSpecification if so. A merge is necessary when there are more than setMergeFactor(int) segments at a given level. When multiple levels have too many segments, this method will return multiple merges, allowing the MergeScheduler to use concurrency.

Parameters:

infos - the total set of segments in the index

writer - IndexWriter instance

Throws:

IOException

setMaxMergeDocs

public void setMaxMergeDocs(int maxMergeDocs)

Determines the largest segment (measured by document count) that may be merged with other segments. Small values (e.g., less than 10,000) are best for interactive indexing, as this limits the length of pauses while indexing to a few seconds. Larger values are best for batched indexing and speedier searches.

The default value is Integer.MAX_VALUE.

The default merge policy (LogByteSizeMergePolicy) also allows you to set this limit by net size (in MB) of the segment, using LogByteSizeMergePolicy.setMaxMergeMB(double).

getMaxMergeDocs

public int getMaxMergeDocs()

Returns the largest segment (measured by document count) that may be merged with other segments.

See Also:

setMaxMergeDocs(int)