org.apache.lucene.codecs.memory

Class DirectPostingsFormat

java.lang.Object

org.apache.lucene.codecs.PostingsFormat

org.apache.lucene.codecs.memory.DirectPostingsFormat

All Implemented Interfaces:

NamedSPILoader.NamedSPI

public final class DirectPostingsFormat
extends PostingsFormat

Wraps Lucene84PostingsFormat format for on-disk storage, but then at read time loads and stores all terms and postings directly in RAM as byte[], int[].

WARNING: This is exceptionally RAM intensive: it makes no effort to compress the postings data, storing terms as separate byte[] and postings as separate int[], but as a result it gives substantial increase in search performance.

This postings format supports TermsEnum.ord() and TermsEnum.seekExact(long).

Because this holds all term bytes as a single byte[], you cannot have more than 2.1GB worth of term bytes in a single segment.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
private static class	DirectPostingsFormat.DirectField
private static class	DirectPostingsFormat.DirectFields
private static class	DirectPostingsFormat.HighFreqDocsEnum
private static class	DirectPostingsFormat.HighFreqPostingsEnum
private static class	DirectPostingsFormat.LowFreqDocsEnum
private static class	DirectPostingsFormat.LowFreqDocsEnumNoPos
private static class	DirectPostingsFormat.LowFreqDocsEnumNoTF
private static class	DirectPostingsFormat.LowFreqPostingsEnum

Field Summary

Fields

Modifier and Type	Field and Description
private static int	DEFAULT_LOW_FREQ_CUTOFF
private static int	DEFAULT_MIN_SKIP_COUNT
private int	lowFreqCutoff
private int	minSkipCount

Fields inherited from class org.apache.lucene.codecs.PostingsFormat

EMPTY

Constructor Summary

Constructors

Constructor and Description
DirectPostingsFormat()
DirectPostingsFormat(int minSkipCount, int lowFreqCutoff) minSkipCount is how many terms in a row must have the same prefix before we put a skip pointer down.

Method Summary

Modifier and Type	Method and Description
FieldsConsumer	fieldsConsumer(SegmentWriteState state) Writes a new segment
FieldsProducer	fieldsProducer(SegmentReadState state) Reads a segment.

Methods inherited from class org.apache.lucene.codecs.PostingsFormat

availablePostingsFormats, forName, getName, reloadPostingsFormats, toString

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

minSkipCount

private final int minSkipCount

lowFreqCutoff

private final int lowFreqCutoff

DEFAULT_MIN_SKIP_COUNT

private static final int DEFAULT_MIN_SKIP_COUNT

See Also:

Constant Field Values

DEFAULT_LOW_FREQ_CUTOFF

private static final int DEFAULT_LOW_FREQ_CUTOFF

See Also:

Constant Field Values

Constructor Detail

DirectPostingsFormat

public DirectPostingsFormat()

DirectPostingsFormat

public DirectPostingsFormat(int minSkipCount,
                            int lowFreqCutoff)

minSkipCount is how many terms in a row must have the same prefix before we put a skip pointer down. Terms with docFreq <= lowFreqCutoff will use a single int[] to hold all docs, freqs, position and offsets; terms with higher docFreq will use separate arrays.

Method Detail

fieldsConsumer

public FieldsConsumer fieldsConsumer(SegmentWriteState state)
                              throws java.io.IOException

Description copied from class: PostingsFormat

Writes a new segment

Specified by:

fieldsConsumer in class PostingsFormat

Throws:

java.io.IOException

fieldsProducer

public FieldsProducer fieldsProducer(SegmentReadState state)
                              throws java.io.IOException

Description copied from class: PostingsFormat

Reads a segment. NOTE: by the time this call returns, it must hold open any files it will need to use; else, those files may be deleted. Additionally, required files may be deleted during the execution of this call before there is a chance to open them. Under these circumstances an IOException should be thrown by the implementation. IOExceptions are expected and will automatically cause a retry of the segment opening logic with the newly revised segments.

Specified by:

fieldsProducer in class PostingsFormat

Throws:

java.io.IOException