org.apache.lucene.search.suggest.document

Class CompletionPostingsFormat

java.lang.Object

org.apache.lucene.codecs.PostingsFormat

org.apache.lucene.search.suggest.document.CompletionPostingsFormat

All Implemented Interfaces:

NamedSPILoader.NamedSPI

Direct Known Subclasses:

Completion50PostingsFormat, Completion84PostingsFormat

public abstract class CompletionPostingsFormat
extends PostingsFormat

A PostingsFormat which supports document suggestion based on indexed SuggestFields. Document suggestion is based on an weighted FST which map analyzed terms of a SuggestField to its surface form and document id.

Files:

• .lkp: Completion Dictionary
• .cmp: Completion Index

Completion Dictionary

The .lkp file contains an FST for each suggest field

• CompletionDict (.lkp) --> Header, FST^{NumSuggestFields}, Footer
• Header --> CodecHeader
• FST --> FST<Long, BytesRef>
• Footer --> CodecFooter

Notes:

• Header is a CodecHeader storing the version information for the Completion implementation.
• FST maps all analyzed forms to surface forms of a SuggestField

Completion Index

The .cmp file contains an index into the completion dictionary, so that it can be accessed randomly.

• CompletionIndex (.cmp) --> Header, NumSuggestFields, Entry^{NumSuggestFields}, Footer
• Header --> CodecHeader
• NumSuggestFields --> Uint32
• Entry --> FieldNumber, CompletionDictionaryOffset, MinWeight, MaxWeight, Type
• FieldNumber --> Uint32
• CompletionDictionaryOffset --> Uint64
• MinWeight --> Uint64
• MaxWeight --> Uint64
• Type --> Byte
• Footer --> CodecFooter

Notes:

• Header is a CodecHeader storing the version information for the Completion implementation.
• NumSuggestFields is the number of suggest fields indexed
• FieldNumber is the fields number from FieldInfos. (.fnm)
• CompletionDictionaryOffset is the file offset of a field's FST in CompletionDictionary (.lkp)
• MinWeight and MaxWeight are the global minimum and maximum weight for the field
• Type indicates if the suggester has context or not

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	CompletionPostingsFormat.FSTLoadMode An enum that allows to control if suggester FSTs are loaded into memory or read off-heap

Field Summary

Fields

Modifier and Type	Field and Description
(package private) static int	COMPLETION_CODEC_VERSION
(package private) static int	COMPLETION_VERSION_CURRENT
(package private) static java.lang.String	DICT_EXTENSION
private CompletionPostingsFormat.FSTLoadMode	fstLoadMode
(package private) static java.lang.String	INDEX_EXTENSION

Fields inherited from class org.apache.lucene.codecs.PostingsFormat

EMPTY

Constructor Summary

Constructors

Constructor and Description
CompletionPostingsFormat(java.lang.String name) Used only by core Lucene at read-time via Service Provider instantiation
CompletionPostingsFormat(java.lang.String name, CompletionPostingsFormat.FSTLoadMode fstLoadMode) Creates a CompletionPostingsFormat that will use the provided fstLoadMode to determine if the completion FST should be loaded on or off heap.

Method Summary

Modifier and Type	Method and Description
protected abstract PostingsFormat	delegatePostingsFormat() Concrete implementation should specify the delegating postings format
FieldsConsumer	fieldsConsumer(SegmentWriteState state) Writes a new segment
FieldsProducer	fieldsProducer(SegmentReadState state) Reads a segment.

Methods inherited from class org.apache.lucene.codecs.PostingsFormat

availablePostingsFormats, forName, getName, reloadPostingsFormats, toString

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

COMPLETION_CODEC_VERSION

static final int COMPLETION_CODEC_VERSION

See Also:

Constant Field Values

COMPLETION_VERSION_CURRENT

static final int COMPLETION_VERSION_CURRENT

See Also:

Constant Field Values

INDEX_EXTENSION

static final java.lang.String INDEX_EXTENSION

See Also:

Constant Field Values

DICT_EXTENSION

static final java.lang.String DICT_EXTENSION

See Also:

Constant Field Values

fstLoadMode

private final CompletionPostingsFormat.FSTLoadMode fstLoadMode

Constructor Detail

CompletionPostingsFormat

public CompletionPostingsFormat(java.lang.String name)

Used only by core Lucene at read-time via Service Provider instantiation

CompletionPostingsFormat

public CompletionPostingsFormat(java.lang.String name,
                                CompletionPostingsFormat.FSTLoadMode fstLoadMode)

Creates a CompletionPostingsFormat that will use the provided fstLoadMode to determine if the completion FST should be loaded on or off heap.

Method Detail

delegatePostingsFormat

protected abstract PostingsFormat delegatePostingsFormat()

Concrete implementation should specify the delegating postings format

fieldsConsumer

public FieldsConsumer fieldsConsumer(SegmentWriteState state)
                              throws java.io.IOException

Description copied from class: PostingsFormat

Writes a new segment

Specified by:

fieldsConsumer in class PostingsFormat

Throws:

java.io.IOException

fieldsProducer

public FieldsProducer fieldsProducer(SegmentReadState state)
                              throws java.io.IOException

Description copied from class: PostingsFormat

Reads a segment. NOTE: by the time this call returns, it must hold open any files it will need to use; else, those files may be deleted. Additionally, required files may be deleted during the execution of this call before there is a chance to open them. Under these circumstances an IOException should be thrown by the implementation. IOExceptions are expected and will automatically cause a retry of the segment opening logic with the newly revised segments.

Specified by:

fieldsProducer in class PostingsFormat

Throws:

java.io.IOException