org.apache.lucene.search

Class FuzzyTermsEnum

java.lang.Object

org.apache.lucene.index.TermsEnum

org.apache.lucene.index.BaseTermsEnum

org.apache.lucene.search.FuzzyTermsEnum

All Implemented Interfaces:

BytesRefIterator

public final class FuzzyTermsEnum
extends BaseTermsEnum

Subclass of TermsEnum for enumerating all terms that are similar to the specified filter term.

Term enumerations are always ordered by BytesRef.compareTo(org.apache.lucene.util.BytesRef). Each term in the enumeration is greater than all that precede it.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static interface	FuzzyTermsEnum.LevenshteinAutomataAttribute reuses compiled automata across different segments, because they are independent of the index
static class	FuzzyTermsEnum.LevenshteinAutomataAttributeImpl Stores compiled automata as a list (indexed by edit distance)

Nested classes/interfaces inherited from class org.apache.lucene.index.TermsEnum

TermsEnum.SeekStatus

Field Summary

Fields

Modifier and Type	Field and Description
private TermsEnum	actualEnum
private CompiledAutomaton[]	automata
private BoostAttribute	boostAtt
private float	bottom
private BytesRef	bottomTerm
private FuzzyTermsEnum.LevenshteinAutomataAttribute	dfaAtt
private MaxNonCompetitiveBoostAttribute	maxBoostAtt
private int	maxEdits
private BytesRef	queuedBottom
(package private) int	realPrefixLength
(package private) Term	term
(package private) int	termLength
(package private) Terms	terms
(package private) int[]	termText
(package private) boolean	transpositions

Fields inherited from class org.apache.lucene.index.TermsEnum

EMPTY

Constructor Summary

Constructors

Constructor and Description
FuzzyTermsEnum(Terms terms, AttributeSource atts, Term term, int maxEdits, int prefixLength, boolean transpositions) Constructor for enumeration of all terms from specified reader which share a prefix of length prefixLength with term and which have at most maxEdits edits.

Method Summary

Modifier and Type	Method and Description
private void	bottomChanged(BytesRef lastTerm) fired when the max non-competitive boost has changed.
private static Automaton[]	buildAutomata(int[] termText, int prefixLength, boolean transpositions, int maxEdits)
static Automaton	buildAutomaton(java.lang.String text, int prefixLength, boolean transpositions, int maxEdits) Builds a binary Automaton to match a fuzzy term
int	docFreq() Returns the number of documents containing the current term.
private TermsEnum	getAutomatonEnum(int editDistance, BytesRef lastTerm) return an automata-based enum for matching up to editDistance from lastTerm, if possible
ImpactsEnum	impacts(int flags) Return a ImpactsEnum.
private boolean	matches(BytesRef termIn, int k) returns true if term is within k edits of the query term
BytesRef	next() Increments the iteration to the next BytesRef in the iterator.
long	ord() Returns ordinal position for current term.
PostingsEnum	postings(PostingsEnum reuse, int flags) Get PostingsEnum for the current term, with control over whether freqs, positions, offsets or payloads are required.
TermsEnum.SeekStatus	seekCeil(BytesRef text) Seeks to the specified term, if it exists, or to the next (ceiling) term.
boolean	seekExact(BytesRef text) Attempts to seek to the exact term, returning true if the term is found.
void	seekExact(BytesRef term, TermState state) Expert: Seeks a specific position by TermState previously obtained from TermsEnum.termState().
void	seekExact(long ord) Seeks to the specified term by ordinal (position) as previously returned by TermsEnum.ord().
private static int[]	stringToUTF32(java.lang.String text)
BytesRef	term() Returns current term.
TermState	termState() Expert: Returns the TermsEnums internal state to position the TermsEnum without re-seeking the term dictionary.
long	totalTermFreq() Returns the total number of occurrences of this term across all documents (the sum of the freq() for each doc that has this term).

Methods inherited from class org.apache.lucene.index.BaseTermsEnum

attributes

Methods inherited from class org.apache.lucene.index.TermsEnum

postings

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

actualEnum

private TermsEnum actualEnum

boostAtt

private final BoostAttribute boostAtt

maxBoostAtt

private final MaxNonCompetitiveBoostAttribute maxBoostAtt

dfaAtt

private final FuzzyTermsEnum.LevenshteinAutomataAttribute dfaAtt

bottom

private float bottom

bottomTerm

private BytesRef bottomTerm

automata

private final CompiledAutomaton[] automata

queuedBottom

private BytesRef queuedBottom

termLength

final int termLength

maxEdits

private int maxEdits

terms

final Terms terms

term

final Term term

termText

final int[] termText

realPrefixLength

final int realPrefixLength

transpositions

final boolean transpositions

Constructor Detail

FuzzyTermsEnum

public FuzzyTermsEnum(Terms terms,
                      AttributeSource atts,
                      Term term,
                      int maxEdits,
                      int prefixLength,
                      boolean transpositions)
               throws java.io.IOException

Constructor for enumeration of all terms from specified reader which share a prefix of length prefixLength with term and which have at most maxEdits edits.

After calling the constructor the enumeration is already pointing to the first valid term if such a term exists.

Parameters:

terms - Delivers terms.

atts - AttributeSource created by the rewrite method of MultiTermQuery thats contains information about competitive boosts during rewrite. It is also used to cache DFAs between segment transitions.

term - Pattern term.

maxEdits - Maximum edit distance.

prefixLength - Length of required common prefix. Default value is 0.

Throws:

java.io.IOException - if there is a low-level IO error

Method Detail

buildAutomaton

public static Automaton buildAutomaton(java.lang.String text,
                                       int prefixLength,
                                       boolean transpositions,
                                       int maxEdits)

Builds a binary Automaton to match a fuzzy term

Parameters:

text - the term to match

prefixLength - length of a required common prefix

transpositions - true if transpositions should count as a single edit

maxEdits - the maximum edit distance of matching terms

stringToUTF32

private static int[] stringToUTF32(java.lang.String text)

buildAutomata

private static Automaton[] buildAutomata(int[] termText,
                                         int prefixLength,
                                         boolean transpositions,
                                         int maxEdits)

getAutomatonEnum

private TermsEnum getAutomatonEnum(int editDistance,
                                   BytesRef lastTerm)
                            throws java.io.IOException

return an automata-based enum for matching up to editDistance from lastTerm, if possible

Throws:

java.io.IOException

bottomChanged

private void bottomChanged(BytesRef lastTerm)
                    throws java.io.IOException

fired when the max non-competitive boost has changed. this is the hook to swap in a smarter actualEnum.

Throws:

java.io.IOException

next

public BytesRef next()
              throws java.io.IOException

Description copied from interface: BytesRefIterator

Increments the iteration to the next BytesRef in the iterator. Returns the resulting BytesRef or null if the end of the iterator is reached. The returned BytesRef may be re-used across calls to next. After this method returns null, do not call it again: the results are undefined.

Returns:

the next BytesRef in the iterator or null if the end of the iterator is reached.

Throws:

java.io.IOException - If there is a low-level I/O error.

matches

private boolean matches(BytesRef termIn,
                        int k)

returns true if term is within k edits of the query term

docFreq

public int docFreq()
            throws java.io.IOException

Description copied from class: TermsEnum

Returns the number of documents containing the current term. Do not call this when the enum is unpositioned. TermsEnum.SeekStatus.END.

Specified by:

docFreq in class TermsEnum

Throws:

java.io.IOException

totalTermFreq

public long totalTermFreq()
                   throws java.io.IOException

Description copied from class: TermsEnum

Returns the total number of occurrences of this term across all documents (the sum of the freq() for each doc that has this term). Note that, like other term measures, this measure does not take deleted documents into account.

Specified by:

totalTermFreq in class TermsEnum

Throws:

java.io.IOException

postings

public PostingsEnum postings(PostingsEnum reuse,
                             int flags)
                      throws java.io.IOException

Description copied from class: TermsEnum

Get PostingsEnum for the current term, with control over whether freqs, positions, offsets or payloads are required. Do not call this when the enum is unpositioned. This method will not return null.

NOTE: the returned iterator may return deleted documents, so deleted documents have to be checked on top of the PostingsEnum.

Specified by:

postings in class TermsEnum

Parameters:

reuse - pass a prior PostingsEnum for possible reuse

flags - specifies which optional per-document values you require; see PostingsEnum.FREQS

Throws:

java.io.IOException

impacts

public ImpactsEnum impacts(int flags)
                    throws java.io.IOException

Description copied from class: TermsEnum

Return a ImpactsEnum.

Specified by:

impacts in class TermsEnum

Throws:

java.io.IOException

See Also:

TermsEnum.postings(PostingsEnum, int)

seekExact

public void seekExact(BytesRef term,
                      TermState state)
               throws java.io.IOException

Description copied from class: TermsEnum

Expert: Seeks a specific position by TermState previously obtained from TermsEnum.termState(). Callers should maintain the TermState to use this method. Low-level implementations may position the TermsEnum without re-seeking the term dictionary.

Seeking by TermState should only be used iff the state was obtained from the same TermsEnum instance.

NOTE: Using this method with an incompatible TermState might leave this TermsEnum in undefined state. On a segment level TermState instances are compatible only iff the source and the target TermsEnum operate on the same field. If operating on segment level, TermState instances must not be used across segments.

NOTE: A seek by TermState might not restore the AttributeSource's state. AttributeSource states must be maintained separately if this method is used.

Overrides:

seekExact in class BaseTermsEnum

Parameters:

term - the term the TermState corresponds to

state - the TermState

Throws:

java.io.IOException

termState

public TermState termState()
                    throws java.io.IOException

Description copied from class: TermsEnum

Expert: Returns the TermsEnums internal state to position the TermsEnum without re-seeking the term dictionary.

NOTE: A seek by TermState might not capture the AttributeSource's state. Callers must maintain the AttributeSource states separately

Overrides:

termState in class BaseTermsEnum

Throws:

java.io.IOException

See Also:

TermState, TermsEnum.seekExact(BytesRef, TermState)

ord

public long ord()
         throws java.io.IOException

Description copied from class: TermsEnum

Returns ordinal position for current term. This is an optional method (the codec may throw UnsupportedOperationException). Do not call this when the enum is unpositioned.

Specified by:

ord in class TermsEnum

Throws:

java.io.IOException

seekExact

public boolean seekExact(BytesRef text)
                  throws java.io.IOException

Description copied from class: TermsEnum

Attempts to seek to the exact term, returning true if the term is found. If this returns false, the enum is unpositioned. For some codecs, seekExact may be substantially faster than TermsEnum.seekCeil(org.apache.lucene.util.BytesRef).

Overrides:

seekExact in class BaseTermsEnum

Returns:

true if the term is found; return false if the enum is unpositioned.

Throws:

java.io.IOException

seekCeil

public TermsEnum.SeekStatus seekCeil(BytesRef text)
                              throws java.io.IOException

Description copied from class: TermsEnum

Seeks to the specified term, if it exists, or to the next (ceiling) term. Returns SeekStatus to indicate whether exact term was found, a different term was found, or EOF was hit. The target term may be before or after the current term. If this returns SeekStatus.END, the enum is unpositioned.

Specified by:

seekCeil in class TermsEnum

Throws:

java.io.IOException

seekExact

public void seekExact(long ord)
               throws java.io.IOException

Description copied from class: TermsEnum

Seeks to the specified term by ordinal (position) as previously returned by TermsEnum.ord(). The target ord may be before or after the current ord, and must be within bounds.

Specified by:

seekExact in class TermsEnum

Throws:

java.io.IOException

term

public BytesRef term()
              throws java.io.IOException

Description copied from class: TermsEnum

Returns current term. Do not call this when the enum is unpositioned.

Specified by:

term in class TermsEnum

Throws:

java.io.IOException