org.apache.lucene.util.bkd

Class BKDReader

java.lang.Object

org.apache.lucene.index.PointValues

org.apache.lucene.util.bkd.BKDReader

All Implemented Interfaces:

Accountable

public final class BKDReader
extends PointValues
implements Accountable

Handles intersection of an multi-dimensional shape in byte[] space with a block KD-tree previously written with BKDWriter.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
private static class	BKDReader.BKDInput
private static class	BKDReader.BKDOffHeapInput
private static class	BKDReader.BKDOnHeapInput
protected static class	BKDReader.BKDReaderDocIDSetIterator Reusable DocIdSetIterator to handle low cardinality leaves.
class	BKDReader.IndexTree Used to walk the in-heap index.
static class	BKDReader.IntersectState Used to track all state for a single call to intersect(org.apache.lucene.index.PointValues.IntersectVisitor).

Nested classes/interfaces inherited from class org.apache.lucene.index.PointValues

PointValues.IntersectVisitor, PointValues.Relation

Field Summary

Fields

Modifier and Type	Field and Description
(package private) int	bytesPerDim
(package private) int	docCount
(package private) IndexInput	in
(package private) int	leafNodeOffset
(package private) byte[]	maxPackedValue
(package private) int	maxPointsInLeafNode
(package private) byte[]	minPackedValue
(package private) int	numDataDims
(package private) int	numIndexDims
(package private) int	numLeaves
protected int	packedBytesLength
(package private) BKDReader.BKDInput	packedIndex
protected int	packedIndexBytesLength
(package private) long	pointCount
(package private) int	version

Fields inherited from class org.apache.lucene.index.PointValues

MAX_DIMENSIONS, MAX_NUM_BYTES

Constructor Summary

Constructors

Constructor and Description
BKDReader(IndexInput in) Caller must pre-seek the provided IndexInput to the index location that BKDWriter.finish(org.apache.lucene.store.IndexOutput) returned
BKDReader(IndexInput in, boolean offHeap) Caller must pre-seek the provided IndexInput to the index location that BKDWriter.finish(org.apache.lucene.store.IndexOutput) returned and specify true to store BKD off-heap (false otherwise)

Method Summary

Modifier and Type	Method and Description
private void	addAll(BKDReader.IntersectState state, boolean grown) Fast path: this is called when the query box fully encompasses all cells under this node.
private long	estimatePointCount(BKDReader.IntersectState state, byte[] cellMinPacked, byte[] cellMaxPacked)
long	estimatePointCount(PointValues.IntersectVisitor visitor) Estimate the number of points that would be visited by PointValues.intersect(org.apache.lucene.index.PointValues.IntersectVisitor) with the given PointValues.IntersectVisitor.
int	getBytesPerDimension() Returns the number of bytes per dimension
int	getDocCount() Returns the total number of documents that have indexed at least one point.
BKDReader.IntersectState	getIntersectState(PointValues.IntersectVisitor visitor) Create a new BKDReader.IntersectState
byte[]	getMaxPackedValue() Returns maximum value for each dimension, packed, or null if PointValues.size(org.apache.lucene.index.IndexReader, java.lang.String) is 0
(package private) long	getMinLeafBlockFP()
byte[]	getMinPackedValue() Returns minimum value for each dimension, packed, or null if PointValues.size(org.apache.lucene.index.IndexReader, java.lang.String) is 0
int	getNumDataDimensions() Returns how many data dimensions are represented in the values
int	getNumIndexDimensions() Returns how many dimensions are used for the index
private int	getTreeDepth()
private void	intersect(BKDReader.IntersectState state, byte[] cellMinPacked, byte[] cellMaxPacked)
void	intersect(PointValues.IntersectVisitor visitor) Finds all documents and points matching the provided visitor.
boolean	isLeafNode(int nodeID)
long	ramBytesUsed() Return the memory usage of this object in bytes.
private void	readCommonPrefixes(int[] commonPrefixLengths, byte[] scratchPackedValue, IndexInput in)
private int	readCompressedDim(IndexInput in)
(package private) int	readDocIDs(IndexInput in, long blockFP, BKDReader.BKDReaderDocIDSetIterator iterator)
private void	readMinMax(int[] commonPrefixLengths, byte[] minPackedValue, byte[] maxPackedValue, IndexInput in)
long	size() Returns the total number of indexed points across all documents.
private void	visitCompressedDocValues(int[] commonPrefixLengths, byte[] scratchPackedValue, IndexInput in, BKDReader.BKDReaderDocIDSetIterator scratchIterator, int count, PointValues.IntersectVisitor visitor, int compressedDim)
private void	visitDocIDs(IndexInput in, long blockFP, PointValues.IntersectVisitor visitor)
(package private) void	visitDocValues(int[] commonPrefixLengths, byte[] scratchDataPackedValue, byte[] scratchMinIndexPackedValue, byte[] scratchMaxIndexPackedValue, IndexInput in, BKDReader.BKDReaderDocIDSetIterator scratchIterator, int count, PointValues.IntersectVisitor visitor)
(package private) void	visitDocValuesNoCardinality(int[] commonPrefixLengths, byte[] scratchDataPackedValue, byte[] scratchMinIndexPackedValue, byte[] scratchMaxIndexPackedValue, IndexInput in, BKDReader.BKDReaderDocIDSetIterator scratchIterator, int count, PointValues.IntersectVisitor visitor)
(package private) void	visitDocValuesWithCardinality(int[] commonPrefixLengths, byte[] scratchDataPackedValue, byte[] scratchMinIndexPackedValue, byte[] scratchMaxIndexPackedValue, IndexInput in, BKDReader.BKDReaderDocIDSetIterator scratchIterator, int count, PointValues.IntersectVisitor visitor)
void	visitLeafBlockValues(BKDReader.IndexTree index, BKDReader.IntersectState state) Visits all docIDs and packed values in a single leaf block
private void	visitSparseRawDocValues(int[] commonPrefixLengths, byte[] scratchPackedValue, IndexInput in, BKDReader.BKDReaderDocIDSetIterator scratchIterator, int count, PointValues.IntersectVisitor visitor)
private void	visitUniqueRawDocValues(byte[] scratchPackedValue, BKDReader.BKDReaderDocIDSetIterator scratchIterator, int count, PointValues.IntersectVisitor visitor)

Methods inherited from class org.apache.lucene.index.PointValues

estimateDocCount, getDocCount, getMaxPackedValue, getMinPackedValue, size

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.lucene.util.Accountable

getChildResources

Field Detail

leafNodeOffset

final int leafNodeOffset

numDataDims

final int numDataDims

numIndexDims

final int numIndexDims

bytesPerDim

final int bytesPerDim

numLeaves

final int numLeaves

in

final IndexInput in

maxPointsInLeafNode

final int maxPointsInLeafNode

minPackedValue

final byte[] minPackedValue

maxPackedValue

final byte[] maxPackedValue

pointCount

final long pointCount

docCount

final int docCount

version

final int version

packedBytesLength

protected final int packedBytesLength

packedIndexBytesLength

protected final int packedIndexBytesLength

packedIndex

final BKDReader.BKDInput packedIndex

Constructor Detail

BKDReader

public BKDReader(IndexInput in)
          throws java.io.IOException

Caller must pre-seek the provided IndexInput to the index location that BKDWriter.finish(org.apache.lucene.store.IndexOutput) returned

Throws:

java.io.IOException

BKDReader

public BKDReader(IndexInput in,
                 boolean offHeap)
          throws java.io.IOException

Caller must pre-seek the provided IndexInput to the index location that BKDWriter.finish(org.apache.lucene.store.IndexOutput) returned and specify true to store BKD off-heap (false otherwise)

Throws:

java.io.IOException

Method Detail

getMinLeafBlockFP

long getMinLeafBlockFP()

getTreeDepth

private int getTreeDepth()

intersect

public void intersect(PointValues.IntersectVisitor visitor)
               throws java.io.IOException

Description copied from class: PointValues

Finds all documents and points matching the provided visitor. This method does not enforce live documents, so it's up to the caller to test whether each document is deleted, if necessary.

Specified by:

intersect in class PointValues

Throws:

java.io.IOException

estimatePointCount

public long estimatePointCount(PointValues.IntersectVisitor visitor)

Description copied from class: PointValues

Estimate the number of points that would be visited by PointValues.intersect(org.apache.lucene.index.PointValues.IntersectVisitor) with the given PointValues.IntersectVisitor. This should run many times faster than PointValues.intersect(IntersectVisitor).

Specified by:

estimatePointCount in class PointValues

addAll

private void addAll(BKDReader.IntersectState state,
                    boolean grown)
             throws java.io.IOException

Fast path: this is called when the query box fully encompasses all cells under this node.

Throws:

java.io.IOException

getIntersectState

public BKDReader.IntersectState getIntersectState(PointValues.IntersectVisitor visitor)

Create a new BKDReader.IntersectState

visitLeafBlockValues

public void visitLeafBlockValues(BKDReader.IndexTree index,
                                 BKDReader.IntersectState state)
                          throws java.io.IOException

Visits all docIDs and packed values in a single leaf block

Throws:

java.io.IOException

visitDocIDs

private void visitDocIDs(IndexInput in,
                         long blockFP,
                         PointValues.IntersectVisitor visitor)
                  throws java.io.IOException

Throws:

java.io.IOException

readDocIDs

int readDocIDs(IndexInput in,
               long blockFP,
               BKDReader.BKDReaderDocIDSetIterator iterator)
        throws java.io.IOException

Throws:

java.io.IOException

visitDocValues

void visitDocValues(int[] commonPrefixLengths,
                    byte[] scratchDataPackedValue,
                    byte[] scratchMinIndexPackedValue,
                    byte[] scratchMaxIndexPackedValue,
                    IndexInput in,
                    BKDReader.BKDReaderDocIDSetIterator scratchIterator,
                    int count,
                    PointValues.IntersectVisitor visitor)
             throws java.io.IOException

Throws:

java.io.IOException

visitDocValuesNoCardinality

void visitDocValuesNoCardinality(int[] commonPrefixLengths,
                                 byte[] scratchDataPackedValue,
                                 byte[] scratchMinIndexPackedValue,
                                 byte[] scratchMaxIndexPackedValue,
                                 IndexInput in,
                                 BKDReader.BKDReaderDocIDSetIterator scratchIterator,
                                 int count,
                                 PointValues.IntersectVisitor visitor)
                          throws java.io.IOException

Throws:

java.io.IOException

visitDocValuesWithCardinality

void visitDocValuesWithCardinality(int[] commonPrefixLengths,
                                   byte[] scratchDataPackedValue,
                                   byte[] scratchMinIndexPackedValue,
                                   byte[] scratchMaxIndexPackedValue,
                                   IndexInput in,
                                   BKDReader.BKDReaderDocIDSetIterator scratchIterator,
                                   int count,
                                   PointValues.IntersectVisitor visitor)
                            throws java.io.IOException

Throws:

java.io.IOException

readMinMax

private void readMinMax(int[] commonPrefixLengths,
                        byte[] minPackedValue,
                        byte[] maxPackedValue,
                        IndexInput in)
                 throws java.io.IOException

Throws:

java.io.IOException

visitSparseRawDocValues

private void visitSparseRawDocValues(int[] commonPrefixLengths,
                                     byte[] scratchPackedValue,
                                     IndexInput in,
                                     BKDReader.BKDReaderDocIDSetIterator scratchIterator,
                                     int count,
                                     PointValues.IntersectVisitor visitor)
                              throws java.io.IOException

Throws:

java.io.IOException

visitUniqueRawDocValues

private void visitUniqueRawDocValues(byte[] scratchPackedValue,
                                     BKDReader.BKDReaderDocIDSetIterator scratchIterator,
                                     int count,
                                     PointValues.IntersectVisitor visitor)
                              throws java.io.IOException

Throws:

java.io.IOException

visitCompressedDocValues

private void visitCompressedDocValues(int[] commonPrefixLengths,
                                      byte[] scratchPackedValue,
                                      IndexInput in,
                                      BKDReader.BKDReaderDocIDSetIterator scratchIterator,
                                      int count,
                                      PointValues.IntersectVisitor visitor,
                                      int compressedDim)
                               throws java.io.IOException

Throws:

java.io.IOException

readCompressedDim

private int readCompressedDim(IndexInput in)
                       throws java.io.IOException

Throws:

java.io.IOException

readCommonPrefixes

private void readCommonPrefixes(int[] commonPrefixLengths,
                                byte[] scratchPackedValue,
                                IndexInput in)
                         throws java.io.IOException

Throws:

java.io.IOException

intersect

private void intersect(BKDReader.IntersectState state,
                       byte[] cellMinPacked,
                       byte[] cellMaxPacked)
                throws java.io.IOException

Throws:

java.io.IOException

estimatePointCount

private long estimatePointCount(BKDReader.IntersectState state,
                                byte[] cellMinPacked,
                                byte[] cellMaxPacked)

ramBytesUsed

public long ramBytesUsed()

Description copied from interface: Accountable

Return the memory usage of this object in bytes. Negative values are illegal.

Specified by:

ramBytesUsed in interface Accountable

getMinPackedValue

public byte[] getMinPackedValue()

Description copied from class: PointValues

Returns minimum value for each dimension, packed, or null if PointValues.size(org.apache.lucene.index.IndexReader, java.lang.String) is 0

Specified by:

getMinPackedValue in class PointValues

getMaxPackedValue

public byte[] getMaxPackedValue()

Description copied from class: PointValues

Returns maximum value for each dimension, packed, or null if PointValues.size(org.apache.lucene.index.IndexReader, java.lang.String) is 0

Specified by:

getMaxPackedValue in class PointValues

getNumDataDimensions

public int getNumDataDimensions()

Description copied from class: PointValues

Returns how many data dimensions are represented in the values

Specified by:

getNumDataDimensions in class PointValues

getNumIndexDimensions

public int getNumIndexDimensions()

Description copied from class: PointValues

Returns how many dimensions are used for the index

Specified by:

getNumIndexDimensions in class PointValues

getBytesPerDimension

public int getBytesPerDimension()

Description copied from class: PointValues

Returns the number of bytes per dimension

Specified by:

getBytesPerDimension in class PointValues

size

public long size()

Description copied from class: PointValues

Returns the total number of indexed points across all documents.

Specified by:

size in class PointValues

getDocCount

public int getDocCount()

Description copied from class: PointValues

Returns the total number of documents that have indexed at least one point.

Specified by:

getDocCount in class PointValues

isLeafNode

public boolean isLeafNode(int nodeID)