org.apache.lucene.store

Class ByteBuffersDataOutput

java.lang.Object

org.apache.lucene.store.DataOutput

org.apache.lucene.store.ByteBuffersDataOutput

All Implemented Interfaces:

Accountable

public final class ByteBuffersDataOutput
extends DataOutput
implements Accountable

A DataOutput storing data in a list of ByteBuffers.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	ByteBuffersDataOutput.ByteBufferRecycler An implementation of a ByteBuffer allocation and recycling policy.

Field Summary

Fields

Modifier and Type	Field and Description
static java.util.function.IntFunction<java.nio.ByteBuffer>	ALLOCATE_BB_ON_HEAP
private java.util.function.IntFunction<java.nio.ByteBuffer>	blockAllocate ByteBuffer supplier.
private int	blockBits Current block size: 2^bits.
private java.util.function.Consumer<java.nio.ByteBuffer>	blockReuse ByteBuffer recycler on reset().
private java.util.ArrayDeque<java.nio.ByteBuffer>	blocks Blocks storing data.
private java.nio.ByteBuffer	currentBlock The current-or-next write block.
static int	DEFAULT_MAX_BITS_PER_BLOCK
static int	DEFAULT_MIN_BITS_PER_BLOCK
private static java.nio.ByteBuffer	EMPTY
private static byte[]	EMPTY_BYTE_ARRAY
private static long	HALF_SHIFT
(package private) static int	MAX_BLOCKS_BEFORE_BLOCK_EXPANSION Maximum number of blocks at the current blockBits block size before we increase the block size (and thus decrease the number of blocks).
private int	maxBitsPerBlock Maximum block size: 2^bits.
static java.util.function.Consumer<java.nio.ByteBuffer>	NO_REUSE A singleton instance of "no-reuse" buffer strategy.
private static int	SURROGATE_OFFSET

Constructor Summary

Constructors

Constructor and Description
ByteBuffersDataOutput()
ByteBuffersDataOutput(int minBitsPerBlock, int maxBitsPerBlock, java.util.function.IntFunction<java.nio.ByteBuffer> blockAllocate, java.util.function.Consumer<java.nio.ByteBuffer> blockReuse)
ByteBuffersDataOutput(long expectedSize)

Method Summary

Modifier and Type	Method and Description
private void	appendBlock()
private int	blockSize()
private static int	computeBlockSizeBitsFor(long bytes)
void	copyTo(DataOutput output) Copy the current content of this object into another DataOutput.
static ByteBuffersDataOutput	newResettableInstance()
long	ramBytesUsed() Return the memory usage of this object in bytes.
void	reset() This method resets this object to a clean (zero-size) state and publishes any currently allocated buffers for reuse to the reuse strategy provided in the constructor.
private void	rewriteToBlockSize(int targetBlockBits)
long	size()
byte[]	toArrayCopy() Return a contiguous array with the current content written to the output.
java.util.ArrayList<java.nio.ByteBuffer>	toBufferList() Return a list of read-only view of ByteBuffer blocks over the current content written to the output.
ByteBuffersDataInput	toDataInput() Return a ByteBuffersDataInput for the set of current buffers (toBufferList()).
java.lang.String	toString()
java.util.ArrayList<java.nio.ByteBuffer>	toWriteableBufferList() Returns a list of writeable blocks over the (source) content buffers.
private static int	UTF16toUTF8(java.lang.CharSequence s, int offset, int length, byte[] buf, java.util.function.IntConsumer bufferFlusher) A consumer-based UTF16-UTF8 encoder (writes the input string in smaller buffers.).
void	writeByte(byte b) Writes a single byte.
void	writeBytes(byte[] b)
void	writeBytes(byte[] b, int length) Writes an array of bytes.
void	writeBytes(byte[] src, int offset, int length) Writes an array of bytes.
void	writeBytes(java.nio.ByteBuffer buffer)
void	writeInt(int v) Writes an int as four bytes.
void	writeLong(long v) Writes a long as eight bytes.
void	writeMapOfStrings(java.util.Map<java.lang.String,java.lang.String> map) Writes a String map.
void	writeSetOfStrings(java.util.Set<java.lang.String> set) Writes a String set.
void	writeShort(short v) Writes a short as two bytes.
void	writeString(java.lang.String v) Writes a string.

Methods inherited from class org.apache.lucene.store.DataOutput

copyBytes, writeVInt, writeVLong, writeZInt, writeZLong

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface org.apache.lucene.util.Accountable

getChildResources

Field Detail

EMPTY

private static final java.nio.ByteBuffer EMPTY

EMPTY_BYTE_ARRAY

private static final byte[] EMPTY_BYTE_ARRAY

ALLOCATE_BB_ON_HEAP

public static final java.util.function.IntFunction<java.nio.ByteBuffer> ALLOCATE_BB_ON_HEAP

NO_REUSE

public static final java.util.function.Consumer<java.nio.ByteBuffer> NO_REUSE

A singleton instance of "no-reuse" buffer strategy.

DEFAULT_MIN_BITS_PER_BLOCK

public static final int DEFAULT_MIN_BITS_PER_BLOCK

See Also:

Constant Field Values

DEFAULT_MAX_BITS_PER_BLOCK

public static final int DEFAULT_MAX_BITS_PER_BLOCK

See Also:

Constant Field Values

MAX_BLOCKS_BEFORE_BLOCK_EXPANSION

static final int MAX_BLOCKS_BEFORE_BLOCK_EXPANSION

Maximum number of blocks at the current blockBits block size before we increase the block size (and thus decrease the number of blocks).

See Also:

Constant Field Values

maxBitsPerBlock

private final int maxBitsPerBlock

Maximum block size: 2^bits.

blockAllocate

private final java.util.function.IntFunction<java.nio.ByteBuffer> blockAllocate

ByteBuffer supplier.

blockReuse

private final java.util.function.Consumer<java.nio.ByteBuffer> blockReuse

ByteBuffer recycler on reset().

blockBits

private int blockBits

Current block size: 2^bits.

blocks

private final java.util.ArrayDeque<java.nio.ByteBuffer> blocks

Blocks storing data.

currentBlock

private java.nio.ByteBuffer currentBlock

The current-or-next write block.

HALF_SHIFT

private static final long HALF_SHIFT

See Also:

Constant Field Values

SURROGATE_OFFSET

private static final int SURROGATE_OFFSET

See Also:

Constant Field Values

Constructor Detail

ByteBuffersDataOutput

public ByteBuffersDataOutput(long expectedSize)

ByteBuffersDataOutput

public ByteBuffersDataOutput()

ByteBuffersDataOutput

public ByteBuffersDataOutput(int minBitsPerBlock,
                             int maxBitsPerBlock,
                             java.util.function.IntFunction<java.nio.ByteBuffer> blockAllocate,
                             java.util.function.Consumer<java.nio.ByteBuffer> blockReuse)

Method Detail

writeByte

public void writeByte(byte b)

Description copied from class: DataOutput

Writes a single byte.

The most primitive data type is an eight-bit byte. Files are accessed as sequences of bytes. All other data types are defined as sequences of bytes, so file formats are byte-order independent.

Specified by:

writeByte in class DataOutput

See Also:

DataInput.readByte()

writeBytes

public void writeBytes(byte[] src,
                       int offset,
                       int length)

Description copied from class: DataOutput

Writes an array of bytes.

Specified by:

writeBytes in class DataOutput

Parameters:

src - the bytes to write

offset - the offset in the byte array

length - the number of bytes to write

See Also:

DataInput.readBytes(byte[],int,int)

writeBytes

public void writeBytes(byte[] b,
                       int length)

Description copied from class: DataOutput

Writes an array of bytes.

Overrides:

writeBytes in class DataOutput

Parameters:

b - the bytes to write

length - the number of bytes to write

See Also:

DataInput.readBytes(byte[],int,int)

writeBytes

public void writeBytes(byte[] b)

writeBytes

public void writeBytes(java.nio.ByteBuffer buffer)

toBufferList

public java.util.ArrayList<java.nio.ByteBuffer> toBufferList()

Return a list of read-only view of ByteBuffer blocks over the current content written to the output.

toWriteableBufferList

public java.util.ArrayList<java.nio.ByteBuffer> toWriteableBufferList()

Returns a list of writeable blocks over the (source) content buffers. This method returns the raw content of source buffers that may change over the lifetime of this object (blocks can be recycled or discarded, for example). Most applications should favor calling toBufferList() which returns a read-only view over the content of the source buffers. The difference between toBufferList() and toWriteableBufferList() is that read-only view of source buffers will always return false from ByteBuffer.hasArray() (which sometimes may be required to avoid double copying).

toDataInput

public ByteBuffersDataInput toDataInput()

Return a ByteBuffersDataInput for the set of current buffers (toBufferList()).

toArrayCopy

public byte[] toArrayCopy()

Return a contiguous array with the current content written to the output. The returned array is always a copy (can be mutated). If the size() of the underlying buffers exceeds maximum size of Java array, an RuntimeException will be thrown.

copyTo

public void copyTo(DataOutput output)
            throws java.io.IOException

Copy the current content of this object into another DataOutput.

Throws:

java.io.IOException

size

public long size()

Returns:

The number of bytes written to this output so far.

toString

public java.lang.String toString()

Overrides:

toString in class java.lang.Object

writeShort

public void writeShort(short v)

Description copied from class: DataOutput

Writes a short as two bytes.

Overrides:

writeShort in class DataOutput

See Also:

DataInput.readShort()

writeInt

public void writeInt(int v)

Description copied from class: DataOutput

Writes an int as four bytes.

32-bit unsigned integer written as four bytes, high-order bytes first.

Overrides:

writeInt in class DataOutput

See Also:

DataInput.readInt()

writeLong

public void writeLong(long v)

Description copied from class: DataOutput

Writes a long as eight bytes.

64-bit unsigned integer written as eight bytes, high-order bytes first.

Overrides:

writeLong in class DataOutput

See Also:

DataInput.readLong()

writeString

public void writeString(java.lang.String v)

Description copied from class: DataOutput

Writes a string.

Writes strings as UTF-8 encoded bytes. First the length, in bytes, is written as a VInt, followed by the bytes.

Overrides:

writeString in class DataOutput

See Also:

DataInput.readString()

writeMapOfStrings

public void writeMapOfStrings(java.util.Map<java.lang.String,java.lang.String> map)

Description copied from class: DataOutput

Writes a String map.

First the size is written as an vInt, followed by each key-value pair written as two consecutive Strings.

Overrides:

writeMapOfStrings in class DataOutput

Parameters:

map - Input map.

writeSetOfStrings

public void writeSetOfStrings(java.util.Set<java.lang.String> set)

Description copied from class: DataOutput

Writes a String set.

First the size is written as an vInt, followed by each value written as a String.

Overrides:

writeSetOfStrings in class DataOutput

Parameters:

set - Input set.

ramBytesUsed

public long ramBytesUsed()

Description copied from interface: Accountable

Return the memory usage of this object in bytes. Negative values are illegal.

Specified by:

ramBytesUsed in interface Accountable

reset

public void reset()

This method resets this object to a clean (zero-size) state and publishes any currently allocated buffers for reuse to the reuse strategy provided in the constructor. Sharing byte buffers for reads and writes is dangerous and will very likely lead to hard-to-debug issues, use with great care.

newResettableInstance

public static ByteBuffersDataOutput newResettableInstance()

Returns:

Returns a new ByteBuffersDataOutput with the reset() capability.

blockSize

private int blockSize()

appendBlock

private void appendBlock()

rewriteToBlockSize

private void rewriteToBlockSize(int targetBlockBits)

computeBlockSizeBitsFor

private static int computeBlockSizeBitsFor(long bytes)

UTF16toUTF8

private static int UTF16toUTF8(java.lang.CharSequence s,
                               int offset,
                               int length,
                               byte[] buf,
                               java.util.function.IntConsumer bufferFlusher)

A consumer-based UTF16-UTF8 encoder (writes the input string in smaller buffers.).