public class SemiExternalOffsetList extends AbstractLongList
index.
This class is a semi-external LongList that
MG4J uses as default for accessing term offsets.
When the number of terms in the index grows, storing each offset as a long in an array can consume hundred of megabytes of memory, and most of this memory is wasted, as it is occupied by offsets of hapax legomena (terms occurring just once in the collection). Instead, this class accesses offsets in their compressed forms, and provides entry points for random access to each offset. At construction time, entry points are computed with a certain step, which is the number of offsets accessible from each entry point, or, equivalently, the maximum number of offsets that will be necessary to read to access a given offset.
This class uses a small (CACHE_MAX_SIZE entries) map to keep track of the most recently used
indices, so to answer queries to those indices more quickly.
Warning: This class is not thread safe, and needs to be synchronised to be used in a multithreaded environment.
AbstractLongList.LongSubList| Modifier and Type | Field and Description |
|---|---|
static int |
CACHE_MAX_SIZE
The maximum number of entry in the cache map.
|
| Constructor and Description |
|---|
SemiExternalOffsetList(InputBitStream offsetRawData,
int offsetStep,
int numOffsets)
Creates a new semi-external list.
|
| Modifier and Type | Method and Description |
|---|---|
long |
getLong(int index) |
int |
size() |
add, add, add, addAll, addAll, addAll, addAll, addAll, addAll, addElements, addElements, compareTo, contains, ensureIndex, ensureRestrictedIndex, equals, get, getElements, hashCode, indexOf, indexOf, iterator, lastIndexOf, lastIndexOf, listIterator, listIterator, longListIterator, longListIterator, longSubList, peek, peekLong, pop, popLong, push, push, rem, remove, remove, removeElements, removeLong, set, set, size, subList, top, topLong, toStringadd, contains, containsAll, containsAll, isEmpty, longIterator, rem, removeAll, removeAll, retainAll, retainAll, toArray, toArray, toArray, toLongArray, toLongArrayclearclone, finalize, getClass, notify, notifyAll, wait, wait, waitadd, clear, contains, containsAll, isEmpty, removeAll, retainAll, toArray, toArraycontainsAll, longIterator, removeAll, retainAll, toArray, toArray, toLongArray, toLongArraypublic static final int CACHE_MAX_SIZE
public SemiExternalOffsetList(InputBitStream offsetRawData, int offsetStep, int numOffsets) throws IOException
offsetRawData - a bit stream containing the offsets in compressed form (γ-encoded deltas).offsetStep - the step used to build random-access entry points.numOffsets - the overall number of offsets (i.e., the number of terms).IOExceptionpublic final long getLong(int index)
public int size()
size in interface Collection<Long>size in interface List<Long>size in class AbstractCollection<Long>