Module org.apache.lucene.core
Class Lucene99ScalarQuantizedVectorsFormat
java.lang.Object
org.apache.lucene.codecs.hnsw.FlatVectorsFormat
org.apache.lucene.codecs.lucene99.Lucene99ScalarQuantizedVectorsFormat
Format supporting vector quantization, storage, and retrieval
-
Field Summary
FieldsModifier and TypeFieldDescriptionprivate static final int
(package private) final byte
(package private) final boolean
(package private) final Float
Controls the confidence interval used to scalar quantize the vectors the default value is calculated as `1-1/(vector_dimensions + 1)`static final float
Dynamic confidence interval(package private) final Lucene99ScalarQuantizedVectorScorer
private static final float
The maximum confidence interval(package private) static final String
(package private) static final String
private static final float
The minimum confidence intervalstatic final String
static final String
private static final FlatVectorsFormat
(package private) static final String
(package private) static final String
(package private) static final int
(package private) static final int
(package private) static final int
-
Constructor Summary
ConstructorsConstructorDescriptionConstructs a format using default graph construction parametersLucene99ScalarQuantizedVectorsFormat
(Float confidenceInterval, int bits, boolean compress) Constructs a format using the given graph construction parameters. -
Method Summary
Modifier and TypeMethodDescriptionstatic float
calculateDefaultConfidenceInterval
(int vectorDimension) fieldsReader
(SegmentReadState state) Returns aKnnVectorsReader
to read the vectors from the index.fieldsWriter
(SegmentWriteState state) Returns aFlatVectorsWriter
to write the vectors to the index.toString()
-
Field Details
-
ALLOWED_BITS
private static final int ALLOWED_BITS- See Also:
-
QUANTIZED_VECTOR_COMPONENT
- See Also:
-
NAME
- See Also:
-
VERSION_START
static final int VERSION_START- See Also:
-
VERSION_ADD_BITS
static final int VERSION_ADD_BITS- See Also:
-
VERSION_CURRENT
static final int VERSION_CURRENT- See Also:
-
META_CODEC_NAME
- See Also:
-
VECTOR_DATA_CODEC_NAME
- See Also:
-
META_EXTENSION
- See Also:
-
VECTOR_DATA_EXTENSION
- See Also:
-
rawVectorFormat
-
MINIMUM_CONFIDENCE_INTERVAL
private static final float MINIMUM_CONFIDENCE_INTERVALThe minimum confidence interval- See Also:
-
MAXIMUM_CONFIDENCE_INTERVAL
private static final float MAXIMUM_CONFIDENCE_INTERVALThe maximum confidence interval- See Also:
-
DYNAMIC_CONFIDENCE_INTERVAL
public static final float DYNAMIC_CONFIDENCE_INTERVALDynamic confidence interval- See Also:
-
confidenceInterval
Controls the confidence interval used to scalar quantize the vectors the default value is calculated as `1-1/(vector_dimensions + 1)` -
bits
final byte bits -
compress
final boolean compress -
flatVectorScorer
-
-
Constructor Details
-
Lucene99ScalarQuantizedVectorsFormat
public Lucene99ScalarQuantizedVectorsFormat()Constructs a format using default graph construction parameters -
Lucene99ScalarQuantizedVectorsFormat
Constructs a format using the given graph construction parameters.- Parameters:
confidenceInterval
- the confidenceInterval for scalar quantizing the vectors, when `null` it is calculated based on the vector dimension. When `0`, the quantiles are dynamically determined by sampling many confidence intervals and determining the most accurate pair.bits
- the number of bits to use for scalar quantization (must be between 1 and 8, inclusive)compress
- whether to compress the vectors, if true, the vectors that are quantized with lte 4 bits will be compressed into a single byte. If false, the vectors will be stored as is. This provides a trade-off of memory usage and speed.
-
-
Method Details
-
calculateDefaultConfidenceInterval
public static float calculateDefaultConfidenceInterval(int vectorDimension) -
toString
-
fieldsWriter
Description copied from class:FlatVectorsFormat
Returns aFlatVectorsWriter
to write the vectors to the index.- Specified by:
fieldsWriter
in classFlatVectorsFormat
- Throws:
IOException
-
fieldsReader
Description copied from class:FlatVectorsFormat
Returns aKnnVectorsReader
to read the vectors from the index.- Specified by:
fieldsReader
in classFlatVectorsFormat
- Throws:
IOException
-