Troubleshoot ScaNN index errors

This document describes errors you might encounter when you generate a ScaNN index. Examples of errors and recommended fixes are also provided.

As a general workaround, you can disable generation of these errors and continue to generate the index. For more information, see Force index creation on empty or small tables.

Cannot create ScaNN index with empty table

Description

When you try to generate an index on a table with no data, or if you try to truncate a table with a ScaNN index generated on it, the following error occurs:

ERROR: Cannot create ScaNN index "[INDEX_NAME]" for table "[TABLE_NAME]", error: FAILED_PRECONDITION: Cannot create ScaNN index "[INDEX_NAME]" with empty table "[TABLE_NAME]". Once the table is populated with data, create the index. See documentation to bypass this validation.

Sample queries that cause the error

  • Query Example A

    create table t1 (a INT, b VECTOR(512));
CREATE TABLE
create index on t1 using ScaNN(b cosine) with (num_leaves = 10, quantizer = 'sq8');

  • Query Example B

    truncate t1;

Make sure that your table is populated with embedding vectors before you generate a ScaNN index.

Cannot create ScaNN index

Description

When you try to generate an index on a table with few rows populated, the following error occurs:

ERROR: Cannot create ScaNN index "[INDEX_NAME]" for table "[TABLE_NAME]", error: INVALID_ARGUMENT: Number of rows ([ACTUAL]) must be larger than ([EXPECTED]).

Sample query that causes the error

create table t1 (a INT, b VECTOR(512));
CREATE TABLE
insert into t1 select (random()*1e9)::int, random_vector(512) from generate_series(1, 5);
INSERT 0 5
create index on t1 using scann(b cosine) with (num_leaves = 100, quantizer = 'sq8');

Make sure that your table is populated with embedding vectors before you generate a ScaNN index. We recommend that the number of rows in the table are greater than the value defined in the num_leaves parameter.

Cannot create ScaNN index for a table with only one row

Description

This error happened because ScaNN (and most ANN indexes like IVFFlat) needs a training set to build its internal structures (clusters/leaves). When you insert exactly one row , you get the following error:

ERROR: Cannot create ScaNN index "[INDEX_NAME]" for table "[TABLE_NAME]", error: INVALID_ARGUMENT: Number of samples "[NUM_SAMPLES]" must be larger than 1 and the number of leaves "[NUM_LEAVES]"

Sample query that causes the error

create table t1 (a INT, b VECTOR(512));
CREATE TABLE
insert into t1 select (random()*1e9)::int, random_vector(512) from generate_series(1, 1);
INSERT 0 1
create index on t1 using scann(b cosine) with (num_leaves = 1, quantizer = 'sq8');

Make sure that your table is populated with embedding vectors before you generate a ScaNN index. We recommend that the number of rows in the table is:

  • Greater than 1
  • Greater than the value defined in the num_leaves parameter

Cannot create ScaNN index on parent partition table

Description

If you created partitioned tables from a parent table, then creating a ScaNN index on the parent table generates the following error:

ERROR: Cannot create ScaNN index "[INDEX_NAME]" for parent partition table "[TABLE_NAME]". Create ScaNN indexes on the child tables instead. See documentation to bypass this validation.

Sample query that causes the error

create table t1 (a INT, b VECTOR(512)) partition by range(a);
CREATE TABLE
CREATE TABLE t1_one_ten PARTITION of t1 for values from (1) to (10);
CREATE TABLE
insert into t1_one_ten select (random()*1e9)::int, random_vector(512) from generate_series(1, 100);
INSERT 0 100
CREATE TABLE t1_eleven_twenty PARTITION of t1 for values from (11) to (20);
CREATE TABLE
insert into t1_eleven_twenty select (random()*1e9)::int, random_vector(512) from generate_series(1, 100);
INSERT 0 100
create index on t1 using scann(b cosine) with (num_leaves = 10, quantizer = 'sq8');
  • You can't generate a ScaNN index on the parent table of a partitioned table.
  • You must generate the ScaNN indexes on the partitioned table.