ChunkingConfig(mapping=None, *, ignore_unknown_fields=False, **kwargs)Serving config for chunking.
Attributes |
|
|---|---|
| Name | Description |
chunk_size |
int
Optional. The chunk sizes to use when splitting documents, in order of level. |
include_ancestor_headings |
bool
Optional. Whether or not to include ancestor headings when splitting. |
semantic_chunking_group_size |
bool
Optional. The number of tokens to group together when evaluating semantic similarity. THIS FIELD IS NOT YET USED. |
breakpoint_percentile_threshold |
int
Optional. The percentile of cosine dissimilarity that must be exceeded between a group of tokens and the next. The smaller this number is, the more chunks will be generated. THIS FIELD IS NOT YET USED. |