[v1.11.0] AI search changes

[brunoocasali]

Following this central issue

Explanation of the feature

Related Meilisearch engine issue:

• Tracking issue: Vector store stabilization meilisearch#4733
• Binary quantization meilisearch#4873

💥 Breaking changes:

• When using the semantic or the hybrid search, hybrid.embedder is now a mandatory parameter in GET and POST /indexes/{:indexUid}/search
• As a consequence, it is now mandatory to pass hybrid even for full-vector search (with only vector and not q)
• embedder is now a mandatory parameter in GET and POST /indexes/{:indexUid}/similar
• Ignore non-zero semanticRatio when vector is passed but not q: a semantic search will be performed.

Changes:

• A new sub setting in embedders setting to enable binary quantization and speed up indexing speed.
• The default model for OpenAI is now text-embedding-3-small instead of text-embedding-ada-002.
• Limit the maximum length of a rendered document template: when the source of an embedder is set to huggingFace, openAi, rest or ollama, then documentTemplateMaxBytes is now available as an optional parameter. This parameter describes the number of bytes in which the rendered document template text should fit when trying to embed a document. Longer texts are truncated to fit.
• Add the ability to query whether a field is searchable in documentTemplate: use field.is_searchable: true if the field is a searchable attribute, otherwise false.