> ## Documentation Index
> Fetch the complete documentation index at: https://ultrarag.openbmb.cn/llms.txt
> Use this file to discover all available pages before exploring further.

# Retriever

## `retriever_init`

**Signature**

```python theme={null}
async def retriever_init(
    model_name_or_path: str,
    backend_configs: Dict[str, Any],
    batch_size: int,
    corpus_path: str,
    gpu_ids: Optional[object] = None,
    is_multimodal: bool = False,
    backend: str = "sentence_transformers",
    index_backend: str = "faiss",
    index_backend_configs: Optional[Dict[str, Any]] = None,
    is_demo: bool = False,
    collection_name: str = "",
) -> None
```

**Function**

* Initializes retrieval service.
* Embedding Backend (backend): Responsible for converting text/images into vectors (Infinity, SentenceTransformers, OpenAI, BM25).
* Index Backend (index\_backend): Responsible for vector storage and retrieval (FAISS, Milvus).
* Demo Mode: If is\_demo=True, forces OpenAI + Milvus configuration, ignoring some parameters.

***

## `retriever_embed`

**Signature**

```python theme={null}
async def retriever_embed(
    embedding_path: Optional[str] = None,
    overwrite: bool = False,
    is_multimodal: bool = False,
) -> None
```

**Function**

* (Non-Demo Mode) Batches calculation of vector representations of corpus and saves as .npy file.
* Only applies to Dense Retriever backend (BM25 not supported).

***

## `retriever_index`

**Signature**

```python theme={null}
async def retriever_index(
    embedding_path: str,
    overwrite: bool = False,
    collection_name: str = "",
    corpus_path: str = ""
) -> None
```

**Function**

* Builds retrieval index.
* FAISS: Reads embedding\_path (.npy) to build local index file.
* Milvus / Demo: Reads corpus\_path (.jsonl), generates vectors and inserts into specified collection\_name.

***

## `retriever_search`

**Signature**

```python theme={null}
async def retriever_search(
    query_list: List[str],
    top_k: int = 5,
    query_instruction: str = "",
    collection_name: str = "",
) -> Dict[str, List[List[str]]]
```

**Function**

* Retrieves single or multiple queries.
* Automatically handles query vectorization (adds query\_instruction) and finds Top-K in specified collection\_name (for Milvus) or default index.

**Output Format (JSON)**

```json theme={null}
{"ret_psg": [["passage 1", "passage 2"], ["..." ]]} 
```

***

## `retriever_batch_search`

**Signature**

```python theme={null}
async def retriever_batch_search(
    batch_query_list: List[List[str]],
    top_k: int = 5,
    query_instruction: str = "",
    collection_name: str = "",
) -> Dict[str, List[List[List[str]]]]
```

**Function**

* Batch version of retriever\_search, accepts nested list input.

**Output Format (JSON)**

```json theme={null}
{"ret_psg_ls": [[["psg 1-1"], ["psg 1-2"]], [["psg 2-1"]]]}
```

***

## `bm25_index`

**Signature**

```python theme={null}
async def bm25_index(
    overwrite: bool = False,
) -> None
```

**Function**

* When `backend="bm25"`, builds BM25 sparse index and saves it.

***

## `bm25_search`

**Signature**

```python theme={null}
async def bm25_search(
    query_list: List[str],
    top_k: int = 5,
) -> Dict[str, List[List[str]]]
```

**Function**

* Keyword retrieval based on BM25 algorithm.

**Output Format (JSON)**

```json theme={null}
{"ret_psg": [["passage 1", "passage 2"], ["..." ]]} 
```

***

## `retriever_deploy_search`

**Signature**

```python theme={null}
async def retriever_deploy_search(
    retriever_url: str,
    query_list: List[str],
    top_k: int = 5,
    query_instruction: str = "",
) -> Dict[str, List[List[str]]]
```

**Function**

* As a client, calls remote retrieval service deployed at retriever\_url for query.

**Output Format (JSON)**

```json theme={null}
{"ret_psg": [["passage 1", "passage 2"], ["..." ]]} 
```

***

## `retriever_exa_search`

**Signature**

```python theme={null}
async def retriever_exa_search(
    query_list: List[str],
    top_k: Optional[int] | None = 5,
    retrieve_thread_num: Optional[int] | None = 1,
) -> Dict[str, List[List[str]]]
```

**Function**

* Calls **Exa** Web retrieval (requires `EXA_API_KEY`).

**Output Format (JSON)**

```json theme={null}
{"ret_psg": [["snippet 1", "snippet 2"], ["..." ]]} 
```

***

## `retriever_tavily_search`

**Signature**

```python theme={null}
async def retriever_tavily_search(
    query_list: List[str],
    top_k: Optional[int] | None = 5,
    retrieve_thread_num: Optional[int] | None = 1,
) -> Dict[str, List[List[str]]]
```

**Function**

* Calls **Tavily** Web retrieval (requires `TAVILY_API_KEY`).

**Output Format (JSON)**

```json theme={null}
{"ret_psg": [["snippet 1", "snippet 2"], ["..." ]]} 
```

***

## `retriever_zhipuai_search`

**Signature**

```python theme={null}
async def retriever_zhipuai_search(
    query_list: List[str],
    top_k: Optional[int] | None = 5,
    retrieve_thread_num: Optional[int] | None = 1,
) -> Dict[str, List[List[str]]]
```

**Function**

* Calls **ZhipuAI** `web_search` (requires `ZHIPUAI_API_KEY`).

**Output Format (JSON)**

```json theme={null}
{"ret_psg": [["snippet 1", "snippet 2"], ["..." ]]} 
```

***

## Configuration

```yaml servers/retriever/parameter.yaml icon="https://mintcdn.com/ultrarag/T7GffHzZitf6TThi/images/yaml.svg?fit=max&auto=format&n=T7GffHzZitf6TThi&q=85&s=69b41e79144bc908039c2ee3abbb1c3b" theme={null}
model_name_or_path: openbmb/MiniCPM-Embedding-Light
corpus_path: data/corpus_example.jsonl
embedding_path: embedding/embedding.npy
collection_name: wiki

# Embedding Backend Configuration
backend: sentence_transformers # options: infinity, sentence_transformers, openai, bm25
backend_configs:
  infinity:
    bettertransformer: false
    pooling_method: auto
    model_warmup: false
    trust_remote_code: true
  sentence_transformers:
    trust_remote_code: true
    sentence_transformers_encode:
      normalize_embeddings: false
      encode_chunk_size: 256
      q_prompt_name: query
      psg_prompt_name: document
      psg_task: null
      q_task: null
  openai:
    model_name: text-embedding-3-small
    base_url: "https://api.openai.com/v1"
    api_key: "abc"
  bm25:
    lang: en
    save_path: index/bm25

# Index Backend Configuration
index_backend: faiss # options: faiss, milvus
index_backend_configs:
  faiss:
    index_use_gpu: True
    index_chunk_size: 10000
    index_path: index/index.index
  milvus:
    uri: index/milvus_demo.db # Local file for Lite, or http://host:port
    token: null
    id_field_name: id
    vector_field_name: vector
    text_field_name: contents
    index_params:
      index_type: AUTOINDEX
      metric_type: IP

batch_size: 16
top_k: 5
gpu_ids: "1"
query_instruction: ""
is_multimodal: false
overwrite: false
retrieve_thread_num: 1
retriever_url: "http://127.0.0.1:64501"
is_demo: false
```

Parameter Description:

| Parameter               | Type | Description                                                                     |
| ----------------------- | ---- | ------------------------------------------------------------------------------- |
| `model_name_or_path`    | str  | Retrieval model path or name (e.g., HuggingFace model ID)                       |
| `corpus_path`           | str  | Input corpus JSONL file path                                                    |
| `embedding_path`        | str  | Vector file save path (`.npy`)                                                  |
| `collection_name`       | str  | Milvus collection name                                                          |
| `backend`               | str  | Select retrieval backend: `infinity`, `sentence_transformers`, `openai`, `bm25` |
| `index_backend`         | str  | Index backend: `faiss`, `milvus`                                                |
| `backend_configs`       | dict | Parameter configuration for each backend (see table below)                      |
| `index_backend_configs` | dict | Parameter configuration for each index backend (see table below)                |
| `batch_size`            | int  | Batch size for vector generation or retrieval                                   |
| `top_k`                 | int  | Number of returned candidate passages                                           |
| `gpu_ids`               | str  | Specify visible GPU devices, e.g., `"0,1"`                                      |
| `query_instruction`     | str  | Query prefix (used by instruction-tuning models)                                |
| `is_multimodal`         | bool | Whether to enable multimodal embedding (e.g., image)                            |
| `overwrite`             | bool | Whether to overwrite if embedding or index file already exists                  |
| `retrieve_thread_num`   | int  | Concurrent thread number for external Web retrieval (Exa/Tavily/Zhipu)          |
| `retriever_url`         | str  | URL of deployed retriever server                                                |
| `is_demo`               | bool | Demo mode switch (forces OpenAI+Milvus, simplified configuration)               |

`backend_configs` Sub-items:

| Backend                    | Parameter                      | Type | Description                                                |
| -------------------------- | ------------------------------ | ---- | ---------------------------------------------------------- |
| **infinity**               | `bettertransformer`            | bool | Whether to enable efficient inference optimization         |
|                            | `pooling_method`               | str  | Pooling method (e.g., `auto`, `mean`)                      |
|                            | `model_warmup`                 | bool | Whether to preload model into VRAM                         |
|                            | `trust_remote_code`            | bool | Whether to trust remote code (Applicable to custom models) |
| **sentence\_transformers** | `trust_remote_code`            | bool | Whether to trust remote model code                         |
|                            | `sentence_transformers_encode` | dict | Encoding detailed parameters, see table below              |
| **openai**                 | `model_name`                   | str  | OpenAI model name (e.g., `text-embedding-3-small`)         |
|                            | `base_url`                     | str  | API base address                                           |
|                            | `api_key`                      | str  | OpenAI API Key                                             |
| **bm25**                   | `lang`                         | str  | Language (determines stop words and tokenizer)             |
|                            | `save_path`                    | str  | Save directory for BM25 sparse index                       |

`sentence_transformers_encode` Parameters:

| Parameter              | Type | Description                                                             |
| ---------------------- | ---- | ----------------------------------------------------------------------- |
| `normalize_embeddings` | bool | Whether to normalize vectors                                            |
| `encode_chunk_size`    | int  | Encoding chunk size (avoid VRAM overflow)                               |
| `q_prompt_name`        | str  | Query template name                                                     |
| `psg_prompt_name`      | str  | Passage template name                                                   |
| `q_task`               | str  | Task description (for cases where specific models need to specify Task) |
| `psg_task`             | str  | Task description (for cases where specific models need to specify Task) |

`index_backend_configs` Parameters:

| Backend | Parameter           | Type | Description                                                          |
| ------- | ------------------- | ---- | -------------------------------------------------------------------- |
| faiss   | index\_use\_gpu     | bool | Whether to use GPU for building and retrieving index                 |
|         | index\_chunk\_size  | int  | Batch size when building index                                       |
|         | index\_path         | str  | Save path for FAISS index file (.index)                              |
| milvus  | uri                 | str  | Milvus connection address (Local file path enables Milvus Lite)      |
|         | token               | str  | Auth Token (if needed)                                               |
|         | id\_field\_name     | str  | Primary key field name (default id)                                  |
|         | vector\_field\_name | str  | Vector field name (default vector)                                   |
|         | text\_field\_name   | str  | Text content field name (default contents)                           |
|         | id\_max\_length     | int  | Maximum length of string primary key                                 |
|         | text\_max\_length   | int  | Maximum length of text field (truncated if exceeded)                 |
|         | metric\_type        | str  | Distance metric type (e.g., IP inner product, L2 Euclidean distance) |
|         | index\_params       | Dict | Index construction parameters (e.g., index\_type: AUTOINDEX)         |
|         | search\_params      | Dict | Retrieval parameters (e.g., nprobe etc.)                             |
|         | index\_chunk\_size  | int  | Batch size when inserting data                                       |