reranker_init
Signature
async def reranker_init(
model_name_or_path: str,
backend_configs: Dict[str, Any],
batch_size: int,
gpu_ids: Optional[object] = None,
backend: str = "infinity",
) -> None
- Initializes reranker backend and model.
reranker_rerank
Signature
async def reranker_rerank(
query_list: List[str],
passages_list: List[List[str]],
top_k: int = 5,
query_instruction: str = "",
) -> Dict[str, List[Any]]
- Reranks candidate passages:
{
"rerank_psg": [
["best passage for q0", "..."],
["best passage for q1", "..."]
]
}
Configuration
servers/reranker/parameter.yamlmodel_name_or_path: openbmb/MiniCPM-Reranker-Light
backend: sentence_transformers # options: infinity, sentence_transformers, openai
backend_configs:
infinity:
bettertransformer: false
pooling_method: auto
device: cuda
model_warmup: false
trust_remote_code: true
sentence_transformers:
device: cuda
trust_remote_code: true
openai:
model_name: text-embedding-3-small
base_url: "https://api.openai.com/v1"
api_key: ""
gpu_ids: 0
top_k: 5
batch_size: 16
query_instruction: ""
| Parameter | Type | Description |
|---|---|---|
model_name_or_path | str | Model path or name (local or HuggingFace repo) |
backend | str | Select backend type: infinity, sentence_transformers or openai |
backend_configs | dict | Exclusive parameter settings for each backend |
gpu_ids | str/int | Specify GPU ID (can be multi-card, e.g., "0,1") |
top_k | int | Number of reranked results returned |
batch_size | int | Sample quantity per batch |
query_instruction | str | Query prefix hint, used for prompt engineering or query modification |
backend_configs Detailed Description:
| Backend | Parameter | Description |
|---|---|---|
| infinity | device | Device type (cuda / cpu) |
bettertransformer | Whether to enable accelerated inference | |
pooling_method | Vector pooling strategy | |
model_warmup | Whether to warmup model | |
trust_remote_code | Whether to trust remote code (Required for HuggingFace models) | |
| sentence_transformers | device | Device type (cuda / cpu) |
trust_remote_code | Whether to trust remote code | |
| openai | model_name | API Model name |
base_url | API access address | |
api_key | OpenAI API Key |