Skip to main content

​
reranker_init

Signature 
async def reranker_init(
    model_name_or_path: str,
    backend_configs: Dict[str, Any],
    batch_size: int,
    gpu_ids: Optional[object] = None,
    backend: str = "infinity",
) -> None
Function
  • Initializes the reranking backend and model.

​
reranker_rerank

Signature 
async def reranker_rerank(
    query_list: List[str],
    passages_list: List[List[str]],
    top_k: int = 5,
    query_instruction: str = "",
) -> Dict[str, List[Any]]
Function
  • Performs reranking on candidate passages.
Output Format (JSON) 
{
  "rerank_psg": [
    ["best passage for q0", "..."],
    ["best passage for q1", "..."]
  ]
}

​
Parameter Configuration

https://mintcdn.com/ultrarag/T7GffHzZitf6TThi/images/yaml.svg?fit=max&auto=format&n=T7GffHzZitf6TThi&q=85&s=69b41e79144bc908039c2ee3abbb1c3bservers/reranker/parameter.yaml
model_name_or_path: openbmb/MiniCPM-Reranker-Light
backend: sentence_transformers # options: infinity, sentence_transformers, openai
backend_configs:
  infinity:
    bettertransformer: false
    pooling_method: auto
    device: cuda
    model_warmup: false
    trust_remote_code: true
  sentence_transformers:
    device: cuda
    trust_remote_code: true
  openai:
    model_name: text-embedding-3-small
    base_url: "https://api.openai.com/v1"
    api_key: ""

gpu_ids: 0
top_k: 5
batch_size: 16
query_instruction: ""
Parameter Description:
ParameterTypeDescription
model_name_or_pathstrModel path or name (local or from HuggingFace Hub)
backendstrBackend type: infinity, sentence_transformers, or openai
backend_configsdictSpecific configuration settings for each backend
gpu_idsstr/intGPU device ID(s) (supports multiple, e.g., "0,1")
top_kintNumber of reranked results to return
batch_sizeintNumber of samples per processing batch
query_instructionstrOptional query prefix for prompt tuning or query modification
Detailed description of backend_configs:
BackendParameterDescription
infinitydeviceDevice type (cuda / cpu)
bettertransformerEnables optimized inference acceleration
pooling_methodVector pooling strategy
model_warmupWhether to preload the model into memory
trust_remote_codeWhether to trust remote code (required for HuggingFace models)
sentence_transformersdeviceDevice type (cuda / cpu)
trust_remote_codeWhether to trust remote code
openaimodel_nameAPI model name
base_urlAPI access URL
api_keyOpenAI API key