Skip to main content

​
reranker_init

Signature 
async def reranker_init(
    model_name_or_path: str,
    backend_configs: Dict[str, Any],
    batch_size: int,
    gpu_ids: Optional[object] = None,
    backend: str = "infinity",
) -> None
Function
  • Initializes reranker backend and model.

​
reranker_rerank

Signature 
async def reranker_rerank(
    query_list: List[str],
    passages_list: List[List[str]],
    top_k: int = 5,
    query_instruction: str = "",
) -> Dict[str, List[Any]]
Function
  • Reranks candidate passages:
Output Format (JSON) 
{
  "rerank_psg": [
    ["best passage for q0", "..."],
    ["best passage for q1", "..."]
  ]
}

​
Configuration

https://mintcdn.com/ultrarag/T7GffHzZitf6TThi/images/yaml.svg?fit=max&auto=format&n=T7GffHzZitf6TThi&q=85&s=69b41e79144bc908039c2ee3abbb1c3bservers/reranker/parameter.yaml
model_name_or_path: openbmb/MiniCPM-Reranker-Light
backend: sentence_transformers # options: infinity, sentence_transformers, openai
backend_configs:
  infinity:
    bettertransformer: false
    pooling_method: auto
    device: cuda
    model_warmup: false
    trust_remote_code: true
  sentence_transformers:
    device: cuda
    trust_remote_code: true
  openai:
    model_name: text-embedding-3-small
    base_url: "https://api.openai.com/v1"
    api_key: ""

gpu_ids: 0
top_k: 5
batch_size: 16
query_instruction: ""
Parameter Description:
ParameterTypeDescription
model_name_or_pathstrModel path or name (local or HuggingFace repo)
backendstrSelect backend type: infinity, sentence_transformers or openai
backend_configsdictExclusive parameter settings for each backend
gpu_idsstr/intSpecify GPU ID (can be multi-card, e.g., "0,1")
top_kintNumber of reranked results returned
batch_sizeintSample quantity per batch
query_instructionstrQuery prefix hint, used for prompt engineering or query modification
backend_configs Detailed Description:
BackendParameterDescription
infinitydeviceDevice type (cuda / cpu)
bettertransformerWhether to enable accelerated inference
pooling_methodVector pooling strategy
model_warmupWhether to warmup model
trust_remote_codeWhether to trust remote code (Required for HuggingFace models)
sentence_transformersdeviceDevice type (cuda / cpu)
trust_remote_codeWhether to trust remote code
openaimodel_nameAPI Model name
base_urlAPI access address
api_keyOpenAI API Key