The Custom Server is used to store custom tool functions that cannot be classified into standard modules (such as Retriever, Generation, Evaluation, etc.).
It provides developers with a flexible extension space for implementing various logical components that cooperate with core RAG modules, such as:
Data cleaning and preprocessing
Keyword extraction or feature construction
Specific task logic (such as answer extraction, formatting, filtering, etc.)
The Custom Server is your free toolbox — any functional logic that does not belong to core Servers can be defined and reused here.
The following takes a common example output_extract_from_boxed to show how to customize and register a Tool.
servers/custom/src/custom.py
Copy
@app.tool(output="ans_ls->pred_ls")def output_extract_from_boxed(ans_ls: List[str]) -> Dict[str, List[str]]: def extract(ans: str) -> str: start = ans.rfind(r"\boxed{") if start == -1: content = ans.strip() else: i = start + len(r"\boxed{") brace_level = 1 end = i while end < len(ans) and brace_level > 0: if ans[end] == "{": brace_level += 1 elif ans[end] == "}": brace_level -= 1 end += 1 content = ans[i : end - 1].strip() content = re.sub(r"^\$+|\$+$", "", content).strip() content = re.sub(r"^\\\(|\\\)$", "", content).strip() if content.startswith(r"\text{") and content.endswith("}"): content = content[len(r"\text{") : -1].strip() content = content.strip("()").strip() content = content.replace("\\", " ") content = content.replace(" ", " ") return content return {"pred_ls": [extract(ans) for ans in ans_ls]}
The function of this tool is to extract the final answer text in \boxed{...} format from the model output string,
and the output result will be mapped to the variable pred_ls for use by downstream evaluation or post-processing modules.
In this example, custom.output_extract_from_boxed is used to extract the standardized answer from the model output,
and then it is passed to evaluation.evaluate for evaluation.
examples/rag_full.yaml