Custom

Function

The Custom Server is used to store custom tool functions that cannot be classified into standard modules (such as Retriever, Generation, Evaluation, etc.). It provides developers with a flexible extension space for implementing various logical components that cooperate with core RAG modules, such as:

Data cleaning and preprocessing
Keyword extraction or feature construction
Specific task logic (such as answer extraction, formatting, filtering, etc.)

The Custom Server is your free toolbox — any functional logic that does not belong to core Servers can be defined and reused here.

Implementation Example

The following takes a common example output_extract_from_boxed to show how to customize and register a Tool.

servers/custom/src/custom.py

@app.tool(output="ans_ls->pred_ls")
def output_extract_from_boxed(ans_ls: List[str]) -> Dict[str, List[str]]:
    def extract(ans: str) -> str:
        start = ans.rfind(r"\boxed{")
        if start == -1:
            content = ans.strip()
        else:
            i = start + len(r"\boxed{")
            brace_level = 1
            end = i
            while end < len(ans) and brace_level > 0:
                if ans[end] == "{":
                    brace_level += 1
                elif ans[end] == "}":
                    brace_level -= 1
                end += 1
            content = ans[i : end - 1].strip()
            content = re.sub(r"^\$+|\$+$", "", content).strip()
            content = re.sub(r"^\\\(|\\\)$", "", content).strip()
            if content.startswith(r"\text{") and content.endswith("}"):
                content = content[len(r"\text{") : -1].strip()
            content = content.strip("()").strip()

        content = content.replace("\\", " ")
        content = content.replace("  ", " ")
        return content

    return {"pred_ls": [extract(ans) for ans in ans_ls]}

The function of this tool is to extract the final answer text in \boxed{...} format from the model output string, and the output result will be mapped to the variable pred_ls for use by downstream evaluation or post-processing modules.

Usage Example

After defining the custom tool, you only need to register the custom module in the Pipeline and call the corresponding Tool:

examples/rag_full.yaml

# MCP Server
servers:
  benchmark: servers/benchmark
  retriever: servers/retriever
  prompt: servers/prompt
  generation: servers/generation
  evaluation: servers/evaluation
  custom: servers/custom

# MCP Client Pipeline
pipeline:
- benchmark.get_data
- retriever.retriever_init
- retriever.retriever_embed
- retriever.retriever_index
- retriever.retriever_search
- generation.generation_init
- prompt.qa_rag_boxed
- generation.generate
- custom.output_extract_from_boxed
- evaluation.evaluate

In this example, custom.output_extract_from_boxed is used to extract the standardized answer from the model output, and then it is passed to evaluation.evaluate for evaluation.

Get Started

RAG Servers

RAG Client

Develop Guide

Typical Implementation

Function

Implementation Example

Usage Example

​Function

​Implementation Example

​Usage Example

Function

Implementation Example

Usage Example