Skip to main content
It is recommended to study this section together with the tutorial Branch Structures.

​
Purpose

In complex RAG reasoning tasks, it is often necessary to dynamically determine the next execution path based on intermediate results (for example, the model’s current output or retrieval results).
The Router Server is designed for exactly this purpose — it evaluates the current state based on input information and returns a custom branch label (state identifier) used to drive branching and dynamic control within the Pipeline.

​
Implementation Example

The following example demonstrates how to implement a simple Router Tool. Suppose that in the current RAG workflow, the model needs to decide whether the retrieved documents already contain sufficient information to answer the question:
If the information is sufficient, the process stops; otherwise, it continues retrieval. Here’s how to implement a Router Tool:
servers/router/src/router.py
@app.tool(output="ans_ls->ans_ls")
def check_model_state(ans_ls: List[str]) -> Dict[str, List[Dict[str, str]]]:
    def check_state(text):
        if "<search>" in text:
            return True
        else:
            return False
    ans_ls = [
        {
            "data": answer,
            "state": "continue" if check_state(answer) else "stop",
        }
        for answer in ans_ls
    ]
    return {"ans_ls": ans_ls}
This Tool adds a state label to each answer to guide the subsequent workflow execution:
  • continue: Information is insufficient — continue retrieval.
  • stop: Information is sufficient — terminate the process.

​
Usage Example

The defined Router Tool must be used in combination with the branch: and router: structures to enable state-based dynamic branching.
https://mintcdn.com/ultrarag/T7GffHzZitf6TThi/images/yaml.svg?fit=max&auto=format&n=T7GffHzZitf6TThi&q=85&s=69b41e79144bc908039c2ee3abbb1c3bexamples/rag_branch.yaml
# MCP Server
servers:
  benchmark: servers/benchmark
  retriever: servers/retriever
  prompt: servers/prompt
  generation: servers/generation
  evaluation: servers/evaluation
  custom: servers/custom
  router: servers/router

# MCP Client Pipeline
pipeline:
- benchmark.get_data
- retriever.retriever_init
- generation.generation_init
- retriever.retriever_search
- loop:
    times: 10
    steps:
    - prompt.check_passages
    - generation.generate
    - branch:
        router:
        - router.check_model_state
        branches:
          continue:
          - prompt.gen_subq
          - generation.generate:
              output:
                ans_ls: subq_ls
          - retriever.retriever_search:
              input:
                query_list: subq_ls
              output:
                ret_psg: temp_psg
          - custom.merge_passages
          stop: []
- prompt.qa_rag_boxed
- generation.generate
- custom.output_extract_from_boxed
- evaluation.evaluate
This example illustrates a typical iterative reasoning process:
When router.check_model_state detects that the model’s output contains the <search> tag, it enters the continue branch to perform another retrieval;
otherwise, it follows the stop branch to exit the loop.