Prompt

Function

The Prompt Tool is the core component for constructing language model inputs (Prompts). Each Prompt Tool is defined by the @app.prompt decorator, and its main responsibilities are: Based on the input content (such as questions, retrieved passages, etc.), load the corresponding template file and generate a standardized PromptMessage, so that it can be directly passed to the Large Language Model (LLM) for generation or reasoning.

Implementation Example

Step 1: Prepare Prompt Template

Please save your prompt template as a file ending with .jinja, for example:

prompt/qa_rag_boxed.jinja

Please answer the following question based on the given documents.
Think step by step.
Provide your final answer in the format \boxed{YOUR_ANSWER}.

Documents:
{{documents}}

Question: {{question}}

Step 2: Implement Tool in Prompt Server

Call the load_prompt_template method to load the template, and implement a tool function in the Prompt Server to assemble the prompt:

servers/prompt/src/prompt.py

@app.prompt(output="q_ls,ret_psg,template->prompt_ls")
def qa_rag_boxed(
    q_ls: List[str], ret_psg: List[str | Any], template: str | Path
) -> list[PromptMessage]:
    template: Template = load_prompt_template(template)
    ret = []
    for q, psg in zip(q_ls, ret_psg):
        passage_text = "\n".join(psg)
        p = template.render(question=q, documents=passage_text)
        ret.append(p)
    return ret

Usage Example

Before calling the model generation tool, you need to construct the input prompt through the corresponding Prompt Tool.

examples/rag_full.yaml

servers:
  benchmark: servers/benchmark
  retriever: servers/retriever
  prompt: servers/prompt
  generation: servers/generation
  evaluation: servers/evaluation
  custom: servers/custom

pipeline:
- benchmark.get_data
- retriever.retriever_init
- retriever.retriever_embed
- retriever.retriever_index
- retriever.retriever_search
- generation.generation_init
- prompt.qa_rag_boxed
- generation.generate
- custom.output_extract_from_boxed
- evaluation.evaluate

Multi-Prompt Tool Calling Scenario

In some complex Pipelines, the model often needs to perform different tasks at different stages — for example, first generating sub-questions, and then generating the final answer based on new retrieval results. In this case, multiple Prompt Tools need to be configured in the same Pipeline, each responsible for different prompt construction logic.

examples/rag_loop.yaml

# MCP Server
servers:
  benchmark: servers/benchmark
  retriever: servers/retriever
  prompt: servers/prompt
  generation: servers/generation
  evaluation: servers/evaluation
  custom: servers/custom

# MCP Client Pipeline
pipeline:
- benchmark.get_data
- retriever.retriever_init
- generation.generation_init
- retriever.retriever_search
- loop:
    times: 3
    steps:
    - prompt.gen_subq
    - generation.generate:
        output:
          ans_ls: subq_ls
    - retriever.retriever_search:
        input:
          query_list: subq_ls
        output:
          ret_psg: temp_psg
    - custom.merge_passages
- prompt.qa_rag_boxed
- generation.generate
- custom.output_extract_from_boxed
- evaluation.evaluate

If you want to load different templates for different tasks, you need to specify independent template field names for each Prompt Tool during registration:

servers/prompt/src/prompt.py

@app.prompt(output="q_ls,ret_psg,template->prompt_ls")
def qa_rag_boxed(
    q_ls: List[str], ret_psg: List[str | Any], template: str | Path
) -> list[PromptMessage]:
    template: Template = load_prompt_template(template)
    ret = []
    for q, psg in zip(q_ls, ret_psg):
        passage_text = "\n".join(psg)
        p = template.render(question=q, documents=passage_text)
        ret.append(p)
    return ret

@app.prompt(output="q_ls,ret_psg,gen_subq_template->prompt_ls")
def gen_subq(
    q_ls: List[str],
    ret_psg: List[str | Any],
    template: str | Path,
) -> List[PromptMessage]:
    template: Template = load_prompt_template(template)
    all_prompts = []
    for q, psg in zip(q_ls, ret_psg):
        passage_text = "\n".join(psg)
        p = template.render(question=q, documents=passage_text)
        all_prompts.append(p)
    return all_prompts

Subsequently, add the corresponding template field in servers/prompt/parameter.yaml:

Please ensure this modification is completed before executing the build command.

servers/prompt/parameter.yaml

# servers/prompt/parameter.yaml

# QA
template: prompt/qa_boxed.jinja

# RankCoT
kr_template: prompt/RankCoT_knowledge_refinement.jinja
qa_template: prompt/RankCoT_question_answering.jinja

# Search-R1
search_r1_gen_template: prompt/search_r1_append.jinja

# R1-Searcher
r1_searcher_gen_template: prompt/r1_searcher_append.jinja

# For other prompts, please add parameters here as needed

# Take webnote as an example:
webnote_gen_plan_template: prompt/webnote_gen_plan.jinja
webnote_init_page_template: prompt/webnote_init_page.jinja
webnote_gen_subq_template: prompt/webnote_gen_subq.jinja
webnote_fill_page_template: prompt/webnote_fill_page.jinja
webnote_gen_answer_template: prompt/webnote_gen_answer.jinja

gen_subq_template: prompt/gen_subq.jinja

Run the following command to compile the Pipeline:

ultrarag build rag_loop.yaml

The system will automatically register the new field in the generated parameter file:

examples/rag_loop_parameter.yaml

...
prompt:
  gen_subq_template: prompt/gen_subq.jinja
  template: prompt/qa_boxed.jinja
retriever:
  backend: sentence_transformers
...

Then the Pipeline can be executed normally.

Get Started

RAG Servers

RAG Client

Develop Guide

Typical Implementation

Function

Implementation Example

Step 1: Prepare Prompt Template

Step 2: Implement Tool in Prompt Server

Usage Example

Multi-Prompt Tool Calling Scenario

​Function

​Implementation Example

​Step 1: Prepare Prompt Template

​Step 2: Implement Tool in Prompt Server

​Usage Example

​Multi-Prompt Tool Calling Scenario

Function

Implementation Example

Step 1: Prepare Prompt Template

Step 2: Implement Tool in Prompt Server

Usage Example

Multi-Prompt Tool Calling Scenario