# LlamaIndex Integration Guide

How to build LlamaIndex agents that work with the UiPath platform. This guide covers project structure, entrypoint detection, LLM models, and common pitfalls.

## Scaffolding a New Project

If there is **no existing agent code**, scaffold a LlamaIndex project with:

```bash
mkdir my-agent && cd my-agent
uip codedagent new my-agent
```

This generates `main.py` (with a Workflow template), `llama_index.json`, and `pyproject.toml`. Then modify `main.py` to implement your actual agent logic.

> **Prerequisite:** `uipath-llamaindex` must be installed for the LlamaIndex template to be used.

## Project Structure

LlamaIndex agents use the `Workflow` class from `llama_index.core.workflow` and a `llama_index.json` configuration file.

```
my-agent/
├── main.py               # Workflow definition — exports `agent` variable
├── llama_index.json       # Workflow configuration — maps names to workflow variables
├── pyproject.toml         # Dependencies (must include uipath-llamaindex)
├── .env                   # Environment variables
└── ...                    # Generated by uip codedagent init
```

**`llama_index.json`** tells the runtime where to find your workflow:

```json
{
  "workflows": {
    "agent": "main.py:workflow"
  }
}
```

- `"agent"` — the entrypoint name (used with `uip codedagent run agent`)
- `"main.py:workflow"` — path to the Python file and the variable name (`file:variable`)
- The variable can be named anything — just ensure it matches the config

---

## Required Dependencies

Every LlamaIndex agent needs these in `pyproject.toml`:

```toml
[project]
name = "my-agent"
version = "0.1.0"
description = "My LlamaIndex agent"
requires-python = ">=3.11"

dependencies = [
    "uipath-llamaindex",
]

[dependency-groups]
dev = [
    "uipath-dev",
]
```

**Key point:** `uipath-llamaindex` is required — it registers the LlamaIndex runtime factory that enables `uip codedagent run`, `uip codedagent init`, and deployment to detect and execute your workflow.

For agents that use specific LLM providers, add the appropriate model dependency:

```toml
dependencies = [
    "uipath-llamaindex",
    "llama-index-llms-openai>=0.6.10",    # If using OpenAI models
    "llama-index-llms-bedrock>=0.3.0",     # If using AWS Bedrock
    "llama-index-llms-google-genai>=0.8.0", # If using Google Gemini
]
```

---

## LLM Models

Use UiPath's LLM wrapper instead of raw `OpenAI()` from llama-index. This routes through UiPath's LLM Gateway for centralized model management and billing.

### UiPathOpenAI

Instantiate LLM clients inside a `@step` (not at module level, not as a class attribute on the `Workflow`). See [../lifecycle/build.md](../lifecycle/build.md) § Additional Instructions for the full rule.

```python
from uipath_llamaindex.llms import UiPathOpenAI

# Inside a @step:
llm = UiPathOpenAI()
```

- Routes through UiPath's passthrough endpoint (Azure OpenAI API format)
- No API key needed — uses UiPath authentication

### Selecting a Model

```python
from uipath_llamaindex.llms import UiPathOpenAI, OpenAIModel

llm = UiPathOpenAI(model=OpenAIModel.GPT_4O_2024_11_20)
```

The enum is `OpenAIModel` (singular). `GeminiModel` and `BedrockModel` are also available from `uipath_llamaindex.llms` for non-OpenAI vendors. Run `uip codedagent list-models` to list the models available in your tenant.

---

## Workflow Definition

### Input/Output Events

LlamaIndex agents use `StartEvent` and `StopEvent` subclasses for input and output schemas:

```python
from llama_index.core.workflow import StartEvent, StopEvent, Event, Workflow, step

class MyStartEvent(StartEvent):
    query: str
    max_results: int = 5

class MyStopEvent(StopEvent):
    answer: str
    sources: list[str] = []
```

- `StartEvent` subclass → becomes the agent's **input schema**
- `StopEvent` subclass → becomes the agent's **output schema**
- `Event` subclass → intermediate events passed between steps

### Building a Workflow

```python
from llama_index.core.workflow import StartEvent, StopEvent, Event, Workflow, step
from uipath_llamaindex.llms import UiPathOpenAI

class QueryEvent(StartEvent):
    query: str

class AnswerEvent(StopEvent):
    answer: str

class IntermediateEvent(Event):
    processed_query: str

class MyAgent(Workflow):
    @step
    async def process_query(self, ev: QueryEvent) -> IntermediateEvent:
        return IntermediateEvent(processed_query=ev.query.strip().lower())

    @step
    async def generate_answer(self, ev: IntermediateEvent) -> AnswerEvent:
        llm = UiPathOpenAI()
        response = await llm.acomplete(f"Answer this question: {ev.processed_query}")
        return AnswerEvent(answer=str(response))

workflow = MyAgent(timeout=60, verbose=False)
```

**Important:** The variable name MUST match what's in `llama_index.json` (e.g., `workflow` if the config says `main.py:workflow`) and be an instantiated `Workflow`. Instantiate LLM clients inside the step that uses them — class attributes on a `Workflow` evaluate at class-definition (import) time and will fail before auth is configured.

### Step Decorator

Use `@step` on async methods inside a `Workflow` class:
- Each step receives an event and returns another event
- Steps are automatically connected by their event types
- The workflow starts when a `StartEvent` (or subclass) is emitted
- The workflow ends when a `StopEvent` (or subclass) is returned

---

## Tracing

LlamaIndex agents get **automatic tracing** via the `openinference-instrumentation-llama-index` package (included with `uipath-llamaindex`). You do NOT need `@traced()` on workflow steps.

Use `@traced()` only on helper functions **outside** the workflow:

```python
from uipath.tracing import traced

@traced(name="postprocess", span_type="tool")
async def postprocess(text: str) -> str:
    return text.strip().lower()
```

---

## Running `uip codedagent init`

After creating your workflow, generate project configuration:

```bash
uv sync
uip codedagent init
```

### How Entrypoint Detection Works

1. `uip codedagent init` loads all registered middleware via Python entry points
2. The `uipath-llamaindex` package registers a LlamaIndex middleware
3. The middleware checks for `llama_index.json` — if found, it handles init
4. The runtime imports your Python file and validates the workflow variable
5. Input/Output schemas are extracted from your `StartEvent`/`StopEvent` subclasses

### Troubleshooting `uip codedagent init`

**"No function entrypoints found"**
- Ensure `llama_index.json` exists with the correct `"workflows"` mapping
- The file path and variable name must match exactly: `"main.py:agent"`
- Ensure `uipath-llamaindex` is installed (`uv sync`)

**Import errors during init**
- `uip codedagent init` imports your Python file to introspect the workflow
- Avoid side effects at module level (API calls, heavy initialization)
- Instantiate the LLM inside each `@step` or via an instance-level lazy property, not a class attribute

**Schema not detected**
- Ensure you use custom `StartEvent` and `StopEvent` subclasses with typed fields
- Plain `StartEvent` / `StopEvent` without fields will use default schemas

---

## Running Your Agent

```bash
uip codedagent run agent '{"query": "What is Python?"}'
```

The entrypoint name comes from the key in `llama_index.json` `"workflows"`.

---

## Conversational Agents

For the cross-framework contract (the `isConversational` flag, local-run options), see `../capabilities/conversational-agents.md`.

### In-Process Input

Type the workflow's `StartEvent` with a single `user_msg: str` field — the runtime extracts the inline text from the wire envelope's `contentParts` and hands the workflow `user_msg: str`.

```python
from llama_index.core.workflow import StartEvent

class ChatStartEvent(StartEvent):
    user_msg: str
```

The wire envelope (see `../capabilities/conversational-agents.md` § Wire Envelope) is converted before your workflow sees it; the workflow does not handle `role` / `contentParts`.

### Two Implementation Options

Both are valid — pick based on the agent's needs:

- **Prebuilt agent** — `AgentWorkflow.from_tools_or_functions(...)` from `llama_index.core.agent.workflow`. The `user_msg` input is wired in automatically; export the workflow instance from your entry point.
- **Custom workflow** — any `Workflow` subclass whose first step accepts a `StartEvent` with `user_msg: str`. The first step can be fully deterministic (e.g. validation or routing) before any LLM call.

### Local Run

See `../capabilities/conversational-agents.md` § Running Locally for the `--keep-state-file` flag (required on every turn) and § Wire Envelope for the `turn1.json` shape.

---

## UiPath Platform Integration

### Context Grounding (RAG)

Two RAG primitives ship in `uipath-llamaindex`. Pick by workflow shape:

| Primitive | Import | Use when |
|-----------|--------|----------|
| `ContextGroundingQueryEngine` | `uipath_llamaindex.query_engines` | Retrieval + LLM synthesis in one call, or exposing the index as a `QueryEngineTool` in an agentic workflow. Constructor REQUIRES `response_synthesizer` (no default). |
| `ContextGroundingRetriever` | `uipath_llamaindex.retrievers` | Deterministic workflow RAG — retrieve raw passages in a `@step`, then synthesize the answer with your own prompt and LLM call. |

The query engine wraps `ContextGroundingRetriever` + your synthesizer internally — use the retriever directly when the workflow already has its own synthesis step.

> **Prefer using these primitives — not raw `sdk.context_grounding.search()` / `search_async()`.** They return LlamaIndex `NodeWithScore` objects that plug into synthesizers and tools, and declare `index_name` + `folder_path` at one call site — the exact shape of the `index` binding entry (see [../lifecycle/bindings-reference.md](../lifecycle/bindings-reference.md)). `index` bindings have no virtual fallback: the index must exist in Orchestrator before `uip codedagent push`.

Both accept `index_name`, `folder_path` (or `folder_key`), and `number_of_results` (default 10). Always pass `folder_path`, even when the index lives in the execution folder resolved from your auth context — omitting it can leave the `index` binding incorrectly applied. Instantiate both inside a `@step` — never at module level (same lazy-client rule as LLMs, § UiPathOpenAI above).

#### ContextGroundingQueryEngine

Build the required `response_synthesizer` with `get_response_synthesizer(llm=UiPathOpenAI())` — omitting it raises a constructor error:

```python
from llama_index.core.response_synthesizers import get_response_synthesizer
from uipath_llamaindex.llms import UiPathOpenAI
from uipath_llamaindex.query_engines import ContextGroundingQueryEngine

# Inside a @step:
query_engine = ContextGroundingQueryEngine(
    index_name="my_knowledge_base",
    folder_path="Shared",
    response_synthesizer=get_response_synthesizer(llm=UiPathOpenAI()),
)
response = await query_engine.aquery(ev.question)
```

As an agent tool:

```python
from llama_index.core.tools import QueryEngineTool, ToolMetadata

tools = [
    QueryEngineTool(
        query_engine=query_engine,
        metadata=ToolMetadata(
            name="knowledge_base",
            description="Search the knowledge base for information"
        ),
    )
]
```

#### Workflow RAG with ContextGroundingRetriever

```python
from llama_index.core.workflow import StartEvent, StopEvent, Workflow, step
from uipath_llamaindex.llms import UiPathOpenAI
from uipath_llamaindex.retrievers import ContextGroundingRetriever

class QuestionEvent(StartEvent):
    question: str

class AnswerEvent(StopEvent):
    answer: str

class RagAgent(Workflow):
    @step
    async def answer(self, ev: QuestionEvent) -> AnswerEvent:
        retriever = ContextGroundingRetriever(
            index_name="my_knowledge_base",
            folder_path="Shared",
            number_of_results=5,
        )
        nodes = await retriever.aretrieve(ev.question)
        if not nodes:
            return AnswerEvent(answer="No relevant passages found in the knowledge base.")
        passages = "\n\n".join(n.node.get_content() for n in nodes)
        llm = UiPathOpenAI()
        response = await llm.acomplete(
            "Answer the question using ONLY the passages below. "
            "If they do not answer it, say so.\n\n"
            f"Passages:\n{passages}\n\nQuestion: {ev.question}"
        )
        return AnswerEvent(answer=str(response))

workflow = RagAgent(timeout=60)
```

### Human-in-the-Loop

```python
from llama_index.core.workflow import Context, HumanResponseEvent, InputRequiredEvent

@step
async def human_review(self, ctx: Context, ev: ReviewEvent) -> ApprovedEvent:
    response = await ctx.wait_for_event(
        HumanResponseEvent,
        waiter_id="review_request",
        waiter_event=InputRequiredEvent(
            prefix="Please review this result. Approve? (yes/no)"
        ),
    )
    approved = response.response.strip().lower() == "yes"
    return ApprovedEvent(approved=approved)
```

### Process Invocation

```python
from uipath_llamaindex.models import InvokeProcessEvent, WaitJobEvent

@step
async def invoke_process(self, ev: TriggerEvent) -> WaitJobEvent:
    return InvokeProcessEvent(
        process_name="MyProcess",
        input_arguments={"data": ev.data},
    )
```

---

## Complete Examples

### Simple Workflow Agent

```python
# main.py
from llama_index.core.workflow import StartEvent, StopEvent, Event, Workflow, step
from uipath_llamaindex.llms import UiPathOpenAI

class QueryEvent(StartEvent):
    query: str

class ResultEvent(StopEvent):
    answer: str
    confidence: float

class AnalysisEvent(Event):
    analysis: str

class AnalysisAgent(Workflow):
    _llm: UiPathOpenAI | None = None

    @property
    def llm(self) -> UiPathOpenAI:
        if self._llm is None:
            self._llm = UiPathOpenAI()
        return self._llm

    @step
    async def analyze(self, ev: QueryEvent) -> AnalysisEvent:
        response = await self.llm.acomplete(
            f"Analyze this question and identify the key topic: {ev.query}"
        )
        return AnalysisEvent(analysis=str(response))

    @step
    async def answer(self, ev: AnalysisEvent) -> ResultEvent:
        response = await self.llm.acomplete(
            f"Based on this analysis: {ev.analysis}\nProvide a concise answer."
        )
        return ResultEvent(answer=str(response), confidence=0.85)

workflow = AnalysisAgent(timeout=60, verbose=False)
```

### FunctionAgent (Chat Agent with Tools)

For simpler agents that don't need custom workflow steps, use `FunctionAgent`:

```python
# main.py
from llama_index.core.agent.workflow import FunctionAgent
from llama_index.core.tools import FunctionTool
from uipath_llamaindex.llms import UiPathOpenAI

def search_knowledge_base(query: str) -> str:
    """Search the knowledge base for information."""
    return f"Results for: {query}"

def build_workflow() -> FunctionAgent:
    tools = [FunctionTool.from_defaults(fn=search_knowledge_base)]
    return FunctionAgent(
        tools=tools,
        llm=UiPathOpenAI(),
        system_prompt="You are a helpful assistant. Use tools to find information.",
    )
```

FunctionAgent uses a default input/output schema:
- Input: `{"user_msg": "your question"}`
- Output: agent's response string

---

## Next Steps

- **[Agent Patterns](agent-patterns.md)** — Architecture patterns with full code examples
- **[SDK Services](../capabilities/sdk-services.md)** — Use UiPath platform services in your workflow steps
- **[Tracing](../capabilities/tracing.md)** — Advanced tracing for helper functions outside the workflow
- **[Deployment](../lifecycle/deployment.md)** — Package and publish your LlamaIndex agent