n8n AI Agent Node Deep Dive: Routing Workflows with Local LLMs

n8n's AI agent node is one of the most powerful tools in the self-hosted automation stack. It lets you drop an intelligent decision layer into any workflow, powered by the LLM of your choice — including local models running on your own hardware.

What the AI agent node does

The AI agent node in n8n acts as a smart router. Instead of hard-coding conditional branches, you give the agent a prompt and let it decide: which path to follow, how to classify inbound data, whether to transform content, or when to hand off to a human.

This is useful for:

• **Classifying incoming support tickets** by urgency, topic, or customer tier
• **Extracting structured data** from unstructured text like emails or notes
• **Routing documents** to the right team or process based on content
• **Deciding whether to automate or escalate** when confidence is low

If you are new to n8n itself, start with Build Your Own AI Assistant with n8n for the fundamentals.

Connect a local model to the agent node

The AI agent node works with any OpenAI-compatible API endpoint. That means Ollama, LM Studio, or TabbyAPI all work out of the box:

```bash

Ensure Ollama is running with your chosen model

ollama pull llama3.2:3b

ollama serve

```

In n8n, create a new **AI Agent** node and set:

• **Connection** → OpenAI-compatible API
• **Base URL** → http://localhost:11434/v1
• **Model** → llama3.2:3b (or your preferred model)

Choosing the right local model for agentic routing

| Model | Parameters | Best for | VRAM |

|-------|-----------|----------|------|

| Llama 3.2 3B | 3B | Fast classification, simple routing | 4 GB |

| Qwen 2.5 7B | 7B | Complex decisions, multi-step reasoning | 8 GB |

| Mistral 7B | 7B | Structured output, JSON extraction | 8 GB |

| Llama 3.1 8B | 8B | Balanced quality and speed | 8 GB |

For hardware sizing guidance, read Best Hardware for Self-Hosted AI before scaling up.

Build a multi-step routing pipeline

A typical agent-routing workflow looks like this:

1. **Trigger** — webhook, email, or scheduled timer

2. **Process** — extract or transform the inbound payload

3. **AI Agent** — classify the intent and route accordingly

4. **Branch** — send to different sub-workflows based on the agent's decision

5. **Output** — notification, database write, API call, or human review queue

Example: Classify inbound support emails

Set the AI agent's system prompt to something like:

> You are a support ticket classifier. Given an email subject and body, classify it as one of: billing, technical, feature_request, or other. Return ONLY the category name as a single word.

Then use a **Switch** node to route each category to the appropriate handling workflow.

Prompt patterns for reliable agent decisions

Local models are capable, but they need tighter instruction formats. A few patterns that work well:

• **Output constraints** — specify exact format (JSON, single word, numbered list)
• **Few-shot examples** — give two or three clear before-and-after pairs
• **Confidence thresholds** — ask the agent to include a confidence score and define a cutoff for escalation

For more on prompt design, see Prompt Tuning for Local LLMs Without Overcomplicating Things.

Troubleshooting common issues

| Problem | Likely cause | Fix |

|---------|-------------|-----|

| Agent returns nothing | Model not responding | Check Ollama status and endpoint URL |

| Inconsistent classifications | Model too small for the task | Use a 7B+ model |

| Slow decisions | CPU-only inference | Add GPU or use a smaller quantised model |

| JSON output broken | Prompt unclear | Add explicit output format instructions |

Conclusion

n8n's AI agent node bridges the gap between traditional automation rules and intelligent decision-making. With local models, you get smart routing without sending business data to a third party. Start with one classification workflow, test thoroughly, then expand to more complex routing scenarios.

FAQ

Can the AI agent node use multiple models?

Yes. You can create separate agent nodes, each configured with a different model, and route between them based on task complexity.

Is the AI agent node slower than a regular switch node?

Yes — LLM calls add latency. Use regular switch nodes for deterministic rules and AI agent nodes only where classification or judgment is genuinely needed.

What happens if the model is down?

The node will error. Add error handling with n8n's Error Trigger or a fallback branch to a human queue.