RAG Fundamentals with n8n
Retrieve, augment, and generate correctly using n8n. Chunking, embeddings, and reranking.
Key takeaways
- To retrieve, augment, and generate correctly in n8n, split the workflow into: retrieve, decide, act, and evaluate.
- Structure your prompts with a system frame, task frame, and output frame — every time.
- Validate every LLM output against a JSON schema before it enters your business logic.
- Log tokens, latency, and cost per execution so you can optimize what matters.
AI workflows in n8n go beyond calling a single LLM. To ship reliably you need to retrieve, augment, and generate correctly — and this guide covers exactly how, with chunking, embeddings, and reranking that hold up in production. Expect concrete node choices, prompt templates, and evaluation loops, not marketing copy.
Architecture
A production AI workflow in n8n has four layers: input normalization, retrieval, model call, and post-processing. Each layer is a sub-workflow you can test in isolation. To retrieve, augment, and generate correctly, wire the retrieval layer to a vector store (Pinecone, Qdrant, or PGVector) and the model layer to a provider you can swap.
Keep prompts in files or a database, not hard-coded in a Set node. That way you can A/B-test prompts without editing the workflow.
- Use the AI Agent node for tool use; use plain LLM nodes for pure generation.
- Cache identical prompts to cut cost by 30-60%.
- Route to a smaller model when the task is easy — save the big model for the hard ones.
Prompting and structured output
For retrieve, augment, and generate correctly, ask the model for JSON that fits a strict schema. Use OpenAI's json_schema mode, Claude's tool-use hack, or Gemini's response schema. Validate the parsed output before touching downstream systems; on failure, retry with a repair prompt.
Never trust a single generation. Add a critic step for high-stakes outputs and a human review queue for anything irreversible.
Evaluation
Ship an eval loop with every AI workflow. Keep a golden set of 20-100 inputs with expected outputs. Run the eval on every prompt or model change and gate deploys on it. When you retrieve, augment, and generate correctly, evals are the difference between a demo and a production system.
Track four numbers per version: pass rate, average latency, average cost, and hallucination rate. Regressions on any one are a blocker.
Frequently asked questions
- Which LLM should I use?
- Start with a mid-tier model (GPT-4o-mini, Claude Haiku, Gemini Flash) for iteration. Escalate to premium models only where evals prove it pays.
- How do I keep AI cost predictable?
- Cache prompts, cap max tokens, and route easy requests to smaller models. Log cost per workflow and alert on outliers.
- Do I need a vector database?
- Only for retrieval. PGVector inside Postgres is the cheapest starting point; move to Pinecone or Qdrant when you exceed a few million vectors.
- How do I test AI workflows?
- Golden sets + LLM-as-judge + human spot-check. Run on every deploy.