AI Agents, From Chatbot to Production

An AI agent is the LLM application taken further — something that can plan on its own, carry out tasks, and interact with its environment.

Foundational papers:

• Yao, S., et al. (2022). “ReAct: Synergizing Reasoning and Acting in Language Models.” ICLR. [paper] — the ReAct framework, the core pattern behind most agents
• Wei, J., et al. (2022). “Chain-of-Thought Prompting Elicits Reasoning in Large Language Models.” NeurIPS. [paper]
• Wang, G., et al. (2024). “A Survey on Large Language Model based Autonomous Agents.” arXiv. [paper] — a survey of the agent landscape

1. The Overall Progression

Chatbot → Tool Use → Agent → Production Agent

What each stage means:

• Chatbot: can only talk, can’t take any action in the outside world
• Tool Use: can call a single tool
• Agent: can plan and carry out multi-step tasks on its own
• Production Agent: the industrial-strength version — error handling, permission control, monitoring, all of it

2. Breaking Down the Core Concepts

2.1 Chatbot

LLM + Prompt

What defines it:

• can only converse
• can’t take any external action
• no state, or only weak state

Foundational paper:

• Vinyals, O., & Le, Q. (2015). “A Neural Conversational Model.” ICML Deep Learning Workshop. [paper]

2.2 Function / Tool / Function Calling

Function

The smallest unit of code that actually does something.

Tool

The LLM-facing description of a function — what it’s called, what it expects, what it does.

Foundational papers:

• Schick, T., et al. (2023). “Toolformer: Language Models Can Teach Themselves to Use Tools.” NeurIPS. [paper] — the foundational work on tool learning
• Parisi, A., et al. (2022). “TALM: Tool Augmented Language Models.” arXiv. [paper]

Function calling

The LLM produces a structured call request, and the runtime executes it.

How it’s implemented in practice:

• OpenAI’s Function Calling API
• Anthropic’s Tool Use
• LangChain Tools

3. What an Agent Actually Is

LLM + Tools + a Loop

Foundational papers:

• Yao, S., et al. (2022). “ReAct: Synergizing Reasoning and Acting in Language Models.” ICLR. [paper]
• Shinn, N., et al. (2023). “Reflexion: Language Agents with Verbal Reinforcement Learning.” NeurIPS. [paper]

The Agent loop

Thought → Action → Observation → Thought...

A worked example from the ReAct paper:

Question: What is the elevation range for the area that the eastern sector of the Colorado orogeny extends into?

Thought 1: I need to search Colorado orogeny, find the area that the eastern sector of the Colorado orogeny extends into, then find the elevation range of that area.
Action 1: Search[Colorado orogeny]
Observation 1: The Colorado orogeny was an episode of mountain building...
Thought 2: I need to search eastern sector of the Colorado orogeny.
Action 2: Search[eastern sector of Colorado orogeny]
...

4. Agent Runtime

The Agent Runtime is the infrastructure that actually executes an agent’s logic.

What it’s responsible for:

• parsing tool calls
• executing the underlying function/API
• controlling the loop (max iterations, timeouts)
• error handling
• permission control
• state management

Common frameworks:

• LangChain
• AutoGPT
• BabyAGI
• CrewAI
• Microsoft AutoGen

References:

• Richards, T. (2023). “AutoGPT.” [GitHub]
• Nakajima, Y. (2023). “BabyAGI.” [GitHub]

5. Tool Retrieval

Using embeddings to semantically match a user’s request against a large tool library, and passing the LLM only the Top-K most relevant tools.

Foundational paper:

• Qin, Y., et al. (2023). “Tool Learning with Foundation Models.” arXiv. [paper] — a survey of tool learning

Flow:

User question → embedding → search the tool library → Top-K tools → LLM picks one to call

6. Skill

A Skill = a composed capability made up of several Tools

A Skill packages up the ability to handle a complex task — think of it as the AI equivalent of a human “skill.”

Examples:

• A “write a blog post” Skill = search tool + writing tool + image generation tool
• A “data analysis” Skill = data query tool + visualization tool + report generation tool

7. Production Agent Architecture

User → Retriever → LLM → Runtime → Tool → Observation → LLM → Response

What production actually requires:

• error handling and retries
• monitoring and logging
• permission control
• cost control
• security auditing

8. Principles Worth Remembering

• The model sets the ceiling; the runtime sets the floor.

  What the model can do defines the agent’s upper bound. How solid the runtime is defines how stable and reliable the agent actually is day to day.

• Retrieve before you reason.

  Pull in the relevant information and tools before trying to execute the task.

• Use Skills to package up complexity.

  Wrap frequently-used, complex tasks into a Skill — it makes things more reusable and more reliable.

9. A Mental Model

10. Wrapping Up

Agent = LLM + Tools + Loop + Runtime + Retrieval (+ Skills)

An AI agent is the LLM application pushed a step further — from a system that just talks, to one that can plan and carry out complex tasks on its own. As the underlying tech keeps improving, agents are going to keep showing up in more and more places.

References

Prompt Engineering

1. Brown, T. B., et al. (2020). “Language Models are Few-Shot Learners.” NeurIPS.
2. Wei, J., et al. (2022). “Chain-of-Thought Prompting Elicits Reasoning in Large Language Models.” NeurIPS.
3. Kojima, T., et al. (2022). “Large Language Models are Zero-Shot Reasoners.” NeurIPS.
4. Zhou, Y., et al. (2022). “Large Language Models Are Human-Level Prompt Engineers.” ICLR.

RAG

1. Lewis, P., et al. (2020). “Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks.” NeurIPS.
2. Guu, K., et al. (2020). “REALM: Retrieval-Augmented Language Model Pre-Training.” ICML.
3. Karpukhin, V., et al. (2020). “Dense Passage Retrieval for Open-Domain Question Answering.” EMNLP.
4. Johnson, J., Douze, M., & Jégou, H. (2019). “Billion-scale similarity search with GPUs.” IEEE TBD.

AI Agent

1. Yao, S., et al. (2022). “ReAct: Synergizing Reasoning and Acting in Language Models.” ICLR.
2. Schick, T., et al. (2023). “Toolformer: Language Models Can Teach Themselves to Use Tools.” NeurIPS.
3. Shinn, N., et al. (2023). “Reflexion: Language Agents with Verbal Reinforcement Learning.” NeurIPS.
4. Wang, G., et al. (2024). “A Survey on Large Language Model based Autonomous Agents.” arXiv.