Agentic AI: from chatbots to tool-using agents

A chatbot answers. An agent acts. The line is the ability to call tools (functions, APIs, shells) and then react to whatever those tools returned — usually in a loop, with the model deciding when to stop.

Concretely: an agent is an LLM wrapped in a control flow that lets it (a) read its current state, (b) propose a next action, (c) execute that action, (d) observe the result, and (e) decide whether to go again. Everything else — memory, planning, multiple agents talking — is layered on top of that core loop.

Almost every agent framework boils down to the same loop:

1. Perceive: take the user's request + any prior context.
2. Think: the LLM emits either a tool call or a final answer.
3. Act: if it's a tool call, the runtime executes it.
4. Observe: feed the tool result back into context.
5. Go to 2.

The LLM is doing all the deciding. The runtime is just a switch: "is this output a tool call or a finished answer?" That's it. Everything fancy (memory, retries, planning) is bolted onto this skeleton.

Each iteration: the model picks a tool or returns. The runtime obediently routes.

An agent can only do what its tools let it do. Pick tools deliberately — each one is a verb in the agent's vocabulary.

• Read-only tools (search_docs, read_file, query_db) are cheap and safe to expose generously.
• Write tools (send_email, git_push, charge_card) need careful design: explicit confirmations, dry-run modes, scoped permissions.
• Compound tools (run_python) are the most powerful and the most dangerous — they essentially hand the model arbitrary code execution.

A common mistake is giving the agent 50 tools "just in case". The LLM's tool-selection accuracy drops as the menu grows. Aim for 5–15 high-signal tools.

Agents have three memory layers:

• Working memory = the current context window. Free, lossless, capped (e.g. 200K tokens). Resets every turn.
• Episodic memory = past interactions, stored somewhere external (DB, vector store) and retrieved on demand.
• Semantic memory = distilled facts the agent has learned, often summarised so they fit cheaply.

The craft is deciding what graduates from working → episodic → semantic. "Just stuff everything in the prompt" doesn't scale past a few sessions. "Store everything as an embedding" gets noisy fast.

Two dominant patterns:

• ReAct (Reason + Act): the model interleaves a thought, a tool call, and an observation each turn. Simple, fast to implement, and what most frameworks use by default.
• Plan-and-Execute: the model first emits a multi-step plan, then a separate executor runs each step. Better for long tasks where drift matters; worse when the plan is wrong (which is often).

ReAct is the default for a reason. Reach for Plan-and-Execute when steps are expensive (e.g. each one runs a 30-second SQL query) and you need to commit to a strategy upfront.

Sometimes one agent isn't enough — you split the work across specialists. A researcher agent searches the web, a writer agent drafts the report, a critic agent reviews it.

The upside: each agent has a tighter prompt and tool set, so they make fewer mistakes. The downside: coordination is now a problem too. Agents must agree on a shared scratchpad, handoff conditions, and a termination rule.

Rule of thumb: don't split into multiple agents until one agent has clearly hit its ceiling. Two mediocre agents are worse than one good one.

Things that break in real agents, ranked by how often they bite:

1. Infinite loops — the model retries the same failing tool call. Fix with a hard step limit (max_iterations=20) and a same-call detector.
2. Context exhaustion — the conversation grows past the window. Fix with compaction or sliding summaries.
3. Hallucinated tool calls — model invents a tool that doesn't exist. Fix with strict schema validation + a polite error message back into context.
4. Confident wrong answers — model declares done with the wrong result. Fix by adding a verification step or LLM-as-judge.
5. Cost runaway — each turn is one full model call. Set a token budget per session, not just per call.

Agents are slow and expensive. Each turn is a full model call, and tasks routinely take 5–30 turns. If your task can be solved by:

• a single prompt with a structured output schema, don't use an agent
• a deterministic script with one LLM step, don't use an agent
• a RAG pipeline that retrieves once then answers, don't use an agent

Use an agent when the path is genuinely uncertain: the model needs to look something up, decide what to do based on the result, and possibly try again. "Browse the docs and answer this specific question" is agent-shaped. "Summarise this PDF" is not.

Here's the entire pattern in Python with the Anthropic SDK. Real frameworks add error handling, parallel tool calls, and observability — but the skeleton is this small.

import anthropic
client = anthropic.Anthropic()
messages = [{"role": "user", "content": "..."}]
while True:
    r = client.messages.create(model="claude-opus-4-7", tools=TOOLS, messages=messages, max_tokens=4096)
    messages.append({"role": "assistant", "content": r.content})
    if r.stop_reason == "end_turn":
        break
    tool_results = [run_tool(b) for b in r.content if b.type == "tool_use"]
    messages.append({"role": "user", "content": tool_results})

Everything else is decoration.