LangChain: Building Your First LLM Application

Making a single API call to a Large Language Model (LLM) like GPT-4 is straightforward. However, building a real-world application involves more than one call. You need to manage prompts, handle chat history, connect to external data sources, and structure the model's output. Doing this from scratch for every project leads to a lot of repeated, boilerplate code.

LangChain solves this problem by providing a framework for developing LLM-powered applications. It's not just an API wrapper; it's a set of standard, composable building blocks (components) and a way to connect them (chains). Think of it like a web framework (e.g., Ruby on Rails, Django) but for AI. It provides high-level abstractions for common tasks, letting you focus on your application's logic instead of the plumbing.

LangChain is built around a few core concepts that you'll use constantly. Understanding them is key to using the framework effectively.

• Models: These are wrappers around the actual language models, like those from OpenAI, Anthropic, or Google. LangChain provides a standardized invoke() interface for all models, so you can often swap them out with minimal code changes.
• Prompts: LLMs are controlled by instructions, or prompts. Prompt templates allow you to create reusable, dynamic prompts by defining a structure with placeholders for user input or other information.
• Chains: The heart of LangChain. Chains are how you combine components. The most basic chain links a prompt template to a model. More complex chains can involve multiple models, data retrieval steps, and output formatting. The standard way to build chains is with the LangChain Expression Language (LCEL).

LangChain's ChatModel abstraction provides a standard way to interact with chat-based LLMs. The primary method is invoke(), which takes a list of Message objects (like SystemMessage or HumanMessage) and returns an AIMessage object containing the model's response.

Here's a simple example using OpenAI's API. Before running, make sure you have the necessary libraries (pip install langchain-openai) and your OPENAI_API_KEY is set as an environment variable.

from langchain_openai import ChatOpenAI
from langchain_core.messages import HumanMessage, SystemMessage

# Initialize the model
model = ChatOpenAI(model="gpt-4o-mini")

# Prepare the messages
messages = [
    SystemMessage(content="You are a helpful assistant that translates English to French."),
    HumanMessage(content="Hello, how are you?")
]

# Invoke the model
response = model.invoke(messages)

print(response.content)
# Expected output: "Bonjour, comment ça va ?"

SystemMessage sets the AI's behavior, while HumanMessage provides the user's query.

Hardcoding prompts directly in your application logic is inflexible. If you want to change the instructions or add more context, you have to dig into the code. Prompt templates solve this by separating the prompt's structure from the data it will contain.

A ChatPromptTemplate lets you define a sequence of messages with placeholders (wrapped in curly braces {}). You can then use the .invoke() method on the template, passing a dictionary to fill in these placeholders.

from langchain_core.prompts import ChatPromptTemplate

template = ChatPromptTemplate.from_messages([
    ("system", "You are a helpful assistant that translates {input_language} to {output_language}."),
    ("human", "{text}")
])

prompt_value = template.invoke({
    "input_language": "English",
    "output_language": "Spanish",
    "text": "I love programming."
})

# The result is a structured prompt object
print(prompt_value.to_messages())

This creates a complete, formatted prompt that is ready to be sent to a ChatModel.

At its core, a chain connects a prompt template to a model. The user's input fills the template, which generates a complete prompt. This prompt is then sent to the model, which produces the final output.

LangChain Expression Language (LCEL) is the declarative way to build chains. The key operator is the pipe (|), which connects components, feeding the output of the component on the left to the input of the component on the right. This makes composing chains intuitive and readable.

Let's combine the prompt template and model from the previous steps into our first chain.

from langchain_openai import ChatOpenAI
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.output_parsers import StrOutputParser

# 1. Define Prompt Template
template = ChatPromptTemplate.from_template(
    "Tell me a short, family-friendly joke about {topic}."
)

# 2. Define Model
model = ChatOpenAI(model="gpt-4o-mini")

# 3. Create the chain using the pipe operator
chain = template | model | StrOutputParser()

# 4. Invoke the chain with the input variable
response = chain.invoke({"topic": "computers"})

print(response)
#-> Why was the computer cold? Because it left its Windows open!

Notice we added StrOutputParser(), which is a simple parser that just extracts the string content from the model's AIMessage output.

LLMs produce text, but applications often need structured data, like JSON. Output Parsers are components that parse the model's raw output into a more usable format.

The JsonOutputParser is particularly useful. To use it effectively, you must do two things:

1. Instruct the model in the prompt to format its output as JSON.
2. Pipe the model's output to an instance of JsonOutputParser.

You can even pass a pydantic model to the parser to specify the exact JSON schema you expect.

from langchain_openai import ChatOpenAI
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.output_parsers import JsonOutputParser
from langchain_core.pydantic_v1 import BaseModel, Field

class Joke(BaseModel):
    setup: str = Field(description="the setup of the joke")
    punchline: str = Field(description="the punchline of the joke")

parser = JsonOutputParser(pydantic_object=Joke)

prompt = ChatPromptTemplate.from_template(
    "Tell me a joke about {topic}.\n{format_instructions}"
)

model = ChatOpenAI(model="gpt-4o-mini")

chain = prompt | model | parser

result = chain.invoke({
    "topic": "cats",
    "format_instructions": parser.get_format_instructions()
})

print(result)
#-> {'setup': 'Why are cats so good at video games?', 'punchline': 'Because they have nine lives!'}

The parser.get_format_instructions() method provides the model with the schema it needs to follow.

By adding an output parser, we can transform the model's raw text output into a more useful, structured format like JSON, which is easier for our application to work with programmatically.

LLMs have a knowledge cut-off and don't know about your private data. Retrieval Augmented Generation (RAG) is a powerful pattern that addresses this. Instead of just asking the model a question directly, you first retrieve relevant information from your own documents and then provide that information to the model as context.

The typical RAG flow is:

1. Retrieve: Use the user's query to search a database of your documents (often a vector store) and find the most relevant chunks of text.
2. Augment: Add these retrieved chunks of text into the prompt, along with the original user query.
3. Generate: Pass the augmented prompt to the LLM and ask it to generate an answer based only on the provided context.

LangChain provides all the components needed to build a RAG chain, including document loaders, text splitters, embedding models, and retrievers. This pattern is fundamental to building chatbots that can answer questions about specific, up-to-date information.

LangChain is more than just a library; it's an ecosystem. As you build more complex applications, two other tools become essential:

• LangSmith: A platform for debugging, tracing, and monitoring your LLM applications. When a chain doesn't work as expected, LangSmith gives you a detailed, step-by-step view of every component's inputs and outputs. This is invaluable for identifying the source of errors or unexpected behavior.

• LangServe: A library for deploying your LangChain chains as a production-ready REST API. With just a few lines of code, you can take any chain you've built and expose it as a web service with features like automatic type validation and asynchronous streaming.

Together, the core library, LangSmith, and LangServe provide a complete toolkit for building, debugging, and deploying LLM-powered applications.