Welcome to the third part of my introduction series on LangChain. You can start with the first article here: Introduction to LangChain: A Framework for LLM Powered Applications

LangChain uses large language models as the building blocks of natural language applications. But the question is, how do you program these models to do precisely what you want? How do you provide the right input and ensure you get the right output?

This is where LangChain Prompts come in. LangChain Prompts are a powerful and easy way to construct inputs for language models. They allow you to specify what you want the model to do, how you want it to do it, and what you want it to return.

In this post, I will show you how to use LangChain Prompts to program language models for various use cases. We will cover the main features of LangChain Prompts, such as LLM Prompt Templates, Chat Prompt Templates, Example Selectors, and Output Parsers. I will also provide some examples and code snippets to help you get started.

Prompt Templates

A Prompt Template enables you to reuse prompts while adding the ability to insert dynamic content.

With prompts you can do much more than just ask a question. You can prime the model with how to answer by giving instructions (”Act like a marketing consultant”) or providing examples to help the model understand how to answer.

A Prompt Template consists of two parts: input variables and a template

• Input variables are a set of variables that represent the information you want to include in the prompt.
• A template is a string that provides the text of the prompt and includes any of the variables you want to be included

To create a Prompt Template using LangChain, you just need to import the PromptTemplate class and pass the input variables and template as arguments. For example:

from langchain import PromptTemplate

# Define the template
template = """
Act as a SEO expect.
Provide an SEO optimized description for the following product: {product}
"""

# Create the prompt template
prompt = PromptTemplate(
	input_variables=["product"], 
	template=template
)

# pass in an input to return a formatted prompt
prompt.format(product="Electric Scooter")

# -> '\\nAct as a SEO expect.\\nProvide an SEO optimized description for the following product: Electric Scooter\\n'

You can see that the provided input has been dynamically inserted into the prompt.

Under the hood, LangChain uses the built-in string library’s Formatter class to parse the template text using the passed in variables. That’s why the variables in the template have the curly braces ({}) around them.

Here’s an example of what is happening to the template.

from string import Formatter

formatter = Formatter()
format_string = "Hello, {name}! You are {age} years old."
result = formatter.format(format_string, name="John", age=30)

print(result)  # Output: "Hello, John! You are 30 years old."

PromptTemplate defaults to treating the provided template string as a Python f-string, but you can specify other template formats with the template_format argument (currently only other supported format is jinja2)

# Make sure jinja2 is installed before running this

jinja2_template = "Tell me a {{ adjective }} joke about {{ content }}"
prompt_template = PromptTemplate.from_template(template=jinja2_template, template_format="jinja2")

prompt_template.format(adjective="funny", content="chickens")
# -> Tell me a funny joke about chickens.

Few Shot Prompt Templates

Besides the standard PromptTemplate LangChain also supports creating and storing FewShotPromptTemplate

Here is an example from the docs that shows how to construct one

from langchain import PromptTemplate, FewShotPromptTemplate

# First, create the list of few shot examples.
examples = [
    {"word": "happy", "antonym": "sad"},
    {"word": "tall", "antonym": "short"},
]

# Next, we specify the template to format the examples we have provided.
# We use the `PromptTemplate` class for this.
example_formatter_template = """
Word: {word}
Antonym: {antonym}\\n
"""
example_prompt = PromptTemplate(
    input_variables=["word", "antonym"],
    template=example_formatter_template,
)

# Finally, we create the `FewShotPromptTemplate` object.
few_shot_prompt = FewShotPromptTemplate(
    # These are the examples we want to insert into the prompt.
    examples=examples,
    
		# This is how we want to format the examples when we insert them into the prompt.
    example_prompt=example_prompt,
    
		# The prefix is some text that goes before the examples in the prompt.
    # Usually, this consists of intructions.
    prefix="Give the antonym of every input",
    
		# The suffix is some text that goes after the examples in the prompt.
    # Usually, this is where the user input will go
    suffix="Word: {input}\\nAntonym:",
    
		# The input variables are the variables that the overall prompt expects.
    input_variables=["input"],
    
		# The example_separator is the string we will use to join the prefix, examples, and suffix together with.
    example_separator="\\n\\n",
)

# We can now generate a prompt using the `format` method.
print(few_shot_prompt.format(input="big"))

# -> Give the antonym of every input
# -> 
# -> Word: happy
# -> Antonym: sad
# ->
# -> Word: tall
# -> Antonym: short
# ->
# -> Word: big
# -> Antonym:

Partial Prompt Templates

Sometimes when constructing a prompt from a template you will already have a value you want to pass in, or perhaps you have a dynamic value (such as the current date) you want to insert as a variable value into the prompt template.

LangChain solves this through “partial” prompt templates. When constructing a prompt template you can proactively pass in some of the variable values before you have access to all of them.

from langchain.prompts import PromptTemplate

partial_prompt = PromptTemplate(template="{foo}{bar}", input_variables=["foo", "bar"], partial_variables={"foo": "foo"))

print(partial_prompt.format(bar="baz"))

# -> foobaz

Using partial functions

You can also use partials with a function to fetch a variable value. For example, you might need to grab some user data or get a date based on the current date.

from datetime import datetime
from dateutil.relativedelta import relativedelta

def date_100_years_ago():
    # Get the current date
    today = datetime.date.today()
    # Subtract 100 years from the current date using relativedelta
    years_ago = today - relativedelta(years=100)
    # Return the date 100 years ago as a string
    return years_ago.strftime("%Y-%m-%d")

prompt = PromptTemplate(
    template="Tell me a {adjective} fact about the day {date}", 
    input_variables=["adjective", "date"],
    partial_variables={“date": date_100_years_ago} # <— use function name here
);

print(partial_prompt.format(adjective="interesting"))
# -> Tell me a interesting fact about the day 1923-05-10

Storing (Serializing) Prompt Templates in Separate Files

It is not only possible, but often preferable, to save prompts as separate files rather than as Python code. This method of storing your prompts simplifies the process of managing, sharing, and versioning your prompts, apart from your code.

LangChain provides a prompt “loader” for loading prompts, not just from local files, but also from the LangChain Hub, which is meant to be a public collection of prompts available to use. Currently json and yaml files are supported.

Let's take a look at an example. We are using the same template as before, which was initially coded directly in the Python file. However, this time, we've saved it in the adjective_joke_prompt.json file

# load the prompt using a file
prompt_template = load_prompt("adjective_joke_prompt.json")

# create a prompt using the variables
prompt_template.format(adjective="funny", content="chickens")
# -> Tell me a funny joke about chickens.

Here is the structure of a prompt template file (using JSON):

{
	# what type of prompt. Currently supports "prompt" and "few_shot"
	"_type": "prompt",

	# the input variables used in the template
	"input_variables": ["adjective", "content"],
	
	# the template text of the prompt, including variable placeholders
	"template": "Tell me a {{ adjective }} joke about {{ content }}",
	# alternatively the template text can be loaded from a file
	"template_path": "adjective_joke_prompt_template.txt"
	# NOTE: both "template" and "template_path" cannot be used at the same time!
	
	# the format of the template 
	"template_format": "jinja2",
	
	# currently only the "RegexParser" is supported for "output_parser"
	# this is example of a date parser
	"output_parser": {
		"_type": "regex_parser",
		"regex": "(\\d{4})-(\\d{2})-(\\d{2})",
		"output_keys": ["year", "month", "day"]
	}
	
}
	

And here’s an example of the FewShotPromptTemplate template file

{
    "_type": "few_shot",
    "input_variables": ["adjective"],
    "prefix": "Write antonyms for the following words.",
    "example_prompt": {
        "_type": "prompt",
        "input_variables": ["input", "output"],
        "template": "Input: {input}\\nOutput: {output}"
    },
    "examples": [
        {"input": "happy", "output": "sad"},
        {"input": "tall", "output": "short"}
    ],
    "suffix": "Input: {adjective}\\nOutput:"
}

And here’s the same example but loading the example prompt and examples from a file

{
    "_type": "few_shot",
    "input_variables": ["adjective"],
    "prefix": "Write antonyms for the following words.",
    "example_prompt_path": "example_prompt.json",
    "examples": "examples.json",
    "suffix": "Input: {adjective}\\nOutput:"
}

Keep in mind, the examples field can either be a dictionary or a filepath. This is a bit different from how other fields are modified to {field}_path to load from an external file. I've submitted a PR to address this inconsistency in the LangChain Github repo. Your support can help get this update implemented, so please consider commenting!

To load from the LangChain Hub, you need to specify a “filepath” that aligns with the hub repository's folder structure. Use lc:// as the file prefix. The loader will identify it as a LangChain hub reference and automatically fetch the example from the repository in the background.

from langchain.prompts import load_prompt

prompt = load_prompt("lc://prompts/conversation/prompt.json")
prompt.format(history="", input="What is 1 + 1?")

Benefits of using Prompt Templates:

• They make your prompts modular and reusable. You can use the same Prompt Template for different inputs and outputs without changing the code.
• They make your prompts readable and understandable. You can easily see what information is included in the prompt and how it is formatted.
• They make your prompts flexible and customizable. You can change any part of the Prompt Template according to your needs and preferences.

Chat Prompt Templates

I covered Chat Prompt Templates some in my last article but here’s a recap.

With a Chat Model you have three types of messages:

1. SystemMessage - This sets the behavior and objectives of the LLM. You would give specific instructions here like, “Act like a Marketing Manager.” or “Return only a JSON response and no explanation text”
2. HumanMessage - This is where you would input the user’s prompts to be sent to the LLM
3. AIMessage - This is where you store the responses from the LLM when passing back the chat history to the LLM in future requests

There is also a generic ChatMessage that takes an arbitrary “role” input that can be used in a situation that requires something other than System/Human/AI. But in general, you’ll use the three types above.

You construct these different message types into a MessagePromptTemplate and then can build a ChatPromptTemplatefrom all the messages.

Here’s a code example

from langchain.prompts.chat import (
    ChatPromptTemplate,
    SystemMessagePromptTemplate,
    HumanMessagePromptTemplate,
)
...

system_template="You are a helpful assistant that translates {input_language} to {output_language}."
system_message_prompt = SystemMessagePromptTemplate.from_template(system_template)

# SystemMessagePromptTemplate(prompt=PromptTemplate(input_variables=['input_language', 'output_language'], output_parser=None, partial_variables={}, template='You are a helpful assistant that translates {input_language} to {output_language}.', template_format='f-string', validate_template=True), additional_kwargs={})

human_template="{text}"
human_message_prompt = HumanMessagePromptTemplate.from_template(human_template)

# HumanMessagePromptTemplate(prompt=PromptTemplate(input_variables=['text'], output_parser=None, partial_variables={}, template='{text}', template_format='f-string', validate_template=True), additional_kwargs={})

chat_prompt = ChatPromptTemplate.from_messages([system_message_prompt, human_message_prompt])

# ChatPromptTemplate(input_variables=['output_language', 'input_language', 'text'], output_parser=None, partial_variables={}, messages=[SystemMessagePromptTemplate(prompt=PromptTemplate(input_variables=['input_language', 'output_language'], output_parser=None, partial_variables={}, template='You are a helpful assistant that translates {input_language} to {output_language}.', template_format='f-string', validate_template=True), additional_kwargs={}), HumanMessagePromptTemplate(prompt=PromptTemplate(input_variables=['text'], output_parser=None, partial_variables={}, template='{text}', template_format='f-string', validate_template=True), additional_kwargs={})])

# get a chat completion from the formatted messages
chat(chat_prompt.format_prompt(input_language="English", output_language="French", text="I love programming.").to_messages())

# AIMessage(content="J'adore la programmation.", additional_kwargs={})

Example Selectors

Example Selectors are functions designed to dynamically select relevant examples for your prompts. Examples are pieces of text that illustrate how to perform a task or achieve a goal using natural language. For example, if you want to summarize an article, an example might be a short summary of another article on a similar topic.

Example Selectors allow you to:

• Determine where to get examples from. You can use any source of text data, such as files, databases, websites, or APIs.
• Define how many examples to get. You can choose how many examples you want to include in your prompt.
• Establish filter criteria. You can apply any criteria or conditions to select only the examples that match your requirements.
• Rank examples based on a specific metric. You can sort the examples by any metric or score that reflects their relevance or quality.

LangChain currently provides the following pre-built example selector tools:

To create a custom example selector in LangChain, you define a class that inherits from the BaseExampleSelector and implements at least an add_example method and a select_examples method.

This example selects 2 random examples and then shows how to use with a FewShotPromptTemplate:

from langchain.prompts.example_selector.base import BaseExampleSelector
from typing import Dict, List
import numpy as np

class CustomExampleSelector(BaseExampleSelector):
    
    def __init__(self, examples: List[Dict[str, str]]):
        self.examples = examples
    
    def add_example(self, example: Dict[str, str]) -> None:
        """Add new example to store for a key."""
        self.examples.append(example)

    def select_examples(self, input_variables: Dict[str, str]) -> List[dict]:
        """Select which examples to use based on the inputs."""
        return np.random.choice(self.examples, size=2, replace=False)

examples = [
    {"input": "happy", "output": "sad"},
    {"input": "tall", "output": "short"},
    {"input": "energetic", "output": "lethargic"},
    {"input": "sunny", "output": "gloomy"},
]

# Initialize example selector.
example_selector = CustomExampleSelector(examples)

# Add new example to the set of examples
example_selector.add_example({"input": "windy", "output": "calm"},)
example_selector.examples
# -> [{"input": "happy", "output": "sad"}, "input": "tall", "output": "short"}, {"input": "energetic", "output": "lethargic"}, {"input": "sunny", "output": "gloomy"}, {"input": "windy", "output": "calm"}]

# Use in a FewShotPromptTemplate
similar_prompt = FewShotPromptTemplate(
    # We provide an ExampleSelector instead of examples.
    example_selector=example_selector,
    example_prompt=example_prompt,
    prefix="Give the antonym of every input",
    suffix="Input: {adjective}\\nOutput:", 
    input_variables=["adjective"],
)

Benefits of using Example Selectors

• They improve the quality and efficiency of your prompts. You can include different examples that show the variety and richness of natural language.
• They make your prompts quality and efficient. You can select only the best examples that are relevant and useful for your task.
• They make your prompts dynamic and adaptive. You can update your examples based on new data or feedback.

Providing examples to LLMs can dramatically enhance the quality of the output. It’s a much more cost-effective and effective method to get specific outputs as opposed to fine-tuning a model which can be up to six-times more expensive and requires extensive time for proper tuning and high-quality training data.

Output Parsers

Language models will always output raw text, but we often need that text put into a structure we can use in our application. Output Parsers are classes that help you parse and structure the output of language models. Output Parsers allow you to:

• Define the desired format of the output. You can use any format that suits your needs, such as JSON, XML, CSV, or plain text.
• Determine the validation rules for the output. You can apply any rules or constraints to check if the output is correct and complete.
• Set up error handling procedures. You can define what to do if the output is invalid or incomplete, such as retrying, skipping, or raising an exception.

LangChain provides documentation on the following pre-built Output Parsers

To create a custom Output Parser, you need to inherit from the BaseOutputParser class and implement, at a minimum, the following two methods::

• get_format_instructions() -> str: A method that returns a string containing instructions that will be passed in the prompt to instruct the language model on how to format the output text.
• parse(str) -> Any: A method that takes in a string (assumed to be the response from a language model) and parses it into some structure.

The CommaSeparatedListOutputParser code demonstrates how it should look like

from __future__ import annotations
from abc import abstractmethod
from typing import List
from langchain.schema import BaseOutputParser

class ListOutputParser(BaseOutputParser):
    """Class to parse the output of an LLM call to a list."""
    @property
    def _type(self) -> str:
        return "list"

    @abstractmethod
    def parse(self, text: str) -> List[str]:
        """Parse the output of an LLM call."""

class CommaSeparatedListOutputParser(ListOutputParser):
    """Parse out comma separated lists."""
    def get_format_instructions(self) -> str:
        return (
            "Your response should be a list of comma separated values, "
            "eg: `foo, bar, baz`"
        )

    def parse(self, text: str) -> List[str]:
        """Parse the output of an LLM call."""
        return text.strip().split(", ")

And how it is used in practice

from langchain.output_parsers import CommaSeparatedListOutputParser
from langchain.prompts import PromptTemplate, ChatPromptTemplate, HumanMessagePromptTemplate
from langchain.llms import OpenAI
from langchain.chat_models import ChatOpenAI

output_parser = CommaSeparatedListOutputParser()
format_instructions = output_parser.get_format_instructions()

prompt = PromptTemplate(
    template="List five {subject}.\\n{format_instructions}",
    input_variables=["subject"],
    partial_variables={"format_instructions": format_instructions}
)

model = OpenAI(temperature=0)
_input = prompt.format(subject="ice cream flavors")

output = model(_input)

output_parser.parse(output)
# -> ['Vanilla', 'Chocolate', 'Strawberry', 'Mint Chocolate Chip', 'Cookies and Cream']

The benefits of using Output Parsers are:

• They provide structure and consistency to your outputs. You can get the output in a format that is easy to store and process.
• They enhance the accuracy and robustness of your outputs. You can ensure the output is correct and complete according to your specifications.
• They offer flexibility and customization for your outputs. You can modify any part of the Output Parser according to your needs and preferences.

Wrapping Up

In this post, I have shown you how to use LangChain Prompts to program language models and chat models for various use cases. We have covered the main features of LangChain Prompts, including Prompt Templates, Example Selectors, and Output Parsers. I have also provided some examples and code snippets to help you get started.

Prompts really are the “meat” of an LLM application so understanding how to construct them will be crucial to creating the most effective outputs. I encourage you to use all the tools LangChain provides to not only construct prompts but also manage errors and parse the output.

As always, if you have comments or questions, please pop them into the comments below.

To keep up with future posts and support my work, be sure to:

• Subscribe to the site
• Follow me on Twitter or LinkedIn
• Share this article