A Job-Oriented Dialogue system (ToD) is a system that assists customers in reaching a selected activity, equivalent to reserving a restaurant, planning a journey itinerary or ordering supply meals.
We all know that we instruct LLMs utilizing prompts, however how can we implement these ToD programs in order that the dialog all the time revolves across the activity we wish the customers to realize? A technique of doing that’s through the use of prompts, reminiscence and software calling. FortunatelyLangChain + LangGraph may also help us tie all this stuff collectively.
On this article, you’ll learn to construct a Job Oriented Dialogue System that helps customers create Consumer Tales with a excessive degree of high quality. The system is all primarily based on LangGraph’s Prompt Generation from User Requirements tutorial.
On this tutorial we assume you already know the way to use LangChain. A Consumer Story has some parts like goal, success standards, plan of execution and deliverables. The person ought to present every of them, and we have to “maintain their hand” into offering them one after the other. Doing that utilizing solely LangChain would require lots of ifs and elses.
With LangGraph we will use a graph abstraction to create cycles to regulate the dialogue. It additionally has built-in persistence, so we don’t want to fret about actively monitoring the interactions that occur inside the graph.
The principle LangGraph abstraction is the StateGraph, which is used to create graph workflows. Every graph must be initialized with a state_schema: a schema class that every node of the graph makes use of to learn and write data.
The circulate of our system will encompass rounds of LLM and person messages. The principle loop will comprise these steps:
- Consumer says one thing
- LLM reads the messages of the state and decides if it’s able to create the Consumer Story or if the person ought to reply once more
Our system is straightforward so the schema consists solely of the messages that have been exchanged within the dialogue.
from langgraph.graph.message import add_messagesclass StateSchema(TypedDict):
messages: Annotated[list, add_messages]
The add_messages methodology is used to merge the output messages from every node into the present listing of messages within the graph’s state.
Talking about nodes, one other two important LangGraph ideas are Nodes and Edges. Every node of the graph runs a operate and every edge controls the circulate of 1 node to a different. We even have START and END digital nodes to inform the graph the place to start out the execution and the place the execution ought to finish.
To run the system we’ll use the .stream() methodology. After we construct the graph and compile it, every spherical of interplay will undergo the START till the END of the graph and the trail it takes (which nodes ought to run or not) is managed by our workflow mixed with the state of the graph. The next code has the principle circulate of our system:
config = {"configurable": {"thread_id": str(uuid.uuid4())}}whereas True:
person = enter("Consumer (q/Q to give up): ")
if person in {"q", "Q"}:
print("AI: Byebye")
break
output = None
for output in graph.stream(
{"messages": [HumanMessage(content=user)]}, config=config, stream_mode="updates"
):
last_message = subsequent(iter(output.values()))["messages"][-1]
last_message.pretty_print()
if output and "immediate" in output:
print("Accomplished!")
At every interplay (if the person didn’t kind “q” or “Q” to give up) we run graph.stream() passing the message of the person utilizing the “updates” stream_mode, which streams the updates of the state after every step of the graph (https://langchain-ai.github.io/langgraph/concepts/low_level/#stream-and-astream). We then get this final message from the state_schema and print it.
On this tutorial we’ll nonetheless learn to create the nodes and edges of the graph, however first let’s speak extra in regards to the structure of ToD programs basically and learn to implement one with LLMs, prompts and software calling.
The principle parts of a framework to construct Finish-to-Finish Job-Oriented Dialogue programs are [1]:
- Pure Language Understanding (NLU) for extracting the intent and key slots of customers
- Dialogue State Monitoring (DST) for tracing customers’ perception state given dialogue
- Dialogue Coverage Studying (DPL) to decide the subsequent step to take
- Pure Language Era (NLG) for producing dialogue system response
Through the use of LLMs, we will mix a few of these parts into just one. The NLP and the NLG parts are straightforward peasy to implement utilizing LLMs since understanding and producing dialogue responses are their specialty.
We are able to implement the Dialogue State Monitoring (DST) and the Dialogue Coverage Studying (DPL) through the use of LangChain’s SystemMessage to prime the AI habits and all the time go this message each time we work together with the LLM. The state of the dialogue also needs to all the time be handed to the LLM at each interplay with the mannequin. Because of this we are going to be sure the dialogue is all the time centered across the activity we wish the person to finish by all the time telling the LLM what the purpose of the dialogue is and the way it ought to behave. We’ll do this first through the use of a immediate:
prompt_system_task = """Your job is to collect data from the person in regards to the Consumer Story they should create.You must receive the next data from them:
- Goal: the purpose of the person story. ought to be concrete sufficient to be developed in 2 weeks.
- Success standards the sucess standards of the person story
- Plan_of_execution: the plan of execution of the initiative
- Deliverables: the deliverables of the initiative
In case you are not capable of discern this data, ask them to make clear! Don't try to wildly guess.
Every time the person responds to one of many standards, consider whether it is detailed sufficient to be a criterion of a Consumer Story. If not, ask questions to assist the person higher element the criterion.
Don't overwhelm the person with too many questions without delay; ask for the knowledge you want in a manner that they don't have to jot down a lot in every response.
All the time remind them that in the event that they have no idea the way to reply one thing, you possibly can assist them.
After you'll be able to discern all the knowledge, name the related software."""
After which appending this immediate everytime we ship a message to the LLM:
def domain_state_tracker(messages):
return [SystemMessage(content=prompt_system_task)] + messages
One other vital idea of our ToD system LLM implementation is software calling. Should you learn the final sentence of the prompt_system_task once more it says “After you’ll be able to discern all the knowledge, name the related software”. This fashion, we’re telling the LLM that when it decides that the person offered all of the Consumer Story parameters, it ought to name the software to create the Consumer Story. Our software for that might be created utilizing a Pydantic mannequin with the Consumer Story parameters.
Through the use of solely the immediate and power calling, we will management our ToD system. Lovely proper? Really we additionally want to make use of the state of the graph to make all this work. Let’s do it within the subsequent part, the place we’ll lastly construct the ToD system.
Alright, time to do some coding. First we’ll specify which LLM mannequin we’ll use, then set the immediate and bind the software to generate the Consumer Story:
import os
from dotenv import load_dotenv, find_dotenvfrom langchain_openai import AzureChatOpenAI
from langchain_core.pydantic_v1 import BaseModel
from typing import Listing, Literal, Annotated
_ = load_dotenv(find_dotenv()) # learn native .env file
llm = AzureChatOpenAI(azure_deployment=os.environ.get("AZURE_OPENAI_CHAT_DEPLOYMENT_NAME"),
openai_api_version="2023-09-01-preview",
openai_api_type="azure",
openai_api_key=os.environ.get('AZURE_OPENAI_API_KEY'),
azure_endpoint=os.environ.get('AZURE_OPENAI_ENDPOINT'),
temperature=0)
prompt_system_task = """Your job is to collect data from the person in regards to the Consumer Story they should create.
You must receive the next data from them:
- Goal: the purpose of the person story. ought to be concrete sufficient to be developed in 2 weeks.
- Success standards the sucess standards of the person story
- Plan_of_execution: the plan of execution of the initiative
In case you are not capable of discern this data, ask them to make clear! Don't try to wildly guess.
Every time the person responds to one of many standards, consider whether it is detailed sufficient to be a criterion of a Consumer Story. If not, ask questions to assist the person higher element the criterion.
Don't overwhelm the person with too many questions without delay; ask for the knowledge you want in a manner that they don't have to jot down a lot in every response.
All the time remind them that in the event that they have no idea the way to reply one thing, you possibly can assist them.
After you'll be able to discern all the knowledge, name the related software."""
class UserStoryCriteria(BaseModel):
"""Directions on the way to immediate the LLM."""
goal: str
success_criteria: str
plan_of_execution: str
llm_with_tool = llm.bind_tools([UserStoryCriteria])
As we have been speaking earlier, the state of our graph consists solely of the messages exchanged and a flag to know if the person story was created or not. Let’s create the graph first utilizing StateGraph and this schema:
from langgraph.graph import StateGraph, START, END
from langgraph.graph.message import add_messagesclass StateSchema(TypedDict):
messages: Annotated[list, add_messages]
created_user_story: bool
workflow = StateGraph(StateSchema)
The subsequent picture reveals the construction of the ultimate graph:
On the high now we have a talk_to_user node. This node can both:
- Finalize the dialogue (go to the finalize_dialogue node)
- Determine that it’s time to attend for the person enter (go to the END node)
For the reason that important loop runs perpetually (whereas True), each time the graph reaches the END node, it waits for the person enter once more. This may turn into extra clear after we create the loop.
Let’s create the nodes of the graph, beginning with the talk_to_user node. This node must hold observe of the duty (maintaing the principle immediate throughout all of the dialog) and in addition hold the message exchanges as a result of it’s the place the state of the dialogue is saved. This state additionally retains which parameters of the Consumer Story are already stuffed or not utilizing the messages. So this node ought to add the SystemMessage each time and append the brand new message from the LLM:
def domain_state_tracker(messages):
return [SystemMessage(content=prompt_system_task)] + messagesdef call_llm(state: StateSchema):
"""
talk_to_user node operate, provides the prompt_system_task to the messages,
calls the LLM and returns the response
"""
messages = domain_state_tracker(state["messages"])
response = llm_with_tool.invoke(messages)
return {"messages": [response]}
Now we will add the talk_to_user node to this graph. We’ll do this by giving it a reputation after which passing the operate we’ve created:
workflow.add_node("talk_to_user", call_llm)
This node ought to be the primary node to run within the graph, so let’s specify that with an edge:
workflow.add_edge(START, "talk_to_user")
Up to now the graph seems like this:
To manage the circulate of the graph, we’ll additionally use the message courses from LangChain. We’ve got 4 varieties of messages:
- SystemMessage: message for priming AI habits
- HumanMessage: message from a human
- AIMessage: the message returned from a chat mannequin as a response to a immediate
- ToolMessage: message containing the results of a software invocation, used for passing the results of executing a software again to a mannequin
We’ll use the kind of the final message of the graph state to regulate the circulate on the talk_to_user node. If the final message is an AIMessage and it has the tool_calls key, then we’ll go to the finalize_dialogue node as a result of it’s time to create the Consumer Story. In any other case, we must always go to the END node as a result of we’ll restart the loop because it’s time for the person to reply.
The finalize_dialogue node ought to construct the ToolMessage to go the end result to the mannequin. The tool_call_id area is used to affiliate the software name request with the software name response. Let’s create this node and add it to the graph:
def finalize_dialogue(state: StateSchema):
"""
Add a software message to the historical past so the graph can see that it`s time to create the person story
"""
return {
"messages": [
ToolMessage(
content="Prompt generated!",
tool_call_id=state["messages"][-1].tool_calls[0]["id"],
)
]
}workflow.add_node("finalize_dialogue", finalize_dialogue)
Now let’s create the final node, the create_user_story one. This node will name the LLM utilizing the immediate to create the Consumer Story and the knowledge that was gathered in the course of the dialog. If the mannequin determined that it was time to name the software then the values of the important thing tool_calls ought to have all the information to create the Consumer Story.
prompt_generate_user_story = """Based mostly on the next necessities, write a great person story:{reqs}"""
def build_prompt_to_generate_user_story(messages: listing):
tool_call = None
other_msgs = []
for m in messages:
if isinstance(m, AIMessage) and m.tool_calls: #tool_calls is from the OpenAI API
tool_call = m.tool_calls[0]["args"]
elif isinstance(m, ToolMessage):
proceed
elif tool_call shouldn't be None:
other_msgs.append(m)
return [SystemMessage(content=prompt_generate_user_story.format(reqs=tool_call))] + other_msgs
def call_model_to_generate_user_story(state):
messages = build_prompt_to_generate_user_story(state["messages"])
response = llm.invoke(messages)
return {"messages": [response]}
workflow.add_node("create_user_story", call_model_to_generate_user_story)
With all of the nodes are created, it’s time so as to add the edges. We’ll add a conditional edge to the talk_to_user node. Keep in mind that this node can both:
- Finalize the dialogue if it’s time to name the software (go to the finalize_dialogue node)
- Determine that we have to collect person enter (go to the END node)
Because of this we’ll solely examine if the final message is an AIMessage and has the tool_calls key; in any other case we must always go to the END node. Let’s create a operate to examine this and add it as an edge:
def define_next_action(state) -> Literal["finalize_dialogue", END]:
messages = state["messages"]if isinstance(messages[-1], AIMessage) and messages[-1].tool_calls:
return "finalize_dialogue"
else:
return END
workflow.add_conditional_edges("talk_to_user", define_next_action)
Now let’s add the opposite edges:
workflow.add_edge("finalize_dialogue", "create_user_story")
workflow.add_edge("create_user_story", END)
With that the graph workflow is completed. Time to compile the graph and create the loop to run it:
reminiscence = MemorySaver()
graph = workflow.compile(checkpointer=reminiscence)config = {"configurable": {"thread_id": str(uuid.uuid4())}}
whereas True:
person = enter("Consumer (q/Q to give up): ")
if person in {"q", "Q"}:
print("AI: Byebye")
break
output = None
for output in graph.stream(
{"messages": [HumanMessage(content=user)]}, config=config, stream_mode="updates"
):
last_message = subsequent(iter(output.values()))["messages"][-1]
last_message.pretty_print()
if output and "create_user_story" in output:
print("Consumer story created!")
Let’s lastly take a look at the system:
With LangGraph and LangChain we will construct programs that information customers by structured interactions lowering the complexity to create them through the use of the LLMs to assist us management the conditional logic.
With the mixture of prompts, reminiscence administration, and power calling we will create intuitive and efficient dialogue programs, opening new prospects for person interplay and activity automation.
I hope that this tutorial assist you to higher perceive the way to use LangGraph (I’ve spend a few days banging my head on the wall to grasp how all of the items of the library work collectively).
All of the code of this tutorial will be discovered right here: dmesquita/task_oriented_dialogue_system_langgraph (github.com)
Thanks for studying!
[1] Qin, Libo, et al. “Finish-to-end task-oriented dialogue: A survey of duties, strategies, and future instructions.” arXiv preprint arXiv:2311.09008 (2023).
[2] Immediate technology from person necessities. Out there at: https://langchain-ai.github.io/langgraph/tutorials/chatbots/information-gather-prompting