Parsing the LLM's output to call a function typically involves these steps, especially when using APIs designed for function calling or tool use:
-
Receive the LLM Response: Your application receives a response object (often JSON) from the LLM API.
-
Check for Tool Calls: The first crucial step is to determine if the LLM decided to call a tool. Many LLM APIs indicate this in the response. For example, OpenAI's API has a finish_reason field that might be set to "tool_calls". You'll need to check this field.
-
Access the tool_calls Array: If the finish_reason indicates tool calls, you'll then access the specific part of the response that contains the list of tool calls. In OpenAI's API, this is typically under a key like message['tool_calls'].
-
Iterate Through Tool Calls (if multiple): A single LLM response might contain instructions to call multiple tools. You'll likely iterate through each item in the tool_calls array.
-
Extract Function Name: For each tool call, extract the intended function's name. This is usually found under a key like function['name'].
-
Extract Function Arguments: Also extract the arguments the LLM determined for that function. These are typically found under function['arguments']. This is often a JSON string that needs to be parsed further.
-
Parse the Arguments String: Since the arguments are usually provided as a JSON string, you'll need to use a JSON parser (like Python's built-in json module) to convert this string into a Python dictionary or object.
- Example (Python):
import json
arguments_string = tool_call['function']['arguments']
try:
arguments_dict = json.loads(arguments_string)
except json.JSONDecodeError as e:
print(f"Error decoding arguments JSON: {e}")
# Handle the error, perhaps by logging or skipping this call
continue
-
Map Function Name to Actual Python Function: You'll need a way to map the function name provided by the LLM (e.g., "get_weather") to the actual Python function defined in your codebase (e.g., def get_weather(latitude: float, longitude: float): ...). A common way to do this is:
- A Dictionary: Create a dictionary where keys are the tool names (as strings) and values are the actual function objects.
available_tools = {
"get_weather": get_weather_function,
"schedule_event": schedule_event_function,
# ... other tools
}
- Conditional Logic: Use
if/elif/else statements based on the tool_name.
-
Call the Python Function with Parsed Arguments: Once you have the Python function object and the parsed arguments dictionary, you can call the function. You'll need to ensure the arguments from the dictionary match the expected parameters of your Python function. Python's **kwargs can be very helpful here.
- Example (Python):
tool_name = tool_call['function']['name']
if tool_name in available_tools:
function_to_call = available_tools[tool_name]
try:
# Safely call the function with arguments
tool_response = function_to_call(**arguments_dict)
# Now you have the result from your tool
except Exception as e:
print(f"Error calling function {tool_name}: {e}")
# Handle execution errors
else:
print(f"Unknown tool: {tool_name}")
# Handle cases where the LLM requested a tool your app doesn't recognize
-
Handle the Tool's Output: The tool_response you get back from calling your Python function is then typically sent back to the LLM in a subsequent API call to allow the LLM to formulate a final user-facing response.
This process ensures that the LLM's intent to use a tool is correctly interpreted by your application, and the appropriate Python function is invoked with the right parameters.
