Integrate external APIs, databases, and services through function calling capabilities
Extend AI agents beyond text generation by integrating external APIs, databases, and services through function calling capabilities.
Simple Analogy: Think of an AI agent as a smart assistant who can read and write, but initially can't do much else. Tool use is like giving this assistant access to a phone, calculator, filing cabinet, and internet connection. Now they can call people, do math, look up files, and search for information!
Without tools, language models can only generate text based on their training. With tool use (also called function calling), agents can:
Define available tools with clear descriptions, parameters, and expected outputs. The agent needs to understand what each tool does and when to use it.
The agent analyzes the user's request and decides which tool(s) to use. Modern LLMs can reason about tool selection based on the task requirements.
The framework executes the selected tool with the appropriate parameters and returns the result to the agent.
The agent incorporates the tool's output into its response, providing the user with accurate, real-time information.
Tools that fetch real-time data from external sources like weather APIs, stock market data, news feeds, or web search engines.
Tools for performing calculations, statistical analysis, data processing, or code execution that require deterministic logic.
Tools for querying databases, accessing file systems, or retrieving information from structured data sources.
Tools for sending messages, triggering external actions, or controlling other systems and devices.
Modern LLMs use the tool descriptions, parameter specifications, and current context to decide which tool(s) to invoke. The selection process involves:
Vertex AI Extensions are structured API wrappers that enable models to connect with external APIs for real-time data processing and action execution. They offer enterprise-grade security, data privacy, and performance guarantees.
Extensions: Automatically executed by Vertex AI
Function Calling: Requires manual execution by user/client
Clear descriptions & parameters
Analyzes query & selects tools
Framework runs external function
Incorporates results into answer
