PydanticAI example connecting to Sap Generative AI Hub via LiteLLM

How PydanticAI works

LLM access via LiteLLM Proxy

PydanticAI Agents supports the LiteLLM Proxy for access via OpenAI API calls.

Follow the details in LiteLLM Proxy setup for SAP Gen. AI Hub

Installation

%pip install pydantic-ai

Set env variables

Add the following variables from the service key in a file called “.env” and put it in the same folder where you run the notebook:

LITELLM_PROXY_API_KEY=sk-1234
PROXY_BASE_URL=http://localhost:4000

Run the PydanticAI with LiteLLM and SAP LLMs

import os

import litellm
from dotenv import load_dotenv
from pydantic_ai import Agent, RunContext
from pydantic_ai.models.openai import OpenAIChatModel
from pydantic_ai.providers.litellm import LiteLLMProvider

Load your credentials as environment variables.

litellm.use_litellm_proxy = True
load_dotenv()
api_base = os.getenv("PROXY_BASE_URL")
api_key = os.getenv("LITELLM_PROXY_API_KEY")

Define the model with OpenAIChatModel object with LiteLLMProvider object with your proxy credentials

model = OpenAIChatModel(
    "sap/gpt-5",
    provider=LiteLLMProvider(
        api_base=api_base,
        api_key=api_key,
    ),
)

Define the agent

agent = Agent(
    model=model,
    system_prompt="You are a helpful weather assistant. "
    "When the user asks about a specific city, "
    "use the 'get_weather' tool to find the weather information. "
    "Provide the TV weather report in two sentences including a small joke.",
)

Define the agent tool.

@agent.tool
def get_weather(city: RunContext[str]) -> str:
    """Mock function"""
    city_normalized = city.prompt.lower().replace(" ", "")

    mock_weather_db = {
        "newyork": "The weather in New York is sunny with a temperature of 25°C.",
        "london": "It's cloudy in London with a temperature of 15°C.",
        "tokyo": "Tokyo is experiencing light rain and a temperature of 18°C.",
    }

    if city_normalized in mock_weather_db:
        return mock_weather_db[city_normalized]
    else:
        return f"The weather in {city} is sunny with a temperature of 20°C."

Run agent

result = await agent.run("London")
print(result.output)