LiteLLM Proxy example connecting to Sap Generative AI Hub

How to configure the proxy server

Follow the details in LiteLLM Proxy

Installation

%pip install “litellm[proxy]”

Config file for SAP models

Create the config.yaml in the directory where you run proxy and add those base settings. Get the service key from your SAP BTP tenant with AI subscription. And put the variables in the config file.

model_list:
  - model_name: "sap/*"
    litellm_params:
      model: "sap/*"



litellm_settings:
  drop_params: True

general_settings:
  master_key: sk-1234
  store_model_in_db: False


environment_variables:
   AICORE_AUTH_URL: "https://* * * .authentication.sap.hana.ondemand.com/oauth/token",
   AICORE_CLIENT_ID: " *** ",
   AICORE_CLIENT_SECRET: " *** ",
   AICORE_RESOURCE_GROUP: " *** ",
   AICORE_BASE_URL: "https://api.ai.***.cfapps.sap.hana.ondemand.com/

Run the proxy

%litellm --config ./config.yaml