Graphic tutorial: Use LiteLLM to build a unified large model gateway in ten minutes图文教程:十分钟用 LiteLLM 搭一个统一大模型网关
Download the LiteLLM official source code package from this site. After installing it, you can use one address and a set of OpenAI format interfaces to simultaneously call hundreds of models such as DeepSeek, Tongyi Qianwen, and Gemini. You can also manage your budget and monitor usage.
下载本站的 LiteLLM 官方源码包,装好后用一个地址、一套 OpenAI 格式接口同时调用 DeepSeek、通义千问、Gemini 等上百种模型,还能管预算、看用量。

Step 1: Install LiteLLM第一步:安装 LiteLLM
Go to the "AI Tools" section of this site to download "LiteLLM v1.102.1 Official Source Code Package" (tar.gz, about 18.5MB).
The computer needs to install Python 3.10 or above first, and then execute in the directory where the file is located: pip install litellm-1.102.1.tar.gz
After installation, execute litellm --version. If the version number can be printed, it means the installation is successful.
到本站「AI工具」栏目下载《LiteLLM v1.102.1 官方源码包》(tar.gz,约 18.5MB)。
电脑需先装好 Python 3.10 以上版本,然后在文件所在目录执行:pip install litellm-1.102.1.tar.gz
装完执行 litellm --version,能打印版本号就说明安装成功。

Step 2: Configure the model key第二步:配置模型密钥
Set the service key you want to use as an environment variable, such as OPENAI_API_KEY, DEEPSEEK_API_KEY, GEMINI_API_KEY, etc.
Windows users can add it in "System Properties → Environment Variables"; Mac/Linux users write ~/.bashrc or ~/.zshrc.
把你要用的服务密钥设为环境变量,例如 OPENAI_API_KEY、DEEPSEEK_API_KEY、GEMINI_API_KEY 等。
Windows 用户可在「系统属性 → 环境变量」里添加;Mac/Linux 用户写进 ~/.bashrc 或 ~/.zshrc。

Step 3: Start the gateway and test第三步:启动网关并测试
Execute litellm --model deepseek/deepseek-chat to start the local gateway (default address http://0.0.0.0:4000).
Use any client in OpenAI format to point the base_url to http://localhost:4000/v1 and write the model name as deepseek/deepseek-chat to have a direct conversation.
When changing to another model, you only need to change the model name prefix (such as gemini/gemini-2.5-pro), without changing a single line of code.
执行 litellm --model deepseek/deepseek-chat 启动本地网关(默认地址 http://0.0.0.0:4000)。
用任意 OpenAI 格式的客户端把 base_url 指向 http://localhost:4000/v1,模型名写成 deepseek/deepseek-chat 就能直接对话。
换其他模型只改模型名前缀(如 gemini/gemini-2.5-pro),代码一行不用动。

Step 4: Advanced - Budget, Log and Guardrails第四步:进阶——预算、日志与护栏
Write a config.yaml to hang multiple models at the same time and set the daily spending limit for each user.
Add --detailed_debug when starting to see the model, token and time-consuming details of each call in the terminal.
The new guardrail capability of v1.102 can intercept sensitive content in real time during the streaming output process. It is recommended to enable it in enterprise access scenarios.
写一个 config.yaml 可同时挂多个模型、设置每用户每日花费上限。
启动时加 --detailed_debug 可以在终端看到每次调用的模型、Token 与耗时明细。
v1.102 的新护栏能力可在流式输出过程中实时拦截敏感内容,企业接入场景建议开启。

Notes使用提醒
This article is compiled from public sources and ships with the matching resource. Product features and pricing are subject to the official page. Resources are for learning and exchange only — please respect the original license.
本文整理自公开资料并附上配套资源;涉及产品的功能与价格以官方页面为准。资源仅供学习交流,请遵循来源许可。
本篇为图文教程,直接按步骤操作即可,无需下载文件。
阅读与点赞数据保存在你的浏览器本地,欢迎留下你的想法。
今日正能量学一点,用一点;今天种下的种子,会长成明天的能力。去免费下载专区 →广告
评论 文明发言,让讨论更有价值