Skip to content

Feat/add litellm provider#1135

Open
RheagalFire wants to merge 4 commits into
PDFMathTranslate:mainfrom
RheagalFire:feat/add-litellm-provider
Open

Feat/add litellm provider#1135
RheagalFire wants to merge 4 commits into
PDFMathTranslate:mainfrom
RheagalFire:feat/add-litellm-provider

Conversation

@RheagalFire
Copy link
Copy Markdown

@RheagalFire RheagalFire commented May 12, 2026

Summary

  • Adds LiteLLM as a new translation service provider, giving users access to 100+ LLM providers (Anthropic, Bedrock, Vertex AI, Cohere, Mistral, etc.) through a single unified interface.
  • Follows the existing translator class pattern. Drop-in, no changes to existing providers.

Motivation

PDFMathTranslate currently supports individual providers (OpenAI, Gemini, Groq, Deepseek, etc.), each requiring a separate translator class. LiteLLM is a lightweight Python SDK that provides a unified completion() interface across 100+ providers. Users specify the provider via the model string (e.g. anthropic/claude-sonnet-4-5, bedrock/anthropic.claude-v2, vertex_ai/gemini-pro) and LiteLLM routes the call to the correct provider API. This eliminates the need to add individual translator classes for each new provider.

Changes

  • pdf2zh/translator.py - new LiteLLMTranslator class extending BaseTranslator
  • pdf2zh/converter.py - registered in import list and translator loop
  • pdf2zh/gui.py - registered in import, service_map, babeldoc_translate_file loop, and isinstance check for rich text support
  • pdf2zh/pdf2zh.py - registered in import and translator loop
  • pyproject.toml - added litellm>=1.55.0,<1.85 as optional dependency (pip install pdf2zh[litellm])

Implementation details

  • Optional dependency: litellm is an optional extra (like argostranslate), so the base install is unaffected. Lazy-imported with a clear error message if not installed.
  • drop_params=True: Silently drops provider-unsupported kwargs (e.g. strict, seed), preventing cross-provider failures.
  • Streaming: Always enabled, matching OpenAITranslator behavior.
  • Retry on rate limits: Uses tenacity with exponential backoff on litellm.RateLimitError.
  • Think-tag filtering: Strips <think> tags from reasoning model outputs (DeepSeek-R1, QwQ, etc.).
  • Rich text support: Enabled in babeldoc mode via isinstance check alongside OpenAITranslator.
  • Provider auth: LiteLLM reads provider-specific env vars automatically (ANTHROPIC_API_KEY, OPENAI_API_KEY, GEMINI_API_KEY, etc.).

Service table entry

Service Name Env vars Defaults Reference
LiteLLM litellm LITELLM_MODEL + provider-specific keys (e.g. ANTHROPIC_API_KEY) gpt-4o-mini See LiteLLM Providers

Example usage

Use -s litellm:model to specify the provider and model:

pip install pdf2zh[litellm]

# Translate using Anthropic Claude
export ANTHROPIC_API_KEY="sk-..."
pdf2zh example.pdf -s litellm:anthropic/claude-sonnet-4-5

# Translate using AWS Bedrock
export AWS_ACCESS_KEY_ID="..." AWS_SECRET_ACCESS_KEY="..." AWS_REGION_NAME="us-east-1"
pdf2zh example.pdf -s litellm:bedrock/anthropic.claude-v2

# Translate using Google Vertex AI
export VERTEX_PROJECT="my-project" VERTEX_LOCATION="us-central1"
pdf2zh example.pdf -s litellm:vertex_ai/gemini-pro

Or specify the model with environment variables:

export LITELLM_MODEL=anthropic/claude-sonnet-4-5
export ANTHROPIC_API_KEY="sk-..."
pdf2zh example.pdf -s litellm

GUI: Select "LiteLLM" from the service dropdown, set LITELLM_MODEL to your desired model string (e.g. anthropic/claude-sonnet-4-5). Set the provider's API key as an environment variable before launching the GUI.

Tests

Live E2E test against Anthropic Claude (anthropic/claude-sonnet-4-5):

>>> t = LiteLLMTranslator('en', 'zh', 'anthropic/claude-sonnet-4-5')
>>> t.do_translate('Hello world')
'你好世界'

>>> t.do_translate('Machine learning is transforming the world.')
'机器学习正在改变世界。'

>>> t.do_translate('The equation {{v0}} shows that {{v1}} is proportional to {{v2}}.')
'方程 {{v0}} 表明 {{v1}} 与 {{v2}} 成正比。'

Formula placeholders ({{v0}}, {{v1}}, {{v2}}) are preserved correctly in translated output.

Risk / Compatibility

  • Additive only. Existing providers untouched.
  • litellm is an optional extra. Base install unaffected.
  • Pinned to >=1.55.0,<1.85 for stability.

@RheagalFire
Copy link
Copy Markdown
Author

@reycn do you have any update on this PR?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant