Byaidu/PDFMathTranslate
Fork: 614 Star: 8688 (更新于 2024-12-17 15:19:20)
license: AGPL-3.0
Language: Python .
PDF scientific paper translation with preserved formats - 基于 AI 完整保留排版的 PDF 文档全文双语翻译,支持 Google/DeepL/Ollama/OpenAI 等服务,提供 CLI/GUI/Docker
最后发布版本: v1.8.8 ( 2024-12-15 22:30:17)
PDF scientific paper translation and bilingual comparison.
- 📊 Preserve formulas, charts, table of contents, and annotations (preview).
- 🌐 Support multiple languages, and diverse translation services.
- 🤖 Provides commandline tool, interactive user interface, and Docker
Feel free to provide feedback in GitHub Issues, Telegram Group or QQ Group.
Updates
- [Nov. 26 2024] CLI now supports online file(s) (by @reycn)
- [Nov. 24 2024] ONNX support to reduce dependency sizes (by @Wybxc)
- [Nov. 23 2024] 🌟 Public Service online! (by @Byaidu)
- [Nov. 23 2024] Firewall for preventing web bots (by @Byaidu)
- [Nov. 22 2024] GUI now supports Italian, and has been improved (by @Byaidu, @reycn)
- [Nov. 22 2024] You can now share your deployed service to others (by @Zxis233)
- [Nov. 22 2024] Now supports Tencent Translation (by @hellofinch)
- [Nov. 21 2024] GUI now supports downloading dual-document (by @reycn)
- [Nov. 20 2024] 🌟 Demo online! (by @reycn)
Preview
Public Service 🌟
Free Service (https://pdf2zh.com/)
You can try our public service online without installation.
Hugging Face Demo
You can try our demo on HuggingFace without installation. Note that the computing resources of the demo are limited, so please avoid abusing them.
Installation and Usage
We provide four methods for using this project: Commandline, Portable, GUI, and Docker.
Method I. Commandline
-
Python installed (3.8 <= version <= 3.12)
-
Install our package:
pip install pdf2zh
-
Execute translation, files generated in current working directory:
pdf2zh document.pdf
Method II. Portable
No need to pre-install Python environment
Download setup.bat and double-click to run
Method III. GUI
-
Python installed (3.8 <= version <= 3.12)
-
Install our package:
pip install pdf2zh
-
Start using in browser:
pdf2zh -i
-
If your browswer has not been started automatically, goto
http://localhost:7860/
See documentation for GUI for more details.
Method IV. Docker
-
Pull and run:
docker pull byaidu/pdf2zh docker run -d -p 7860:7860 byaidu/pdf2zh
-
Open in browser:
http://localhost:7860/
For docker deployment on cloud service:
Advanced Options
Execute the translation command in the command line to generate the translated document example-mono.pdf
and the bilingual document example-dual.pdf
in the current working directory. Use Google as the default translation service.
In the following table, we list all advanced options for reference:
Option | Function | Example |
---|---|---|
files | Local files | pdf2zh ~/local.pdf |
links | Online files | pdf2zh http://arxiv.org/paper.pdf |
-i |
Enter GUI | pdf2zh -i |
-p |
Partial document translation | pdf2zh example.pdf -p 1 |
-li |
Source language | pdf2zh example.pdf -li en |
-lo |
Target language | pdf2zh example.pdf -lo zh |
-s |
Translation service | pdf2zh example.pdf -s deepl |
-t |
Multi-threads | pdf2zh example.pdf -t 1 |
-o |
Output dir | pdf2zh example.pdf -o output |
-f , -c |
Exceptions | pdf2zh example.pdf -f "(MS.*)" |
--share |
[Get gradio public link] | pdf2zh -i --share |
-a |
[add authorization and custom login page] | pdf2zh -i -a users.txt [auth.html] |
Full / partial document translation
-
Entire document
pdf2zh example.pdf
-
Part of the document
pdf2zh example.pdf -p 1-3,5
Specify source and target languages
See Google Languages Codes, DeepL Languages Codes
pdf2zh example.pdf -li en -lo ja
Translate with Different Services
The table below outlines the required environment variables for each translation service. Make sure to set them before using the respective service.
Translator | Service | Environment Variables | Default Values | Notes |
---|---|---|---|---|
Google (Default) | google |
None | N/A | None |
Bing | bing |
None | N/A | None |
DeepL | deepl |
DEEPL_AUTH_KEY |
[Your Key] |
See DeepL |
DeepLX | deeplx |
DEEPLX_ENDPOINT |
https://api.deepl.com/translate |
See DeepLX |
Ollama | ollama |
OLLAMA_HOST , OLLAMA_MODEL |
http://127.0.0.1:11434 , gemma2 |
See Ollama |
OpenAI | openai |
OPENAI_BASE_URL , OPENAI_API_KEY , OPENAI_MODEL |
https://api.openai.com/v1 , [Your Key] , gpt-4o-mini |
See OpenAI |
AzureOpenAI | azure-openai |
AZURE_OPENAI_BASE_URL , AZURE_OPENAI_API_KEY , AZURE_OPENAI_MODEL |
[Your Endpoint] , [Your Key] , gpt-4o-mini |
See Azure OpenAI |
Zhipu | zhipu |
ZHIPU_API_KEY , ZHIPU_MODEL |
[Your Key] , glm-4-flash |
See Zhipu |
ModelScope | ModelScope |
MODELSCOPE_API_KEY , MODELSCOPE_MODEL |
[Your Key] , Qwen/Qwen2.5-Coder-32B-Instruct |
See ModelScope |
Silicon | silicon |
SILICON_API_KEY , SILICON_MODEL |
[Your Key] , Qwen/Qwen2.5-7B-Instruct |
See SiliconCloud |
Gemini | gemini |
GEMINI_API_KEY , GEMINI_MODEL |
[Your Key] , gemini-1.5-flash |
See Gemini |
Azure | azure |
AZURE_ENDPOINT , AZURE_API_KEY |
https://api.translator.azure.cn , [Your Key] |
See Azure |
Tencent | tencent |
TENCENTCLOUD_SECRET_ID , TENCENTCLOUD_SECRET_KEY |
[Your ID] , [Your Key] |
See Tencent |
Dify | dify |
DIFY_API_URL , DIFY_API_KEY |
[Your DIFY URL] , [Your Key] |
See Dify,Three variables, lang_out, lang_in, and text, need to be defined in Dify's workflow input. |
AnythingLLM | anythingllm |
AnythingLLM_URL , AnythingLLM_APIKEY |
[Your AnythingLLM URL] , [Your Key] |
See anything-llm |
Use -s service
or -s service:model
to specify service:
pdf2zh example.pdf -s openai:gpt-4o-mini
Or specify model with environment variables:
set OPENAI_MODEL=gpt-4o-mini
pdf2zh example.pdf -s openai
Translate wih exceptions
Use regex to specify formula fonts and characters that need to be preserved:
pdf2zh example.pdf -f "(CM[^RT].*|MS.*|.*Ital)" -c "(\(|\||\)|\+|=|\d|[\u0080-\ufaff])"
Preserve Latex
, Mono
, Code
, Italic
, Symbol
and Math
fonts by default:
pdf2zh example.pdf -f "(CM[^R]|(MS|XY|MT|BL|RM|EU|LA|RS)[A-Z]|LINE|LCIRCLE|TeX-|rsfs|txsy|wasy|stmary|.*Mono|.*Code|.*Ital|.*Sym|.*Math)"
Specify threads
Use -t
to specify how many threads to use in translation:
pdf2zh example.pdf -t 1
API
Python
from pdf2zh import translate, translate_stream
params = {"lang_in": "en", "lang_out": "zh", "service": "google", "thread": 4}
file_mono, file_dual = translate(files=["example.pdf"], **params)[0]
with open("example.pdf", "rb") as f:
stream_mono, stream_dual = translate_stream(stream=f.read(), **params)
HTTP
pip install pdf2zh[backend]
pdf2zh --flask
pdf2zh --celery worker
curl http://localhost:11008/v1/translate -F "file=@example.pdf" -F "data={\"lang_in\":\"en\",\"lang_out\":\"zh\",\"service\":\"google\",\"thread\":4}"
{"id":"d9894125-2f4e-45ea-9d93-1a9068d2045a"}
curl http://localhost:11008/v1/translate/d9894125-2f4e-45ea-9d93-1a9068d2045a
{"info":{"n":13,"total":506},"state":"PROGRESS"}
curl http://localhost:11008/v1/translate/d9894125-2f4e-45ea-9d93-1a9068d2045a
{"state":"SUCCESS"}
curl http://localhost:11008/v1/translate/d9894125-2f4e-45ea-9d93-1a9068d2045a/mono --output example-mono.pdf
curl http://localhost:11008/v1/translate/d9894125-2f4e-45ea-9d93-1a9068d2045a/dual --output example-dual.pdf
curl http://localhost:11008/v1/translate/d9894125-2f4e-45ea-9d93-1a9068d2045a -X DELETE
TODO
Parse layout with DocLayNet based models, PaddleX, PaperMage, SAM2
Fix page rotation, table of contents, format of lists
Fix pixel formula in old papers
Async retry except KeyboardInterrupt
Knuth–Plass algorithm for western languages
Support non-PDF/A files
Acknowledgements
-
Document merging: PyMuPDF
-
Document parsing: Pdfminer.six
-
Document extraction: MinerU
-
Multi-threaded translation: MathTranslate
-
Layout parsing: DocLayout-YOLO
-
Document standard: PDF Explained, PDF Cheat Sheets
-
Multilingual Font: Go Noto Universal
Contributors
Star History
最近版本更新:(数据更新于 2024-12-17 16:56:33)
2024-12-15 22:30:17 v1.8.8
2024-12-08 22:55:53 v1.8.7
2024-12-07 02:55:46 v1.8.6
2024-12-07 02:02:51 v1.8.5
2024-12-04 02:08:41 v1.8.4
2024-12-03 20:08:57 v1.8.3
2024-12-03 19:37:36 v1.8.2
2024-11-29 14:08:54 v1.8.1
2024-11-25 21:47:00 v1.8.0
2024-11-22 23:07:49 v1.7.9
主题(topics):
chinese, document, edit, english, japanese, korean, latex, math, modify, obsidian, openai, pdf, pdf2zh, python, russian, translate, translation, zotero
Byaidu/PDFMathTranslate同语言 Python最近更新仓库
2024-12-18 13:59:35 hiddify/Hiddify-Manager
2024-12-18 04:19:38 xtekky/gpt4free
2024-12-18 03:07:12 ultralytics/ultralytics
2024-12-17 23:42:54 huggingface/transformers
2024-12-17 22:31:54 home-assistant/core
2024-12-17 21:53:19 XiaoMi/ha_xiaomi_home