fix(images): vendor spacy zh model + nltk data for air-gapped GitHub builds
CodeQL Advanced / Analyze (actions) (push) Canceled after 0s
CodeQL Advanced / Analyze (java-kotlin) (push) Canceled after 0s
CodeQL Advanced / Analyze (javascript-typescript) (push) Canceled after 0s
CodeQL Advanced / Analyze (python) (push) Canceled after 0s
Docker Image Build & Push / manifest (push) Canceled after 0s
Docker Image Build & Push / build-and-push-arm (push) Canceled after 0s
Docker Image Build & Push / build-and-push-amd (push) Canceled after 0s
Docker Image Build & Push / manifest (push) Canceled after 0s
Docker Image Build & Push / build-and-push-arm (push) Canceled after 0s
Docker Image Build & Push / build-and-push-amd (push) Canceled after 0s
Python Backend Docker Image CI / Build and Push Backend Python Docker Image (push) Canceled after 0s
Runtime Docker Image CI / Build and Push Runtime Docker Image (push) Canceled after 0s

- runtime: install zh_core_web_sm from vendored wheel (reused from runtime-ray)
  instead of 'python -m spacy download' (GitHub 403 on server)
- runtime / runtime-ray / backend-python: replace nltk.download (GitHub +
  nltk_data.nltk.org unreachable from server) with vendored zips extracted
  to /usr/local/nltk_data/packages/{tokenizers,taggers}
This commit is contained in:
figmar
2026-08-31 05:09:05 +08:00
parent 7fcef906de
commit 99cdc24e44
5 changed files with 10 additions and 5 deletions
+3 -2
View File
@@ -58,8 +58,9 @@ RUN --mount=type=cache,target=$POETRY_CACHE_DIR \
# 注意:使用 [default] 确保包含 job submission client
RUN pip install "ray[default]>=2.9.0"
# 下载 NLTK 数据
RUN python -c "import nltk; nltk.download(['punkt_tab','averaged_perceptron_tagger_eng'], download_dir='/usr/local/nltk_data')"
# 下载 NLTK 数据(离线:使用仓库内 vendor 的数据包,服务器无法访问 GitHub)
COPY scripts/images/common/nltk-data/ /tmp/nltk-data/
RUN python -c "import zipfile, os; os.makedirs('/usr/local/nltk_data/packages/tokenizers', exist_ok=True); os.makedirs('/usr/local/nltk_data/packages/taggers', exist_ok=True); zipfile.ZipFile('/tmp/nltk-data/punkt_tab.zip').extractall('/usr/local/nltk_data/packages/tokenizers'); zipfile.ZipFile('/tmp/nltk-data/averaged_perceptron_tagger_eng.zip').extractall('/usr/local/nltk_data/packages/taggers')"
ENV NLTK_DATA=/usr/local/nltk_data
# 复制应用代码
Binary file not shown.
+2 -1
View File
@@ -83,6 +83,7 @@ ENV PATH="$PATH:$JAVA_HOME/bin"
WORKDIR /opt/runtime
COPY scripts/images/runtime-ray/zh_core_web_sm-3.8.0-py3-none-any.whl /tmp/
COPY scripts/images/common/nltk-data/ /tmp/nltk-data/
ENV UV_LINK_MODE=copy
@@ -92,7 +93,7 @@ RUN --mount=type=cache,target=/root/.cache/uv \
&& uv pip install -r /opt/runtime/deepdata/ops/pyproject.toml --system \
&& uv pip install "ray[default]>=2.9.0" --system \
&& uv pip install /tmp/zh_core_web_sm-3.8.0-py3-none-any.whl --system \
&& python -c "import nltk; nltk.download('punkt_tab'); nltk.download('averaged_perceptron_tagger_eng')" \
&& python -c "import zipfile, os; os.makedirs('/usr/local/nltk_data/packages/tokenizers', exist_ok=True); os.makedirs('/usr/local/nltk_data/packages/taggers', exist_ok=True); zipfile.ZipFile('/tmp/nltk-data/punkt_tab.zip').extractall('/usr/local/nltk_data/packages/tokenizers'); zipfile.ZipFile('/tmp/nltk-data/averaged_perceptron_tagger_eng.zip').extractall('/usr/local/nltk_data/packages/taggers')" \
&& echo "/usr/local/lib/ops/site-packages" > /usr/local/lib/python3.11/site-packages/ops.pth
+5 -2
View File
@@ -15,6 +15,9 @@ COPY runtime/ops /opt/runtime/deepdata/ops
COPY runtime/ops/user /opt/runtime/user
COPY scripts/images/runtime/start.sh /opt/runtime/start.sh
COPY scripts/images/common/log-rotate-copytruncate.sh /usr/local/bin/log-rotate-copytruncate.sh
# 🟢 离线安装:spacy 中文模型(复用 runtime-ray 已 vendor 的 wheel)+ nltk 数据包
COPY scripts/images/runtime-ray/zh_core_web_sm-3.8.0-py3-none-any.whl /tmp/
COPY scripts/images/common/nltk-data/ /tmp/nltk-data/
ENV PYTHONPATH=/opt/runtime/deepdata/
ENV UV_EXTRA_INDEX_URL="https://download.pytorch.org/whl/cpu"
@@ -25,8 +28,8 @@ WORKDIR /opt/runtime
RUN --mount=type=cache,target=/root/.cache/uv \
uv pip install -e . --system \
&& uv pip install -r /opt/runtime/deepdata/ops/pyproject.toml --system \
&& python -m spacy download zh_core_web_sm \
&& python -c "import nltk; nltk.download('punkt_tab'); nltk.download('averaged_perceptron_tagger_eng')" \
&& uv pip install /tmp/zh_core_web_sm-3.8.0-py3-none-any.whl --system \
&& python -c "import zipfile, os; os.makedirs('/usr/local/nltk_data/packages/tokenizers', exist_ok=True); os.makedirs('/usr/local/nltk_data/packages/taggers', exist_ok=True); zipfile.ZipFile('/tmp/nltk-data/punkt_tab.zip').extractall('/usr/local/nltk_data/packages/tokenizers'); zipfile.ZipFile('/tmp/nltk-data/averaged_perceptron_tagger_eng.zip').extractall('/usr/local/nltk_data/packages/taggers')" \
&& echo "/usr/local/lib/ops/site-packages" > /usr/local/lib/python3.11/site-packages/ops.pth
RUN ln -sf /usr/share/zoneinfo/Asia/Shanghai /etc/localtime \