fix(images): vendor spacy zh model + nltk data for air-gapped GitHub builds
CodeQL Advanced / Analyze (actions) (push) Canceled after 0s
CodeQL Advanced / Analyze (java-kotlin) (push) Canceled after 0s
CodeQL Advanced / Analyze (javascript-typescript) (push) Canceled after 0s
CodeQL Advanced / Analyze (python) (push) Canceled after 0s
Docker Image Build & Push / manifest (push) Canceled after 0s
Docker Image Build & Push / build-and-push-arm (push) Canceled after 0s
Docker Image Build & Push / build-and-push-amd (push) Canceled after 0s
Docker Image Build & Push / manifest (push) Canceled after 0s
Docker Image Build & Push / build-and-push-arm (push) Canceled after 0s
Docker Image Build & Push / build-and-push-amd (push) Canceled after 0s
Python Backend Docker Image CI / Build and Push Backend Python Docker Image (push) Canceled after 0s
Runtime Docker Image CI / Build and Push Runtime Docker Image (push) Canceled after 0s
CodeQL Advanced / Analyze (actions) (push) Canceled after 0s
CodeQL Advanced / Analyze (java-kotlin) (push) Canceled after 0s
CodeQL Advanced / Analyze (javascript-typescript) (push) Canceled after 0s
CodeQL Advanced / Analyze (python) (push) Canceled after 0s
Docker Image Build & Push / manifest (push) Canceled after 0s
Docker Image Build & Push / build-and-push-arm (push) Canceled after 0s
Docker Image Build & Push / build-and-push-amd (push) Canceled after 0s
Docker Image Build & Push / manifest (push) Canceled after 0s
Docker Image Build & Push / build-and-push-arm (push) Canceled after 0s
Docker Image Build & Push / build-and-push-amd (push) Canceled after 0s
Python Backend Docker Image CI / Build and Push Backend Python Docker Image (push) Canceled after 0s
Runtime Docker Image CI / Build and Push Runtime Docker Image (push) Canceled after 0s
- runtime: install zh_core_web_sm from vendored wheel (reused from runtime-ray)
instead of 'python -m spacy download' (GitHub 403 on server)
- runtime / runtime-ray / backend-python: replace nltk.download (GitHub +
nltk_data.nltk.org unreachable from server) with vendored zips extracted
to /usr/local/nltk_data/packages/{tokenizers,taggers}
This commit is contained in:
@@ -58,8 +58,9 @@ RUN --mount=type=cache,target=$POETRY_CACHE_DIR \
|
||||
# 注意:使用 [default] 确保包含 job submission client
|
||||
RUN pip install "ray[default]>=2.9.0"
|
||||
|
||||
# 下载 NLTK 数据
|
||||
RUN python -c "import nltk; nltk.download(['punkt_tab','averaged_perceptron_tagger_eng'], download_dir='/usr/local/nltk_data')"
|
||||
# 下载 NLTK 数据(离线:使用仓库内 vendor 的数据包,服务器无法访问 GitHub)
|
||||
COPY scripts/images/common/nltk-data/ /tmp/nltk-data/
|
||||
RUN python -c "import zipfile, os; os.makedirs('/usr/local/nltk_data/packages/tokenizers', exist_ok=True); os.makedirs('/usr/local/nltk_data/packages/taggers', exist_ok=True); zipfile.ZipFile('/tmp/nltk-data/punkt_tab.zip').extractall('/usr/local/nltk_data/packages/tokenizers'); zipfile.ZipFile('/tmp/nltk-data/averaged_perceptron_tagger_eng.zip').extractall('/usr/local/nltk_data/packages/taggers')"
|
||||
ENV NLTK_DATA=/usr/local/nltk_data
|
||||
|
||||
# 复制应用代码
|
||||
|
||||
Binary file not shown.
Binary file not shown.
@@ -83,6 +83,7 @@ ENV PATH="$PATH:$JAVA_HOME/bin"
|
||||
WORKDIR /opt/runtime
|
||||
|
||||
COPY scripts/images/runtime-ray/zh_core_web_sm-3.8.0-py3-none-any.whl /tmp/
|
||||
COPY scripts/images/common/nltk-data/ /tmp/nltk-data/
|
||||
|
||||
ENV UV_LINK_MODE=copy
|
||||
|
||||
@@ -92,7 +93,7 @@ RUN --mount=type=cache,target=/root/.cache/uv \
|
||||
&& uv pip install -r /opt/runtime/deepdata/ops/pyproject.toml --system \
|
||||
&& uv pip install "ray[default]>=2.9.0" --system \
|
||||
&& uv pip install /tmp/zh_core_web_sm-3.8.0-py3-none-any.whl --system \
|
||||
&& python -c "import nltk; nltk.download('punkt_tab'); nltk.download('averaged_perceptron_tagger_eng')" \
|
||||
&& python -c "import zipfile, os; os.makedirs('/usr/local/nltk_data/packages/tokenizers', exist_ok=True); os.makedirs('/usr/local/nltk_data/packages/taggers', exist_ok=True); zipfile.ZipFile('/tmp/nltk-data/punkt_tab.zip').extractall('/usr/local/nltk_data/packages/tokenizers'); zipfile.ZipFile('/tmp/nltk-data/averaged_perceptron_tagger_eng.zip').extractall('/usr/local/nltk_data/packages/taggers')" \
|
||||
&& echo "/usr/local/lib/ops/site-packages" > /usr/local/lib/python3.11/site-packages/ops.pth
|
||||
|
||||
|
||||
|
||||
@@ -15,6 +15,9 @@ COPY runtime/ops /opt/runtime/deepdata/ops
|
||||
COPY runtime/ops/user /opt/runtime/user
|
||||
COPY scripts/images/runtime/start.sh /opt/runtime/start.sh
|
||||
COPY scripts/images/common/log-rotate-copytruncate.sh /usr/local/bin/log-rotate-copytruncate.sh
|
||||
# 🟢 离线安装:spacy 中文模型(复用 runtime-ray 已 vendor 的 wheel)+ nltk 数据包
|
||||
COPY scripts/images/runtime-ray/zh_core_web_sm-3.8.0-py3-none-any.whl /tmp/
|
||||
COPY scripts/images/common/nltk-data/ /tmp/nltk-data/
|
||||
|
||||
ENV PYTHONPATH=/opt/runtime/deepdata/
|
||||
ENV UV_EXTRA_INDEX_URL="https://download.pytorch.org/whl/cpu"
|
||||
@@ -25,8 +28,8 @@ WORKDIR /opt/runtime
|
||||
RUN --mount=type=cache,target=/root/.cache/uv \
|
||||
uv pip install -e . --system \
|
||||
&& uv pip install -r /opt/runtime/deepdata/ops/pyproject.toml --system \
|
||||
&& python -m spacy download zh_core_web_sm \
|
||||
&& python -c "import nltk; nltk.download('punkt_tab'); nltk.download('averaged_perceptron_tagger_eng')" \
|
||||
&& uv pip install /tmp/zh_core_web_sm-3.8.0-py3-none-any.whl --system \
|
||||
&& python -c "import zipfile, os; os.makedirs('/usr/local/nltk_data/packages/tokenizers', exist_ok=True); os.makedirs('/usr/local/nltk_data/packages/taggers', exist_ok=True); zipfile.ZipFile('/tmp/nltk-data/punkt_tab.zip').extractall('/usr/local/nltk_data/packages/tokenizers'); zipfile.ZipFile('/tmp/nltk-data/averaged_perceptron_tagger_eng.zip').extractall('/usr/local/nltk_data/packages/taggers')" \
|
||||
&& echo "/usr/local/lib/ops/site-packages" > /usr/local/lib/python3.11/site-packages/ops.pth
|
||||
|
||||
RUN ln -sf /usr/share/zoneinfo/Asia/Shanghai /etc/localtime \
|
||||
|
||||
Reference in New Issue
Block a user