Skip to content

Commit fbf6c96

Browse files
authored
Add Docker image build and push workflow (#413)
1 parent 8ca6e57 commit fbf6c96

File tree

12 files changed

+148
-14
lines changed

12 files changed

+148
-14
lines changed

.github/workflows/docker.yaml

Lines changed: 67 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,67 @@
1+
#
2+
name: Create and publish a Docker image
3+
4+
on:
5+
release:
6+
types: [published]
7+
8+
workflow_dispatch:
9+
inputs:
10+
ref:
11+
description: 'The git ref (branch or tag) to build the Docker image from.'
12+
required: true
13+
14+
env:
15+
REGISTRY: ghcr.io
16+
IMAGE_NAME: ${{ github.repository }}
17+
18+
jobs:
19+
build-and-push-image:
20+
runs-on: self-hosted
21+
timeout-minutes: 240 # wait up to 4 hours
22+
# Sets the permissions granted to the `GITHUB_TOKEN` for the actions in this job.
23+
permissions:
24+
contents: read
25+
packages: write
26+
attestations: write
27+
id-token: write
28+
steps:
29+
- name: Checkout repository
30+
uses: actions/checkout@v5
31+
# Uses the `docker/login-action` action to log in to the Container registry registry using the account and password that will publish the packages. Once published, the packages are scoped to the account defined here.
32+
- name: Log in to the Container registry
33+
uses: docker/login-action@v2
34+
with:
35+
registry: ${{ env.REGISTRY }}
36+
username: ${{ github.actor }}
37+
password: ${{ secrets.GITHUB_TOKEN }}
38+
# This step uses [docker/metadata-action](https://github.com/docker/metadata-action#about) to extract tags and labels that will be applied to the specified image. The `id` "meta" allows the output of this step to be referenced in a subsequent step. The `images` value provides the base name for the tags and labels.
39+
- name: Extract metadata (tags, labels) for Docker
40+
id: meta
41+
uses: docker/metadata-action@v5
42+
with:
43+
images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
44+
tags: |
45+
type=match,pattern=\d.\d.\d
46+
type=sha
47+
# This step uses the `docker/build-push-action` action to build the image, based on your repository's `Dockerfile`. If the build succeeds, it pushes the image to GitHub Packages.
48+
# It uses the `context` parameter to define the build's context as the set of files located in the specified path. For more information, see [Usage](https://github.com/docker/build-push-action#usage) in the README of the `docker/build-push-action` repository.
49+
# It uses the `tags` and `labels` parameters to tag and label the image with the output from the "meta" step.
50+
- name: Build and push Docker image
51+
id: push
52+
uses: docker/build-push-action@v4
53+
with:
54+
context: .
55+
push: true
56+
file: scripts/docker/Dockerfile
57+
shm-size: 64g
58+
tags: ${{ steps.meta.outputs.tags }}
59+
labels: ${{ steps.meta.outputs.labels }}
60+
61+
# This step generates an artifact attestation for the image, which is an unforgeable statement about where and how it was built. It increases supply chain security for people who consume the image. For more information, see [Using artifact attestations to establish provenance for builds](/actions/security-guides/using-artifact-attestations-to-establish-provenance-for-builds).
62+
- name: Generate artifact attestation
63+
uses: actions/attest-build-provenance@v3
64+
with:
65+
subject-name: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME}}
66+
subject-digest: ${{ steps.push.outputs.digest }}
67+
push-to-registry: true

README.md

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -88,6 +88,7 @@ Trinity-RFT provides functionalities for users with different backgrounds and ob
8888

8989
## 🚀 News
9090

91+
* [2025-11] [[Release Notes](https://github.com/modelscope/Trinity-RFT/releases/tag/v0.3.3)] Trinity-RFT v0.3.3 released: bug fixes.
9192
* [2025-11] Introducing [Learn-to-Ask](https://github.com/modelscope/Trinity-RFT/tree/main/examples/learn_to_ask): a framework for training proactive dialogue agents from offline expert data ([paper](https://arxiv.org/pdf/2510.25441)).
9293
* [2025-11] Introducing [BOTS](https://github.com/modelscope/Trinity-RFT/tree/main/examples/bots): online RL task selection for efficient LLM fine-tuning ([paper](https://arxiv.org/pdf/2510.26374)).
9394
* [2025-11] [[Release Notes](https://github.com/modelscope/Trinity-RFT/releases/tag/v0.3.2)] Trinity-RFT v0.3.2 released: bug fixes and advanced task selection & scheduling.
@@ -129,7 +130,7 @@ Trinity-RFT provides functionalities for users with different backgrounds and ob
129130
Before installing, make sure your system meets the following requirements:
130131

131132
- **Python**: version 3.10 to 3.12 (inclusive)
132-
- **CUDA**: version >= 12.6
133+
- **CUDA**: version >= 12.8
133134
- **GPUs**: at least 2 GPUs
134135

135136

README_zh.md

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -88,6 +88,7 @@ Trinity-RFT 面向不同背景和目标的用户提供相应功能:
8888

8989
## 🚀 新闻
9090

91+
* [2025-11] [[发布说明](https://github.com/modelscope/Trinity-RFT/releases/tag/v0.3.3)] Trinity-RFT v0.3.3 发布:修复若干 Bug。
9192
* [2025-11] 推出 [Learn-to-Ask](https://github.com/modelscope/Trinity-RFT/tree/main/examples/learn_to_ask):利用离线专家数据,训练具备主动问询能力的对话智能体([论文](https://arxiv.org/pdf/2510.25441)).
9293
* [2025-11] 推出 [BOTS](https://github.com/modelscope/Trinity-RFT/tree/main/examples/bots):在线 RL 任务选择,实现高效 LLM 微调([论文](https://arxiv.org/pdf/2510.26374))。
9394
* [2025-11] [[发布说明](https://github.com/modelscope/Trinity-RFT/releases/tag/v0.3.2)] Trinity-RFT v0.3.2 发布:修复若干 Bug 并支持进阶的任务选择和调度。
@@ -129,7 +130,7 @@ Trinity-RFT 面向不同背景和目标的用户提供相应功能:
129130
在安装之前,请确保您的系统满足以下要求:
130131

131132
- **Python**:版本 3.10 至 3.12(含)
132-
- **CUDA**:版本 >= 12.6
133+
- **CUDA**:版本 >= 12.8
133134
- **GPU**:至少 2 块 GPU
134135

135136
## 源码安装(推荐)

docs/sphinx_doc/source/tutorial/example_megatron.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -38,10 +38,10 @@ We provide a Docker setup to simplify environment management.
3838
#### Build the Docker Image
3939

4040

41-
Trinity-RFT provides a dedicated Dockerfile for Megatron-LM located at `scripts/docker_for_megatron/Dockerfile`. You can build the image using the following command:
41+
Trinity-RFT provides a dedicated Dockerfile for Megatron-LM located at `scripts/docker/Dockerfile.megatron`. You can build the image using the following command:
4242

4343
```bash
44-
docker build -f scripts/docker_for_megatron/Dockerfile -t trinity-rft-megatron:latest .
44+
docker build -f scripts/docker/Dockerfile.megatron -t trinity-rft-megatron:latest .
4545
```
4646

4747
> 💡 You can customize the Dockerfile before building — for example, to add pip mirrors or set API keys.

docs/sphinx_doc/source/tutorial/trinity_installation.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,7 @@ For installing Trinity-RFT, you have three options: from source (recommended), v
66
Before installing, ensure your system meets the following requirements:
77

88
- **Python**: Version 3.10 to 3.12 (inclusive)
9-
- **CUDA**: Version >= 12.6
9+
- **CUDA**: Version >= 12.8
1010
- **GPUs**: At least 2 GPUs
1111

1212
---

docs/sphinx_doc/source_zh/tutorial/example_megatron.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -43,11 +43,11 @@ pip install -v --disable-pip-version-check --no-cache-dir --no-build-isolation \
4343

4444
#### 构建 Docker 镜像
4545

46-
Trinity-RFT 提供了专门用于 Megatron-LM 的 Dockerfile,位于 `scripts/docker_for_megatron/Dockerfile`
46+
Trinity-RFT 提供了专门用于 Megatron-LM 的 Dockerfile,位于 `scripts/docker/Dockerfile.megatron`
4747
可以使用以下命令构建镜像:
4848

4949
```bash
50-
docker build -f scripts/docker_for_megatron/Dockerfile -t trinity-rft-megatron:latest .
50+
docker build -f scripts/docker/Dockerfile.megatron -t trinity-rft-megatron:latest .
5151
```
5252

5353
> 💡 你可以在构建前自定义 Dockerfile —— 例如添加 pip 镜像源或设置 API 密钥。

docs/sphinx_doc/source_zh/tutorial/trinity_installation.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,7 @@
66
在安装前,请确保您的系统满足以下要求:
77

88
- **Python**:3.10 至 3.12(包含)
9-
- **CUDA**:大于等于 12.6
9+
- **CUDA**:大于等于 12.8
1010
- **GPU**:至少 2 块 GPU
1111

1212
---

pyproject.toml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
44

55
[project]
66
name = "trinity-rft"
7-
version = "0.3.2"
7+
version = "0.3.3"
88
authors = [
99
{name="Trinity-RFT Team", email="[email protected]"},
1010
]

scripts/docker/Dockerfile

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,4 @@
1+
# This Dockerfile sets up a Trinity-RFT environment with minimal support.
12
# Build and run the docker image with the following command:
23
#
34
# cd <Trinity-RFT root dir>
@@ -9,7 +10,7 @@ FROM nvcr.io/nvidia/cuda:12.8.1-cudnn-devel-ubuntu22.04
910

1011
WORKDIR /workspace
1112

12-
RUN apt update && apt install -y \
13+
RUN chmod 1777 /tmp && apt update && apt install -y \
1314
build-essential \
1415
curl git wget vim tmux net-tools \
1516
python3 python3-pip python3-dev python3-packaging \

scripts/docker_for_megatron/Dockerfile renamed to scripts/docker/Dockerfile.megatron

Lines changed: 5 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,15 +1,16 @@
1+
# This Dockerfile sets up a Trinity-RFT environment with Megatron-LM support.
12
# Build and run the docker image with the following command:
23
#
34
# cd <Trinity-RFT root dir>
4-
# docker build -f scripts/docker_for_megatron/Dockerfile -t trinity-rft-megatron:latest .
5+
# docker build -f scripts/docker/Dockerfile.megatron -t trinity-rft-megatron:latest .
56
# docker run -it --gpus all --shm-size="64g" --rm -v $PWD:/workspace -v <root_path_of_data_and_checkpoints>:/data trinity-rft-megatron:latest
67

78

89
FROM nvcr.io/nvidia/cuda:12.8.1-cudnn-devel-ubuntu22.04
910

1011
WORKDIR /workspace
1112

12-
RUN apt update && apt install -y \
13+
RUN chmod 1777 /tmp && apt update && apt install -y \
1314
build-essential \
1415
curl git wget vim tmux net-tools \
1516
python3 python3-pip python3-dev python3-packaging \
@@ -30,7 +31,8 @@ RUN pip install --upgrade pip \
3031
&& pip install -e .[mm,dev] \
3132
&& pip install flash_attn==2.8.1 --no-build-isolation \
3233
&& pip install -e .[megatron] \
33-
&& pip install -v --disable-pip-version-check --no-cache-dir --no-build-isolation \
34+
&& NVCC_APPEND_FLAGS="--threads 4" APEX_PARALLEL_BUILD=8 pip install -v \
35+
--disable-pip-version-check --no-cache-dir --no-build-isolation \
3436
--config-settings "--build-option=--cpp_ext" \
3537
--config-settings "--build-option=--cuda_ext" \
3638
--resume-retries 20 git+https://github.com/NVIDIA/apex.git

0 commit comments

Comments
 (0)