Skip to content

Commit 4cfe457

Browse files
adoroszlaiadoroszlai
committed
HDDS-12441. Add helper script for building Docker image (#39)
1 parent 15ee476 commit 4cfe457

File tree

4 files changed

+102
-42
lines changed

4 files changed

+102
-42
lines changed

‎.dockerignore‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
.asf.yaml
2+
build.sh
3+
CONTRIBUTING.md
4+
.git
5+
.github
6+
LICENSE
7+
NOTICE
8+
README.md
9+
SECURITY.md

‎CONTRIBUTING.md‎

Lines changed: 70 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,73 @@
1+
<!--
2+
Licensed to the Apache Software Foundation (ASF) under one or more
3+
contributor license agreements. See the NOTICE file distributed with
4+
this work for additional information regarding copyright ownership.
5+
The ASF licenses this file to You under the Apache License, Version 2.0
6+
(the "License"); you may not use this file except in compliance with
7+
the License. You may obtain a copy of the License at
8+
9+
http://www.apache.org/licenses/LICENSE-2.0
10+
11+
Unless required by applicable law or agreed to in writing, software
12+
distributed under the License is distributed on an "AS IS" BASIS,
13+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
14+
See the License for the specific language governing permissions and
15+
limitations under the License.
16+
-->
17+
118
# Contributing
219

3-
For detailed contribution guideline, please check the [contribution guideline of Apache Ozone repository](https://github.com/apache/ozone/blob/master/CONTRIBUTING.md).
20+
For general contribution guideline, please check the [Apache Ozone repository](https://github.com/apache/ozone/blob/master/CONTRIBUTING.md).
21+
22+
Development of the `ozone-runner` image happens on branch `master`, relevant changes are cherry-picked to branch `jdk11`.
23+
24+
## Local Build and Test
25+
26+
### Building
27+
28+
The image can be built simply by running the helper script `build.sh`:
29+
30+
```bash
31+
$ ./build.sh
32+
...
33+
=> => naming to docker.io/apache/ozone-runner:dev
34+
```
35+
36+
This will create a single-platform image for your architecture.
37+
38+
### Testing
39+
40+
To try the image locally with Ozone acceptance tests, define the version to be used:
41+
42+
```bash
43+
export OZONE_RUNNER_VERSION=dev
44+
```
45+
46+
then run [acceptance tests](https://github.com/apache/ozone/blob/master/hadoop-ozone/dist/src/main/compose/README.md) as needed.
47+
48+
## GitHub Workflows
49+
50+
If this is your first time working on the image, please enable GitHub Actions workflows after forking the repo.
51+
52+
### Building
53+
54+
Whenever changes are pushed to your fork, GitHub builds a multi-platform image (for `amd64` and `arm64`), and tags it with the commit SHA. These images can be shared with other developers for feedback. Workflow runs are listed at `https://github.com/<username>/ozone-docker-runner/actions`, images at `https://github.com/<username>/ozone-docker-runner/pkgs/container/ozone-runner`.
55+
56+
### Testing
57+
58+
To run complete Ozone CI with the custom image:
59+
60+
1. Create a new branch in your clone of `apache/ozone`.
61+
2. Update `OZONE_RUNNER_IMAGE` to `ghrc.io/<username>/ozone-runner` in [ci.yml](https://github.com/apache/ozone/blob/bb16f66e22c44b360d42d0cae87024786e27c61b/.github/workflows/ci.yml#L37).
62+
3. Update `docker.ozone-runner.version` to `<commit SHA>` in [hadoop-ozone/dist/pom.xml](https://github.com/apache/ozone/blob/bb16f66e22c44b360d42d0cae87024786e27c61b/hadoop-ozone/dist/pom.xml#L28).
63+
4. Commit the change and push to your fork of `apache/ozone`.
64+
65+
## Publishing Docker Tags (for committers)
466

67+
1. Fetch changes to your local clone.
68+
2. Add a Git tag for the commit following the existing pattern `<date>-<n>-<flavor>`, where
69+
- `<n>` starts at 1, and is incremented if multiple images need to be published the same day)
70+
- `<flavor>` is `jdk21` or `jdk11`
71+
3. Push the Git tag to the official repo (`apache/ozone-docker-runner`). This will trigger a workflow to apply the tag to the Docker image.
72+
4. If the change is relevant for Ozone 1.x versions, cherry-pick it to the `jdk11` branch.
73+
5. Set `Fix Version` of the Jira issue to `runner-<date>-<n>-<flavor>`

‎README.md‎

Lines changed: 4 additions & 41 deletions
Original file line numberDiff line numberDiff line change
@@ -15,47 +15,10 @@
1515
limitations under the License.
1616
-->
1717

18-
# Apache Ozone **runner** base image
18+
# Apache Ozone Runner Base Image
1919

20-
This is the base image to run Apache Hadoop Ozone in docker containers. This is only for test/develop and not for production.
20+
[ozone-runner](https://github.com/apache/ozone-docker-runner) is the base image with tools for running and testing Apache Ozone, but does not include any Ozone artifacts.
2121

22-
The container doesn't include any Ozone specific jar files or release artifacts just an empty environment which includes all the specific tools to run and test Apache Ozone inside containers.
22+
Published to [Docker Hub](https://hub.docker.com/r/apache/ozone-runner) and [GitHub](https://github.com/apache/ozone-docker-runner/pkgs/container/ozone-runner).
2323

24-
The image is available as [apache/ozone-runner](https://hub.docker.com/r/apache/ozone-runner). Build is managed by Docker Hub.
25-
26-
## Development
27-
28-
To build the image, please use:
29-
30-
```
31-
DOCKER_BUILDKIT=1 docker build -t apache/ozone-runner:dev .
32-
```
33-
34-
To test it, build [Apache Ozone](https://github.com/apache/ozone):
35-
36-
```
37-
mvn clean verify -DskipTests -Dskip.npx -DskipShade -Ddocker.ozone-runner.version=dev
38-
```
39-
40-
And start the compose cluster:
41-
42-
```
43-
cd hadoop-ozone/dist/target/ozone-*/compose/ozone
44-
docker-compose up -d
45-
```
46-
47-
*After merging PR, a new tag should pushed to the repository to create a new image. Use the convention: `YYYYMMDD-N` for tags where N is a daily counter (see the existing tags as an example).
48-
49-
After tag is published (and built by Docker Hub), the used runner version can be updated by modifying the `docker.ozone-runner.version` version in [hadoop-ozone/dist/pom.xml](https://github.com/apache/ozone/blob/master/hadoop-ozone/dist/pom.xml)
50-
51-
## Building multi-architecture images
52-
53-
To build images with multiple architectures, use `docker buildx`.
54-
55-
For example, to build images for both `linux/amd64` and `linux/arm64`, run:
56-
57-
```bash
58-
docker buildx build --platform linux/amd64,linux/arm64 -t apache/ozone-runner:dev . --progress=plain
59-
```
60-
61-
It might be slow when building the non-native architecture image due to QEMU emulation.
24+
Images are tagged by date, and come in two flavors: `jdk21` (for Ozone 2.0+) and `jdk11` (for Ozone 1.x).

‎build.sh‎

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
#!/usr/bin/env bash
2+
# Licensed to the Apache Software Foundation (ASF) under one or more
3+
# contributor license agreements. See the NOTICE file distributed with
4+
# this work for additional information regarding copyright ownership.
5+
# The ASF licenses this file to You under the Apache License, Version 2.0
6+
# (the "License"); you may not use this file except in compliance with
7+
# the License. You may obtain a copy of the License at
8+
#
9+
# http://www.apache.org/licenses/LICENSE-2.0
10+
#
11+
# Unless required by applicable law or agreed to in writing, software
12+
# distributed under the License is distributed on an "AS IS" BASIS,
13+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
14+
# See the License for the specific language governing permissions and
15+
# limitations under the License.
16+
17+
set -eu -o pipefail
18+
19+
docker build -t apache/ozone-runner:dev "$@" .

0 commit comments

Comments
 (0)