docs: add docs for how to create a working docker image (#60898)

Co-authored-by: David Gold <[email protected]>

Author: aaronsteers

docs/integrations/custom-connectors.md

@@ -25,11 +25,16 @@ It's easy to build your own connectors for Airbyte. You can learn how to build n
 While the guides in the link above are specific to the languages used most frequently to write integrations, **Airbyte connectors can be written in any language**. Please reach out to us if you'd like help developing connectors in other languages.
 
 :::caution
-If you don't use one of the official development options, remember to set the `AIRBYTE_ENTRYPOINT` environment variable to your Docker image's entrypoint!
-Otherwise, your connector will not run correctly.
-:::
+We strongly recommend creating new connectors using the [Connector Builder](https://docs.airbyte.com/connector-development/connector-builder-ui/overview) or one of the Airbyte CDKs.
+
+While it is not recommended to build Docker images from scratch, it can be useful to build a custom Docker image if you cannot use the Connector Builder, and if you need to use a language other than Python or CDK.
+
+See the following guides for more information on building custom Airbyte Docker images:
 
-Learn how to upload Docker-based custom connectors [here](../../platform/operator-guides/using-custom-connectors/)
+- [Using Custom Connectors](../platform/operator-guides/using-custom-connectors)
+- [Airbyte Docker Protocol](../platform/understanding-airbyte/airbyte-protocol-docker)
+
+:::
 
 ## Upgrading a connector
 

docs/platform/operator-guides/using-custom-connectors.md

@@ -36,6 +36,10 @@ To push and pull images to your private Docker registry, you need to authenticat
 - Your local or CI environment (where you build your connector image) must be able to **push** images to your registry.
 - Your Airbyte instance must be able to **pull** images from your registry.
 
+## 3. Adhere to Airbyte's Docker Image Requirements
+
+See the [Airbyte Protocol Docker Interface](../understanding-airbyte/airbyte-protocol-docker.md) page for specific Docker image requirements, such as required environment variables.
+
 ### For Docker-compose Airbyte deployments
 
 #### On GCP - Artifact Registry:
@@ -107,21 +111,24 @@ You can pull your connector image from your private registry to validate the pre
 
 5. `Add` the connector to save the configuration. You can now select your new connector when setting up a new connection!
 
-
 # Troubleshooting
 
 ## Loading connector docker containers into kind
+
 If you are running Airbyte in kind (kubernetes in docker -- this is the default method for abctl), you must load the docker image of that connector into the cluster. If you are seeing the following error, it likely means that the docker image has not been properly loaded into the cluster.
 
 ![Screenshot of UI error when custom connector container is not loaded in the cluster](./assets/custom-connector-error1.png)
 
 ![Screenshot of K8s error when custom connector container is not loaded in the cluster](./assets/custom-connector-error2.png)
 
 A connector container can be loaded using the following command:
-```
+
+```bash
 kind load docker-image <image-name>:<image-tag> -n airbyte-abctl
 ```
+
 For the example above, the command would be:
-```
+
+```bash
 kind load docker-image airbyte/source-custom:1 -n airbyte-abctl
 ```

docs/platform/understanding-airbyte/airbyte-protocol-docker.md

@@ -51,3 +51,29 @@ The `write` command will consume `AirbyteMessage`s from STDIN.
 - Connectors receive arguments on the command line via JSON files. `e.g. --catalog catalog.json`
 - They read `AirbyteMessage`s from STDIN. The destination `write` action is the only command that consumes `AirbyteMessage`s.
 - They emit `AirbyteMessage`s on STDOUT.
+
+## Additional Docker Image Requirements
+
+### Environment variable: `AIRBYTE_ENTRYPOINT` 
+
+The Docker image must contain an environment variable called `AIRBYTE_ENTRYPOINT`. This must be the same as the `ENTRYPOINT` of the image.
+
+## Non-Root User: `airbyte`
+
+The Docker image should run under a user named `airbyte`.
+
+## Specified `/airbyte` directory
+
+The Docker image must have a directory called `/airbyte`, which the user `airbyte` owns and can write to.
+
+This is the directory to which temporary files will be mounted, including the `config.json` and `catalog.json` files.
+
+## Only write file artifacts to directories permitted by the base image
+
+The connector code must only write to directories permitted within the connector's base image.
+
+For a list of permitted write directories, please consult the base image definitions in the [`airbytehq/airbyte` repo](https://github.com/airbytehq/airbyte), under the [`docker-images` directory](https://github.com/airbytehq/airbyte/tree/master/docker-images).
+
+## Must be an `amd64` or multi-arch image
+
+To run on Airbyte Platform, the image bust be valid for `amd64`. Since most developers contribute from (ARM-based) Mac M-series laptops, we recommend creating a multi-arch image that covers both `arm64/amd64` so that the same image tags work on both ARM and AMD runtimes.

docusaurus/platform_versioned_docs/version-1.6/operator-guides/using-custom-connectors.md

@@ -3,7 +3,7 @@ products: oss-*
 sidebar_label: Uploading custom connectors
 ---
 
-import ContainerProviders from '@site/static/\_docker_image_registries.md';
+import ContainerProviders from '@site/static/_docker_image_registries.md';
 
 # Uploading Docker-based custom connectors
 
@@ -36,6 +36,10 @@ To push and pull images to your private Docker registry, you need to authenticat
 - Your local or CI environment (where you build your connector image) must be able to **push** images to your registry.
 - Your Airbyte instance must be able to **pull** images from your registry.
 
+## 3. Adhere to Airbyte's Docker Image Requirements
+
+See the [Airbyte Protocol Docker Interface](../understanding-airbyte/airbyte-protocol-docker.md) page for specific Docker image requirements, such as required environment variables.
+
 ### For Docker-compose Airbyte deployments
 
 #### On GCP - Artifact Registry:
@@ -119,12 +123,12 @@ If you are running Airbyte in kind (kubernetes in docker -- this is the default
 
 A connector container can be loaded using the following command:
 
-```
+```bash
 kind load docker-image <image-name>:<image-tag> -n airbyte-abctl
 ```
 
 For the example above, the command would be:
 
-```
+```bash
 kind load docker-image airbyte/source-custom:1 -n airbyte-abctl
 ```

docusaurus/platform_versioned_docs/version-1.6/understanding-airbyte/airbyte-protocol-docker.md

@@ -51,3 +51,29 @@ The `write` command will consume `AirbyteMessage`s from STDIN.
 - Connectors receive arguments on the command line via JSON files. `e.g. --catalog catalog.json`
 - They read `AirbyteMessage`s from STDIN. The destination `write` action is the only command that consumes `AirbyteMessage`s.
 - They emit `AirbyteMessage`s on STDOUT.
+
+## Additional Docker Image Requirements
+
+### Environment variable: `AIRBYTE_ENTRYPOINT` 
+
+The Docker image must contain an environment variable called `AIRBYTE_ENTRYPOINT`. This must be the same as the `ENTRYPOINT` of the image.
+
+## Non-Root User: `airbyte`
+
+The Docker image should run under a user named `airbyte`.
+
+## Specified `/airbyte` directory
+
+The Docker image must have a directory called `/airbyte`, which the user `airbyte` owns and can write to.
+
+This is the directory to which temporary files will be mounted, including the `config.json` and `catalog.json` files.
+
+## Only write file artifacts to directories permitted by the base image
+
+The connector code must only write only to directories permitted within the connector's base image.
+
+For a list of permitted write directories, please consult the base image definitions in the [`airbytehq/airbyte` repo](https://github.com/airbytehq/airbyte), under the [`docker-images` directory](https://github.com/airbytehq/airbyte/tree/master/docker-images).
+
+## Must be an `amd64` or multi-arch image
+
+To run on Airbyte Platform, the image bust be valid for `amd64`. Since most developers contribute from (ARM-based) Mac M-series laptops, we recommend creating a multi-arch image that covers both `arm64/amd64` so that the same image tags work on both ARM and AMD runtimes.