diff --git a/models/automations.mdx b/models/automations.mdx index 4dc1e9faaa..c3601925f8 100644 --- a/models/automations.mdx +++ b/models/automations.mdx @@ -8,31 +8,42 @@ import EnterpriseCloudOnly from "/snippets/en/_includes/enterprise-cloud-only.md -This page describes _automations_ in W&B. [Create an automation](/models/automations/create-automations/) to trigger workflow steps, such as automated model testing and deployment, based on an event in W&B, such as when an [artifact](/models/artifacts/) artifact version is created or when a [run metric](/models/runs/) meets or changes by a threshold. +This page describes _automations_ in W&B. [Create an automation](/models/automations/create-automations/) to trigger workflow steps, such as automated model testing and deployment, based on an event in W&B. For example, an automation can notify a Slack channel when a new version is created, trigger an automated testing webhook when the `production` alias is added to an artifact, or start a validation job only when a run's `loss` is within acceptable bounds. -## Overview -An automation can start when a specific [event](/models/automations/automation-events/) occurs in a registry or project. + +Looking for companion tutorials for automations? +- [Learn to automatically trigger a Github Action for model evaluation and deployment](https://wandb.ai/wandb/wandb-model-cicd/reports/Model-CI-CD-with-W-B--Vmlldzo0OTcwNDQw). +- [Watch a video demonstrating automatically deploying a model to a Sagemaker endpoint](https://www.youtube.com/watch?v=s5CMj_w3DaQ). +- [Watch a video series introducing automations](https://youtube.com/playlist?list=PLD80i8An1OEGECFPgY-HPCNjXgGu-qGO6&feature=shared). + -In a [Registry](/models/registry/), an automation can start: -- When a new artifact version is linked to a collection. For example, trigger testing and validation workflows for new candidate models. -- When an alias is added to an artifact version. For example, trigger a deployment workflow when an alias is added to a model version. +## Automation events -In a [project](/models/track/project-page/), an automation can start: -- When a new version is added to an artifact. For example, start a training job when a new version of a dataset artifact is added to a given collection. -- When an alias is added to an artifact version. For example, trigger a PII redaction workflow when the alias "redaction" is added to a dataset artifact. -- When a tag is added to an artifact version. For example, trigger a geo-specific workflow when the tag "europe" is added to an artifact version. -- When a metric for a run meets or exceeds a configured threshold. -- When a metric for a run changes by a configured threshold. -- When a run's status changes to **Running**, **Failed**, or **Finished**. +Automations can be triggered by events related to runs or artifacts in a collection, project, or registry. An automation can start: -Optionally, you can: -- Filter artifacts, aliases, or tags by name -- Filter runs by user or run name. +- When a run metric meets a defined absolute or relative threshold. For example, trigger when a run's `loss` is less than 0.01 or when a run's `accuracy` improves by 5%. +- When an event occurs in a project or registry. For example, trigger when a new version of a model artifact is created or when the `production` alias is added to a model artifact. For more details, see [Automation events and scopes](/models/automations/automation-events/). +## Automation actions + +When an event triggers an automation, it can perform one of these actions: + +### Slack notification +Send a message to a Slack channel with details about the triggering event. The message summarizes the event, with a link to view more details in W&B. + +### Webhook +Call a webhook URL with a JSON payload containing information about the triggering event. This enables integration with external systems like CI/CD pipelines, model deployment services, or custom workflows. The body of the webhook request may be any JSON-serializable payload. + +For implementation details, see: +- [Create a Slack automation](/models/automations/create-automations/slack/) +- [Create a webhook automation](/models/automations/create-automations/webhook/) + +## How automations work + To [create an automation](/models/automations/create-automations/), you: 1. If required, configure [secrets](/platform/secrets/) for sensitive strings the automation requires, such as access tokens, passwords, or sensitive configuration details. Secrets are defined in your **Team Settings**. Secrets are most commonly used in webhook automations to securely pass credentials or tokens to the webhook's external service without exposing it in plain text or hard-coding it in the webhook's payload. diff --git a/models/runs/alert.mdx b/models/runs/alert.mdx index 7e922d1741..b733b63902 100644 --- a/models/runs/alert.mdx +++ b/models/runs/alert.mdx @@ -22,16 +22,11 @@ For example, an automation can notify a Slack channel when a new version is crea Read the [Automations overview](/models/automations/) or [create an automation](/models/automations/create-automations/). - - -## Create an alert - The following guide only applies to alerts in Multi-tenant Cloud. If you're using [W&B Server](/platform/hosting/) in your Private Cloud or on W&B Dedicated Cloud, refer to [Configure Slack alerts in W&B Server](/platform/hosting/monitoring-usage/slack-alerts) to set up Slack alerts. - To set up an alert, take these steps, which are detailed in the following sections: 1. Turn on Alerts in your W&B [User Settings](https://wandb.ai/settings).