Skip to content

Project Proposal: Audit Logging SIG #2409

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 25 commits into
base: main
Choose a base branch
from
+213 −0
Open

Project Proposal: Audit Logging SIG #2409

Changes from 3 commits
Commits
Show all changes
25 commits
Select commit Hold shift + click to select a range
5094fb1
docs(auditlogs): add audit logging sig proposal
mlenkeit Oct 24, 2024
9337b7f
docs(auditlogs): re-number requirements
mlenkeit Oct 24, 2024
75f2c57
docs(auditlogs): remove template instructions
mlenkeit Oct 24, 2024
f81c2f4
docs(auditlogs): use OTel over OTEL
mlenkeit Nov 19, 2024
65ae32e
docs(auditlogs): list @reyang as first sponsor
mlenkeit Nov 19, 2024
776b821
docs(auditlogs): add Microsoft to interested vendors
mlenkeit Nov 19, 2024
6dd519d
docs(auditlogs): add contacts to vendor list
mlenkeit Nov 19, 2024
2ec002d
docs(auditlogs): use consistent punctuation for requirement list
mlenkeit Nov 19, 2024
d7e265f
docs(auditlogs): minor word change in Challenges chapter
mlenkeit Nov 20, 2024
405ddb5
docs(auditlogs): describe guarantee of delivery in appendix
mlenkeit Nov 21, 2024
0adb8e5
docs(auditlogs): add sample audit logs to appendix
mlenkeit Nov 21, 2024
a5ef343
docs(auditlogs): add links to sample audit logs
mlenkeit Nov 21, 2024
711dc46
docs(auditlogs): add links to appendix A
mlenkeit Nov 21, 2024
087865c
docs(auditlogs): use GitHub handle only in staffing list
mlenkeit Nov 21, 2024
3876a31
docs(auditlogs): add svrnm as GC liaison
mlenkeit Nov 21, 2024
8b38626
Merge branch 'audit-logging-sig-project-proposal' of github.com:apeir…
mlenkeit Nov 21, 2024
066501b
docs(auditlogs): minor changes in wording
mlenkeit Nov 22, 2024
70cbac4
docs(auditlogs): shorten requirement ids to pass spell check
mlenkeit Nov 22, 2024
a6b34f1
Merge branch 'main' into audit-logging-sig-project-proposal
mlenkeit Nov 25, 2024
6bc9a5e
docs(auditlogs): propose project lead
mlenkeit Mar 27, 2025
e03dcd8
Merge branch 'main' into audit-logging-sig-project-proposal
mlenkeit Mar 27, 2025
86bd77a
Merge remote-tracking branch 'upstream/main' into audit-logging-sig-p…
mlenkeit May 20, 2025
41329f3
* docs(auditlogs): describe phases approach
mlenkeit May 20, 2025
03d1de1
Add files via upload
hilmarf Sep 19, 2025
2f9813f
Revert "Add files via upload"
hilmarf Sep 19, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
68 changes: 68 additions & 0 deletions projects/audit-logging.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
# Audit Logging

## Background and description

Audit logging describes the capability of capturing audit-trail relevant events of a system to meet compliance requirements. Such events may originate from the infrastructure (e.g. a Kubernetes cluster) up to the application-level. It is a capability that is particularly relevant for providers of enterprise software.

Unlike regular application logs, audit logs are usually subject to long retention periods and software providers must guarantee their completeness (i.e. guarantee of delivery).
Copy link
Member

@reyang reyang Oct 31, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good points! In addition, these are something we might want to consider:

  1. Audit logs might be considered as a critical part of the business, which could result in a different API design strategy - for example, audit logging might require a different API behavior, if the information provided by the caller is invalid, the API might throw exception instead of failing silently and move on.
  2. Audit log might require some sensitive information without redaction due to the regulation requirements (e.g. user identity and client IP address).
  3. The data path could require higher level of access control or privilege.

Copy link
Author

@mlenkeit mlenkeit Nov 19, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@reyang thanks for mentioning these points.

Especially the API behavior is something that we had thought about initially. However, when we first pitched audit logging on Slack, we received the following comment from Ted Young:

As a rule, the OpenTelemetry API never throws an exception. I understand why you might want this, though it is not present in many audit logging systems, which use regular loggers. So a strong case would have to be made on this particular point.

Based on this initial feedback, we decided to file this SIG proposal without proposing such API changes.


Examples of audit logs include:
- permission changes (e.g. of a service account or application user)
- modification of data
- accessing sensitive information
- failed login attempts

### Current challenges

Audit Logging is currently not within the scope of OpenTelemetry

- no semantic conventions for audit logs in OTEL
Copy link
Member

@mtwo mtwo Oct 25, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- no semantic conventions for audit logs in OTEL
- There aren't currently any semantic conventions designed specifically for audit logs in OTEL

- OTEL APIs/SDKs do not provide feedback to the application level whether data (in particular logs) have been successfully delivered to a remote endpoint. To guarantee delivery, either the SDK has to give those guarantees, or provide feedback to the application so that it can take care of guaranteed delivery itself.
- OTEL collectors may lose audit logs in transit (i.e. no guarantee of delivery)

### Goals, objectives, and requirements

The goal of this project is to make OTEL fit for audit logging purposes that meet compliance requirements of enterprise software providers, in particular:

- REQ-CONV-01: Semantic conventions for application-level audit logs are defined
- REQ-CONV-02: Semantic conventions for infrastructure-level audit logs are defined
- REQ-APPL-01: Guaranteed delivery of audit logs exported via OpenTelemetry SDK.
- REQ-PIPE-01: OTEL collector must provide guaranteed delivery of audit logs, including when its process is interrupted

## Deliverables

- semantic convention for audit logs
- extend OTEL APIs/SDKs for audit logging purposes (in collaboration with the respective SIG)
- extend OTEL collector for audit logging purposes (in collaboration with the respective SIG)

## Staffing / Help Wanted

The following vendors are interested in improving this area:
- SAP

Other vendors are invited to join the discussion.

### Required staffing

* Project lead: SAP (name tbd)
* Sponsors: tbd
* GC liaison: tbd
* Engineers:
* SAP will provide a prototype in two languages (tbd; likely two of Java, JavaScript, Go)
Copy link
Member

@reyang reyang Oct 31, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we need prototype in two parts:

  1. API/SDK - this is where we need three programming languages IIRC.
  2. OTel Collector - higher guarantee on data delivery (completeness, integrity, latency, etc.), data path security.

Copy link
Author

@mlenkeit mlenkeit Nov 19, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for pointing this out! It's clear to us, but I'll work on making this clearer in the doc...

* Maintainers/approvers: tbd

## Timeline

TBD based on community involvement.

## Labels

- audit-logging (tbc)

## Project Board

TODO: add link

## SIG Meetings and Other Info

TODO: add information
Loading