docs: explain configuration of Push Datasources

[torwunder]

Problem description

The current implementation seems to support push datasources by providing endpoints to push data into OCP. (https://www.openpolicyagent.org/docs/ocp/api-reference#postput-v1sourcessourcedatapath).

While this works it is not obvious from the documentation how such a datasource can be configured either via config file or API.
The only supported datasource type seems to be http.

If the datasource type is left empty or provided with a different value the bundle build fails.
The datasource directory is not created and still required by the build.

/data
|- <uuid>
    |- sources
        |- <source name>
            |- database 
            |   |- <datasource name>
            |       |- data.json
            |- datasources     <-- missing if datasource type http is not provided         

If the datasources folder is added manually the bundle build is successful.

Ideally a second datasource type is provided similar to JSON/REST in Styra DAS.

[srenatus]

Currently, any source comes with static data that can be updated using PUT /v1/sources/<sourcename>/data/some/path/here.

If the datasource type is left empty or provided with a different value the bundle build fails.
The datasource directory is not created and still required by the build.

Can you please share your config or a reproducible example? I'm not sure I understand how I could trigger this scenario...

[srenatus]

Here's an example:

ocp.yaml:

  admin-user:
    api_key: sesame
    scopes:
    - role: administrator
database:
  sql:
    driver: sqlite
    dsn: sqlite.db
bundles:
  hello-world:
    object_storage:
      filesystem:
        path: bundles/hello-world/bundle.tar.gz
    requirements:
    - source: my-push-source
sources:
  my-push-source: {} # empty definition, basically

Started via opactl run -c ocp.yaml --log-level debug.

The, we'll update the data source, and wait a moment (see debug logs), then unpack the produced bundle:

$ curl -H "Authorization: bearer sesame" http://127.0.0.1:8282/v1/sources/my-push-source/data/foo/bar/baz -d '{"hello":"there"}'
{}
# wait a moment
$ tar xvf bundles/hello-world/bundle.tar.gz /data.json
tar: Removing leading '/' from member names
x data.json
$ jq < data.json
{
  "foo": {
    "bar": {
      "baz": {
        "hello": "there"
      }
    }
  }
}

[torwunder]

I see, I used a definition similar like this:

[..]
sources:
  shared-datasource:
    datasources:
      - name: shared-data
        path: shared-data
bundles:
  data-bundle:
    requirements:
      - source: shared-datasource
    object_storage:
      filesystem:
        path: bundles/data-bundle.tgz
[..]

And then pushed data like this:

PUT http://localhost:8282/v1/sources/shared-datasource/data/shared-data
Authorization: Bearer ....
Content-Type: application/json

{
  "key": "value",
  "key2": "value2"
}

This produced the following error: {"time":"2025-10-29T09:05:19.720255+01:00","level":"WARN","msg":"failed to build a bundle \"data-bundle\": shared-datasource: open .: no such file or directory"}

I think my error came from the assumption that a datasource needs to be in place to push data.
I tried to leave the source config empty but this produced an error that in your example was fixed with {}.

I did not find documentation on how to configure these kind of push sources but might have overlooked it.

[srenatus]

I'm afraid we're still fresh in terms of documentation 😅 I think we'll need better end-to-end examples, for one thing. I'm glad you've got this figured out, though.

I'll convert this to a task about better docs for the push data source, if you don't mind.