OPAL Agent does not receive data source configuration after scope update #779
Description
Describe the bug
When an OPAL Agent is configured to use scopes and connects to an OPAL Server without any scopes being configured, it receives the default data and policy source configuration, which is expected but not useful. However, after configuring a scope via the OPAL Server /scopes endpoint - including both policy and data source configurations - the OPAL Agent only receives the policy source configuration for its scope, the data source configuration is however not propagated, causing all OPA authorization requests to fail due to missing data.
To Reproduce
- Start an OPAL Server and an OPAL Agent configured to use scopes.
- Ensure no scopes are initially configured on the OPAL Server.
- Configure a scope via the
/scopesendpoint, including both a policy source and a data source. - Observe that the OPAL Agent makes use of the configured policy source but does not receive the new data source configuration.
- Make an authorization request to the OPA instance managed by the Agent and note that it results in an unauthorized response due to missing data. Alternatively one can call the
/v1/dataendpoint of OPA and observe, that there are no entries related to data which should actually have been fetched from the above said data sources.
Here the excerpt forom the docker compose yaml file related to agent and server config:
opal_server:
image: permitio/opal-server:0.8.0
environment:
- OPAL_BROADCAST_URI=redis://redis:6379
- OPAL_REDIS_URL=redis://redis:6379
- UVICORN_NUM_WORKERS=1
- OPAL_LOG_FORMAT_INCLUDE_PID=true
- OPAL_LOG_LEVEL=DEBUG
- OPAL_SCOPES=1
- OPAL_POLICY_REFRESH_INTERVAL=30
- UVICORN_LOG_LEVEL=debug
ports:
- "7002:7002"
opal_client:
build:
context: ./opal
dockerfile: ./Dockerfile # uses permitio/opal-client:0.8.0 and adds postgres fetcher
environment:
- OPAL_SERVER_URL=http://opal_server:7002
- OPAL_LOG_FORMAT_INCLUDE_PID=true
- OPAL_LOG_LEVEL=DEBUG
- OPAL_INLINE_OPA_LOG_FORMAT=http
- OPAL_SCOPE_ID=documents
- OPAL_DATA_TOPICS=data
- OPAL_FETCH_PROVIDER_MODULES=opal_common.fetcher.providers,opal_fetcher_postgres.providerAnd here the document sent to the /scopes endpoint after both, the agent and the server are started:
{
"scope_id": "documents",
"policy": {
"source_type": "git",
"url": "https://<edited>/poc_testenvironment",
"auth": {
"auth_type": "github_token",
"token": "<edited>"
},
"directories": [ "policies" ],
"extensions": [ ".rego" ],
"bundle_ignore": [ "*_test.rego" ],
"poll_updates": true,
"branch": "<edited>"
},
"data": {
"entries": [
{
"periodic_update_interval": 30,
"url":"http://caddy:8080/users.json",
"topics": ["documents:data:data"],
"dst_path":"/users"
},
{
"periodic_update_interval": 30,
"url":"http://caddy:8080/organizations.json",
"topics": ["documents:data:data"],
"dst_path":"/organizations"
},
{
"periodic_update_interval": 30,
"url":"http://caddy:8080/departments.json",
"topics": ["documents:data:data"],
"dst_path":"/departments"
},
{
"url": "postgresql://documents@db:5432/documents",
"config": {
"fetcher": "PostgresFetchProvider",
"query": "select id::text, concat('ip:', ip) as parent_id from document;",
"connection_params": {
"password": "documents"
},
"fetch_key": "id"
},
"topics": ["documents:data:data"],
"dst_path": "documents"
}
]
}
}Expected behavior
When a scope is configured via the /scopes endpoint, the OPAL Agent should receive both the policy source and data source configurations.
OPAL version
- Version: 0.8.0