Agreement on how to write connection parameters

[pierluigilenoci]

There is a "small" issue that makes the UpJet AWS provider unusable in conjunction with the SQL provider.
This is because there is a discrepancy between the way the UpJet provider writes credentials and how the SQL provider reads them.
Could the maintainers of both providers discuss and find a solution to unblock all users who would like to create databases and easily execute queries?
List of issues to help you analyze the problem:

• MySQL Instance fills out endpoint field in ConnectionSecret in an unexpected way, breaking crossplane-contrib/sql-provider provider-upjet-aws#778
• Return RDS Instance Endpoint field in the ConnectionSecret without port provider-upjet-aws#798
• Allow writeConnectionSecretToRef to be compatible with SQL community provider provider-upjet-aws#1189
• Agreement on how to write connection parameters provider-upjet-aws#1578
• Remove port from endpoint (if added) #154
• Adds port twice when using connectionStrings from upbound/provider-aws-rds #191

[pierluigilenoci]

The follow-up discussion can also be found here: https://crossplane.slack.com/archives/C01718T2476/p1732801643245679.

[Duologic]

Thanks for brining this up @pierluigilenoci

This was also discussed on the linked Slack thread but sharing here for posterity.

I've never seen this as a big issue as it was never a blocker for me, I've been using these through Compositions, which allow me to map address to endpoint.

Like this (ref):

connectionDetails:
    - fromConnectionSecretKey: username
      name: username
    - fromConnectionSecretKey: attribute.password
      name: password
    - fromConnectionSecretKey: address
      name: endpoint
    - fromConnectionSecretKey: port
      name: port

I remember vaguely that the provider for GCP supported the consistent endpoint/port/username/password but that doesn't seem to be the case for the Upjet provider. The secret from provider-upjet-gcp's CloudSQL connectionDetails does not match in any way that could possibly work with provider-sql without rewriting the secret (with a Composition for example).

The 'vanilla' provider-gcp did return the consistent secrets, hence where I remember that from, as well as the 'vanilla' provider-aws. You can see here that provider-aws maps endpoint.address to endpoint: https://github.com/crossplane-contrib/provider-aws/blob/master/pkg/controller/rds/dbinstance/setup.go#L233

So to be very pedantic, this was a regression introduced by provider-upjet-aws. 😇

Again, as also mentioned by someone else on the Slack thread, this can be resolved with a Composition, which would also be required for CloudSQL in provider-upjet-gcp anyways

I don't think the provider-sql should accomodate for each CSP's generated secret, it creates a maintenance burden to follow multiple upstream providers, as well as multiple versions of that provider in case the connection details change upon a breaking change/major version release.

[beck3905]

@Duologic I just found this thread as I am running into this issue as well trying to use the provider-aws-family and provider-sql together. Could you provide an example of a workaround using a composition please? Also, what if the ProviderConfig offered additional parameters to chose which keys in the secret to reference?

[erickfaustino]

Hi!
I have hit the same problem. When I switched to upjet aws providers, it broke my ability to connect and create user/roles/grants and DBs.

[Duologic]

The connectionDetails in my previous comment is the bit that does the mapping, also have a look here for a more comprehensive description to propagate connection details: crossplane/crossplane#2787

[pierluigilenoci]