Service Instance Generic Extensions/Actions (Take 2)

[Samze]

Why

The specification defines endpoints that allow the lifecycle management of service instances and service bindings. However, a common complaint is that it does not support some of those important “day 2” operations that developers might want, e.g backup and restore. Also that it does not allow service specific operations, e.g. MySQL set leader. To accomplish this Service Brokers authors have the option to either go off-spec or to misuse the spec (e.g. cf update-service -p ‘{“trigger-backup”: true}’).

The specification needs an extension mechanism to allow authors to define new endpoints.

There was a previous PR attempt to provide this functionality through “generic extensions” which was opened in 26 Jan 2018. This PR proved complex to implement in part extensions could be hosted anywhere and it allowed any authentication via OpenAPI security schemas.

What

This is our first attempt to revive generic extensions!

This PR is similar to the previous PR in that:

• It provides extensions that act on a service instance.
• It uses OpenAPI to describe the extension.
• It has an URN for each extension. The hope is services share extension specs and common tooling can be created.

It differs in that:

• The extensions are discoverable from the catalog - allowing users to know what extensions are available before provisioning an instance.
• All extensions must be hosted on the service broker itself (although the service broker may act as a proxy if it likes). Since platforms already talk to brokers this simplifies things, e.g. networking issues.
• All extensions must have the same authentication method already used between platform and broker. This makes it easier to implement both platforms and broker.

In order to not limit platform implementations we have not provided guidance on HOW the extension is triggered. Since the endpoint is hosted by the broker it is likely that platform makes a request to trigger the extension on the user’s behalf. This is the same as how all endpoints are called today from known platforms.

Open questions:

• Should the OpenAPI document be required? OpenAPI allows clients to discover how to consume the endpoints. There are tools that allow for auto generated clients/UX. However it is unknown whether this would be used. Will the people who write tools for the extensions just know how the extension works?
• Is it confusing that the OpenAPI paths live on a base route?
• Should we be more opinionated about how the extension gets trigger?
• Should extensions description be optional?
• Do we need to be explicit about how brokers do async operations using this model?

Notes

• We have not updated the OSBAPI OpenAPI yet for this PR since we wanted to get feedback first.

This PR came from this doc.

Sam & @georgi-lozev

(If this is unmerged in 2025, please close)

Closes #114

[cfdreddbot]

✅ Hey Samze! The commit authors and yourself have already signed the CLA.

[fmui]

Why is the path part of the ID? I think the extension ID should be something globally unique to distinguish - let's say - two different backup and restore specifications.

Here are a few opinions:

• The OpenAPI URL should be strongly recommended, but not required.
• Because the broker credentials are used for authentication, only the platform can call extensions. I don't see reason to call that out explicitly.
• The description should also be recommended but can be optional if the extension ID is globally unique.
• The async behaviour depends on the extension. We shouldn't force a specific async flow.

[jberkhahn]

shouldn't this be and a NAMESPACE specific string(NSS) (emphasis mine)? what namespace is this in reference to exactly?

[georgi-lozev]

Yes it could be if we think it will makes it more clear.
It's a reference to the URN syntax and the parts it consist of. https://tools.ietf.org/html/rfc2141#section-2

[jberkhahn]

just writing down some concerns from the OSB meeting:

• is this ID intended to be unique PER BROKER, or globally unique among ALL BROKERS when pointing to a specific extension? (i.e. all extensions that point http://example.com/extensions/backup_restore.yaml on ALL brokers are intended to be named the same thing on all brokers?)

[Samze]

We have two main reasons for an identifier:

The first is to support the use-case of the community coming up with a standard extensions and perhaps writing platform specific tooling for these extensions. Brokers may then want to opt-in to that already defined extension.

Then we have a separate requirement that a broker itself needs to be able to namespace extensions that it provides, say a broker wants to provide two backup and restore options and the openapi documents have colliding paths. We can use the urn as part of the broker provided path to trigger the extension.

We were hoping that having all extensions provide a globally unique urn would solve both of these problems, so I imagined that the ID (URN) was globally unique. I think perhaps our example here is misleading.

As for the structure of the URN this is something I'm not too sure about. I imagine the domain should be in there e.g. urn:osbext:pivotal.io/v1/mysqlbackups or urn:osbext:openservicebrokerapi.org/v1/community-back. Open to ideas around what this should look like and if URNs are the right thing here.

[fmui]

I also think the recommendation should be something like urn:osbext:<domain>/<some string> . But whatever we do, all the examples in the spec should follow the recommended pattern.

[Samze]

I've changed all examples to follow the pattern: urn:osbext:some-domain:some-version

[mattmcneeney]

Two outstanding things for this PR:

1. How should we deal with the identifier for the extension? @fmui please could you add your thoughts?
2. MUST service authors define an OpenAPI document for an extension or can this happen out-of-band? Do you have use cases for why this shouldn't be required @fmui - is the intention to keep things as simple as possible? How would a platform know what HTTP request to make?

[fmui]

I would expect the extension ID to be globally unique and not having the extension base path in it.
For example, if I see the extension urn:osbext:pivotal.io:mysqlbackups:v1, I know that the broker implements Pivotals MySQL backup. If that is a well-known extension, the platform might not even need the OpenAPI document for this extension (although I would strongly recommend always providing an OpenAPI document). The extension base path would then need to go into another field.
For example:

{
  "id": "urn:osbext:pivotal.io:mysqlbackups:v1",
  "path": "/p-backup",
  "openapi_url": "https://pivotal.io/osb-extensions/backup_restore_v1.yaml"
}

If the broker also supports another backup extension, it is free to set a different path for that extension.

[Samze]

I would expect the extension ID to be globally unique and not having the extension base path in it.
For example, if I see the extension urn:osbext:pivotal.io:mysqlbackups:v1, I know that the broker implements Pivotals MySQL backup. If that is a well-known extension, the platform might not even need the OpenAPI document for this extension (although I would strongly recommend always providing an OpenAPI document). The extension base path would then need to go into another field.
For example:

{
  "id": "urn:osbext:pivotal.io:mysqlbackups:v1",
  "path": "/p-backup",
  "openapi_url": "https://pivotal.io/osb-extensions/backup_restore_v1.yaml"
}

If the broker also supports another backup extension, it is free to set a different path for that extension.

The current proposal specifies the extension path implicitly from the openapi urn.
``:extension_path MUST be the namespace specific string(NSS) part of the extension URN plus the path in the OpenAPI document.. As you suggest, the other approach is that the broker author can explicitly declare which path to use, perhaps that would be less confusing.

[Samze]

We discussed this on the call and decided:

• We will have the broker author specify the path explicitly rather than it be implicitly inferred from the id.
• The OpenAPI document should be required. If a broker really doesn't care they can always just provide a dummy YAML doc.

I will update the PR.

[Samze]

This has been updated.

[mattmcneeney]

Some folks from Heroku are playing around with an experiment for this right now. They may leave comments in this PR when they have them.

If they have something worthy of presenting they will bring to the group for review. Thanks!

[mackwong]

any update?

[lyubo1]

I have couple of questions:

1. The openapi_uri is this relative to something or absolute to the broker when hostname is not provided?
2. For APIs that have relatively small OAS could there be an option to inline it in the Extension object, I recall that OAS supports JSON as well as YAML?

Thanks!

[Samze]

1. In the description of openapi_url we state ...If this is an absolute URI then it MUST have no authentication and be publicly available. If this is a relative URI then it is MUST be hosted on the Service Broker and behind the [Service Broker Authentication](#platform-to-service-broker-authentication). Is this descriptive enough?

2. We could do this. It adds some complication to the spec but may make Broker authors lives easier. Do you think most of your extensions would want to be embedded?

[lyubo1]

Thanks @Samze for clarifying on the description 👍

I don't have strong feelings against either but do like the optionality. Though, the requirement of having created a service instance to get the extension specs makes me lean further away from the embedded part.

I'm 👍 to go as is. Thanks!

[ransombriggs]

This path does not match the format of the example (/broker-extension-path) in that it is missing a leading slash. I think it may be worth calling out explicitly that it must start with a slash if it is going to follow the Path Object spec

[mattmcneeney]

I think it doesn't actually need a /, but we should be consistent.

[Samze]

I have kept the slash (and added it where missing) to show that it is a relative path, but I'm not sure we need to make this explicitly required in the spec.

[linux-foundation-easycla]

The committers are authorized under a signed CLA.

• ✅ Sam Gunaratne (d55069c)

[Samze]

I think it need to define an operation as the implementation of extension, just like instance is the implementation of plan.

Could you elaborate on this?

[WalkerGriggs]

I may be reviving a necro-thread here, but...

I mostly wonder why we're opting to couple extensions to service instances? More to the point: can we generalize this proposal further so that the broker itself can be extended?

I'm thinking about additional broker authz/n implementations, for example.

This wouldn't mean a loss in functionality from the current proposal either. In theory, a broker could still specify an instance_id parameter in the extension's OpenAPI spec, but dropping the broker extension path from /v2/service_instances/:instance_id/extensions/broker-extension-path/backup to /v2/extensions/broker-extension-path/backup might give brokers a bit more flexibility.

Thoughts?

[pivotal-marcela-campo]

@Samze Can you provide some insights here or hand off to someone else?

[Samze]

@WalkerGriggs Yeah there are definitely use-cases for both extensions on brokers and on instances. The original idea was to break down the problem into two, start with extensions on instances and then follow up with extensions on the service broker itself. We started on extensions for instances as we had the most practical usecases for that. e.g. backup an instance.

@pivotal-marcela-campo yeah happy to chat more about this proposal if someone wants to take over driving this. It really stalled because while we had agreement in the PMC about this approach, neither CF or Service catalog had done any experimentation to validate this approach.