Define API Discovery Format

[hlgr360]

Related to #26

General intro (primarily for public apis)
http://nordicapis.com/api-discovery-11-ways-to-find-apis/

Current approaches to API discovery:
https://github.com/zalando-incubator/api-discovery
https://github.com/apis-json/api-json
https://developers.google.com/discovery/v1/reference/

Service Documents for Discovery
https://bitworking.org/projects/atom/draft-ietf-atompub-protocol-14.html
https://github.com/hypoport/hal-service-document-consumer
See also comment train in #26 for more

[hlgr360]

Exploring Kubernetes I stumbled over their very promising format.

MacBook-Air-4:~ h_reinhardt$ kubectl proxy Starting to serve on 127.0.0.1:8001

Navigate to the root of the Url and you receive a json endpoint document

There is precious little information about this out there, but exploring /api yields the following
{ "kind": "APIVersions", "versions": [ "v1" ], "serverAddressByClientCIDRs": [ { "clientCIDR": "0.0.0.0/0", "serverAddress": "10.240.255.5:443" } ] }

Exploring /apis shows a similar structure but centered around API Groups
{ "kind": "APIGroupList", "groups": [ { "name": "apps", "versions": [ { "groupVersion": "apps/v1beta1", "version": "v1beta1" } ], "preferredVersion": { "groupVersion": "apps/v1beta1", "version": "v1beta1" }, "serverAddressByClientCIDRs": [ { "clientCIDR": "0.0.0.0/0", "serverAddress": "10.240.255.5:443" } ] }, { "name": "authentication.k8s.io", "versions": [ { "groupVersion": "authentication.k8s.io/v1beta1", "version": "v1beta1" } ], "preferredVersion": { "groupVersion": "authentication.k8s.io/v1beta1", "version": "v1beta1" }, "serverAddressByClientCIDRs": [ { "clientCIDR": "0.0.0.0/0", "serverAddress": "10.240.255.5:443" } ] }, ...

Similarly, exploring /sawggerapi returns a top-line swagger definition of available APIs

Exploring /api/vi1 returns a resource definition list
{ "kind": "APIResourceList", "groupVersion": "v1", "resources": [ { "name": "bindings", "namespaced": true, "kind": "Binding" }, { "name": "componentstatuses", "namespaced": false, "kind": "ComponentStatus" }, { "name": "configmaps", "namespaced": true, "kind": "ConfigMap" }, { "name": "endpoints", "namespaced": true, "kind": "Endpoints" }, ...

Exploring /api/v1/endpoints returns the actual endpoint
{ "kind": "EndpointsList", "apiVersion": "v1", "metadata": { "selfLink": "/api/v1/endpoints", "resourceVersion": "341715" }, "items": [ { "metadata": { "name": "clamav", "namespace": "default", "selfLink": "/api/v1/namespaces/default/endpoints/clamav", "uid": "82c7db0b-25cc-11e7-9292-0017fa101699", "resourceVersion": "137649", "creationTimestamp": "2017-04-20T13:23:15Z", "labels": { "io.kompose.service": "clamav" } }, ...

[hlgr360]

root.json.txt
api.json.txt
apis.json.txt
swaggerapi.json.txt

[hlgr360]

Here are some links I found, but little about the thoughts behind the API format

https://thenewstack.io/taking-kubernetes-api-spin/
https://kubernetes.io/docs/reference/api-overview/
https://kubernetes.io/docs/admin/kube-apiserver

[hlgr360]

What really appeals to me is the notion of having an endpoint document at the root and the subsequent ability to introspect APIs

[hlgr360]

@DonMartin76 - I would really like to explore the API discovery feature together with Wicked. Like in K8s I would like to be able to retrieve an endpoint document which gives me the UI and apis .. and then I can discover which API's are available and then can subsequently discover and retrieve the swagger definitions of those api.

This would allow us to crawl our API Mgmt infrastructure and discover the various hosted API's and bring those together in an search engine ala apis.io or a portal (zarlando?!).

Permissions and visibility are certainly two interesting topics we should discuss.

[DonMartin76]

There are stubs for these things in wicked, e.g. asking the portal for a JSON representation of the web page (almost) always works. The linking is not done to the last consequence yet, as I am not yet fully convinced this is the right way of doing it (tunneling the API via the portal). Examples:

$ curl -H 'Accept: application/json' https://wicked-demo.haufe.io/apis
$ curl -H 'Accept: application/json' https://wicked-demo.haufe.io/apis/petstore

Adopting a discovery format is certainly interesting, but how to proceed with protected resources is clearly an issue which needs to be discussed.

[hlgr360]

dret [3:12 PM]

if you're interested in teaming up, definitely let me know, @holger. i have started focusing a bit of energy into this space, and plan on spending quite a bit more. and being webby always is something that gets me excited! pretty much everybody asks about this, and interest seems to be picking up. there are various facets.
· how to make the descriptions discoverable? very likely the webby way, but also some ping mechanism at deployment. pivio had something (but also quite a bit more).
· what to provide as description? apis.io is one, APIs.json another pivio is one as well, and so is JSON home. they sometimes have slightly various angles, but to me, the most important point is that they are well-defined and easily extensible. http://dret.net/lectures/summit-nyc-2017/#service-descriptions was my last attempt of a (very short) overview.
· what to include in the description? different of the formats above have different angles. pivio is very infrastructure-y; APIs.json is very taggy and human oriented; JSON home is agnostic. https://github.com/dret/APIdocumentation is where i have only just started a collection of these things (see issue #1). my goal is to turn that into a small site that tells people what they could document. we often have people being worried that they might forget things. my usual response is to tell them: "start with what you need now, and start in a way that you can easily add things as needed."
· how to make the info available? for example, service-desc could be WSDL or OpenAPI or RAML or however you choose to document the technical interface; service-doc might be PDF or HTML or whatever else you think works best for the human-oriented documentation.
· how to use all of this? this is where being webby helps a lot. building tooling that crawls this at service start-up is not hard. pivio already has such a crawler and even a back-end for it. but you can also build simple checks, asking simple questions as "is there change management documentation"? if there is, it still may be horrible, but if there isn't, you can give immediate feedback and alert the team and tell them that they should document their change management strategy.
well, that was a lot of writing. as you probably can tell, i am passionate and have some ideas in this space. and many people are interested in this. so as mentioned above, if you want to team up, i'd be glad to chat and see if we find areas of overlap.