Modelling Kubernetes Custom Types Resources

Question

I am building an opinionated PaaS like service on top of Kubernetes ecosystem.

I have a desire to model an SSHService and SSHUser, I'll either extend Kubernetes api server by registering new types/schemas (looks pretty simple) or using custom resources via ThirdPartyResource http://kubernetes.io/v1.1/docs/design/extending-api.html

I previous built my own API server on non-kubernetes infrastructure. The way I modelled it was somewhat as below, so an admin would do via restful actions:

1) Create SSH Service 2) Create SSh User 3) Add User to SSH Service

The third action would run on the SSH Service resource, which would check the universe to ensure an SSH User with name ref existed within the universe before adding it to its allowed user array attribute.

In Kubernetes I don't think cross resource transaction are supported, or intentional looking at how other things are modeled ** (for example I can create a pod with secret volume referring a secret name that does not exist and this is accepted).

So in Kubernetes world I intend to 1) Create SSh Service with .Spec.AllowedGroups [str] 2) Create SSH User with .Spec.BelongToGroups [str] where groups is just an array of group names as strings

A kubernetes client will watch for changes to ssh service and ssh users where the sets change update back to the API a secret volume (later configmap volume) for passwd/shadow to be used in the SSH container

Is this a sane approach to model custom resources?

Answer 1

First reaction is that if you already have your own API server, and it works, there is no need to rewrite the API in kubernetes style. I'd just try to reuse the thing that works.

If you do want to rewrite, here are my thoughts:

If you need lots of SSHServices, and you need lots of people to use your API for creating SSHServices, then it makes sense to represent the parameters of the ssh service as a ThirdParty resource.

But if you have just 1 or a few SSHServices, and you update it infrequently, then I would not create a ThirdParty resource for it. I would just write an RC that runs the SSH service Pod mount a secret (later configMap) volume that contains a configuration file, in the format of your choice. The config file would include the AllowedGroups. Once you have v1.2 with config map, which will be like in a month, you will be able to update the config by POSTing a new configmap to the apiserver, without needing the SSH service to restart. (It should watch the config file for changes). Basically, you can think of a configMap as a simpler version of ThirdParty resource.

As far as SSHUsers, you could use a ThirdParty resource and have the SSH controller watch the SSHUsers endpoint for changes. (Come to think of it, I'm not sure how you watch a third party resource.)

Or maybe you want to just put the BelongToGroups information into the same ConfigMap. This gives you the "transactionality" you wanted. It just means that updates to the config are serialized and require an operator or cron job to push the config. Maybe that is not so bad?