As a long time Kubernetes user the question I hear most often is “how do I create manifests (the file that describes how to create and manage resources in a cluster)?” When I ask the person posing the question how they are creating resources today, I frequently hear that they cobbled together a bunch of random manifests they found on the ‘net or are using $(kubectl apply -f http://site/manifest) based on a website suggestion.
Learning how to generate manifests from scratch baffled me when I was first getting started with Kubernetes. I couldn’t find a comprehensive guide showing how to create resources from scratch, and the information needed to become proficient with this process was scattered across various sites. To assist folks who are just entering the K8S space I thought I would document the process I use to approach the “how do I create a manifest from scratch?” question.
So let’s begin with the basics. A Kubernetes manifest describes the resources (e.g., Deployments, Services, Pods, etc.) you want to create, and how you want those resources to run inside a cluster. I will describe how to learn more about each resource type later in this post. When you define a resource in a manifest it will contain the following four fields:
apiVersion: apps/v1
kind: Deployment
metadata:
...
spec:
...
The apiVersion: field specifies the API group you want to use to create the resource and the version of the API to use. Kubernetes APIs are aggregated into API groups which allows the API server to group APIs by purpose. If we dissect the apiVersion line “apps” would be the API group and v1 would be the version of the apps API to use. To list the available API groups and their versions you can run kubectl with the “api-versions” option:
$ kubectl api-versions |more
admissionregistration.k8s.io/v1beta1
apiextensions.k8s.io/v1beta1
apiregistration.k8s.io/v1
apiregistration.k8s.io/v1beta1
...
The second line, “kind:", lists the type of resource you want to create. Deployments, ReplicaSets, CronJobs, StatefulSet, etc. are examples of resources you can create. You can use the kubectl “api-resources” command to view the available resource types as well as the API group they are associated with:
$ kubectl api-resources |more
NAME SHORTNAMES APIGROUP NAMESPACED KIND
daemonsets ds apps true DaemonSet
deployments deploy apps true Deployment
replicasets rs apps true ReplicaSet
statefulsets sts apps true StatefulSet
...
With the “api-versions” and “api-resources” commands we can find out the available resources (KIND column), the API group (APIGROUP column) the resource type is associated with, and the API group versions (output from api-versions). This information can be used to fill in the apiVersion: and kind: fields. To understand the purpose of each resource type you can use the kubectl “explain” command:
$ kubectl explain --api-version=apps/v1 replicaset
KIND: ReplicaSet
VERSION: apps/v1
DESCRIPTION:
ReplicaSet ensures that a specified number of pod replicas are running at
any given time.
This will give you a detailed explanation of the resource passed as an argument as well as the fields you can populate. Nifty! Now that we’ve covered the first two fields we can move on to metadata: and spec:. The metadata: section is used to uniquely identify the resource inside a Kubernetes cluster. This is were you name the resource, assign tags, annotations, specify a namespace, etc. To view the fields you can add to the metadata: section you can append the “.metadata” string to the resource type passed to “explain”:
$ kubectl explain deployment.metadata | more
KIND: Deployment
VERSION: extensions/v1beta1
RESOURCE: metadata <Object>
DESCRIPTION:
Standard object metadata.
ObjectMeta is metadata that all persisted resources must have, which
includes all objects users must create.
FIELDS:
annotations <map[string]string>
Annotations is an unstructured key value map stored with a resource that
may be set by external tools to store and retrieve arbitrary metadata. They
are not queryable and should be preserved when modifying objects. More
info: http://kubernetes.io/docs/user-guide/annotations
...
Now that we’ve covered the first 3 fields let’s dig into what makes a manifest ticket. The spec: section! This section describes how to create and manage a resource. You will define the container image to use, the number of replicas in a ReplicaSet, the selector criteria, liveness and readiness probe definitions, etc. here To view the fields you can add to the spec: section you can append the “.spec” string to the resource type passed to explain:
$ kubectl explain deployment.spec | more
KIND: Deployment
VERSION: extensions/v1beta1
RESOURCE: spec <Object>
DESCRIPTION:
Specification of the desired behavior of the Deployment.
DeploymentSpec is the specification of the desired behavior of the
Deployment.
FIELDS:
minReadySeconds <integer>
Minimum number of seconds for which a newly created pod should be ready
without any of its container crashing, for it to be considered available.
Defaults to 0 (pod will be considered available as soon as it is ready)
...
Kubectl explain does a really nice job of showing the values under each section, but stitching these together by hand takes time and a lot of patience. To make this process easier the kubectl developers provided the “-o yaml” and “–dry-run” options. These options can be combined with the run and create commands to generate a basic manifest for the resource passed as an argument:
$ kubectl create deployment nginx --image=nginx -o yaml --dry-run
apiVersion: apps/v1
kind: Deployment
metadata:
creationTimestamp: null
labels:
app: nginx
name: nginx
spec:
replicas: 1
selector:
matchLabels:
app: nginx
strategy: {}
template:
metadata:
creationTimestamp: null
labels:
app: nginx
spec:
containers:
- image: nginx
name: nginx
resources: {}
status: {}
Once you have a basic manifest to work with you can start extending it by adding additional fields to the spec: and metadata: sections. You can also add the “–recursive” option to kubectl explain to get a hierarchical view of the various fields. The following example shows how to recursively show every option you can use to customize the containers field:
$ kubectl explain deployment.spec.template.spec.containers --recursive | more
KIND: Deployment
VERSION: extensions/v1beta1
RESOURCE: containers <[]Object>
DESCRIPTION:
List of containers belonging to the pod. Containers cannot currently be
added or removed. There must be at least one container in a Pod. Cannot be
updated.
A single application container that you want to run within a pod.
FIELDS:
args <[]string>
command <[]string>
env <[]Object>
name <string>
value <string>
valueFrom <Object>
configMapKeyRef <Object>
key <string>
name <string>
optional <boolean>
...
If you want to learn more about a specific field you can pass it to explain to get more information:
$ kubectl explain deployment.spec.selector.matchExpressions.operator
KIND: Deployment
VERSION: extensions/v1beta1
FIELD: operator <string>
DESCRIPTION:
operator represents a key's relationship to a set of values. Valid
operators are In, NotIn, Exists and DoesNotExist.
I hope this brief explanation of how to get started with Kubernetes manifests is helpful. This post is definitely a work in progress and I plan to add to it as questions come in. If you have any questions or comments please hit me up on twitter.
Huge thanks to Duffie Cooley and Joe Beda for sharing $(kubectl explain –recursive) on TGIK. Awesome tip! And if you want to start learning more about Kubernetes from several experts in the community please tune in each Friday to TGIK (Thanks God It’s Kubernetes). Each episode dives deep into how various Kubernetes technologies work. You will be picking up nifty little tips and tricks left and right and wondering why you didn’t start watching it sooner.
References: