Upgrading an RPM to a specific version with yum
This past week I got to spend some time upgrading my CI/CD systems. The Gitlab upgrade process requires stepping to a specific version when you upgrade major versions, which can be a problem if the latest version isn't supported by the upgrade scripts . In these types of situations, you can tell yum to upgrade to a specific version. To list the versions of a package that are available, you can use the search commands "--showduplicates" option: Once you eye the version you want, you can pass it to yum install: This can also be useful if you want to stick to a minor version vs…
$ read more →Using Kubernetes affinity rules to control where your pods are scheduled
Kubernetes has truly revolutioned distributed computing. While it solves a number of super hard problems, it also adds a number of new challenges. One of these challenges is ensuring your Kubernetes clusters are designed with failure domains in mind. Designing around failure domains includes things like provisioning infrastructure across availability zones, ensuring your physical servers are in different racks, or making sure the pods that support your application don't wind up on the same physical Kubernetes worker…
$ read more →Using the Ansible uri module to test web services during playbook execution
Ansible has amazing support for testing services during playbook execution. This is super useful for validating your services are working after a set of changes take place, and when combined with serial you can stop execution if a change negatively impacts one one or more servers in your fleet. Ansible has a number of modules that can be used to test services, including the uri module. The uri module allows Ansible to interact with a web endpoint, and provides numerous options to control its behavior…
$ read more →Debugging Kubernetes network issues with nsenter, dig and tcpdump
As a Kubernetes administrator I frequently find myself needing to debug application and system issues. Most of the issues I encounter can be solved with Grafana dashboards and Prometheus metrics, or by running one or more Elasticsearch queries to examine logs. But there are times when I need to go deeper and actually inspect activity inside a running pod. A lot of debugging guides use the kubectl exec command to run one or more commands inside a container: But what happens if you don't have a shell installed in the container…
$ read more →Controlling the inventory order when running an Ansible playbook
This week I was updating some Ansible application and OS update playbooks. By default, when you run ansible-playbook it will apply your desired configuration to hosts in the order they are listed in the inventory file (or in the order they are returned by a dynamic inventory script). But what if you want to process hosts in a random order? Or by their sorted or reverse sorted names…
$ read more →