Tiidy up and refactoring of tasks (#80)

* Tidy up and refactoring of tasks

  - `k3s_config_dir` derived from `k3s_config_file`, reused throughout the role
    to allow for easy removal of "Rancher" references #73.
  - `k3s_token_location` has moved to be in `k3s_config_dir`.
  - Tasks for creating directories now looped to caputure configuration from
    `k3s_server` and `k3s_agent` and ensure directories exist before k3s
    starts, see #75.
  - Server token collected directly from token file, not symlinked file
    (node-token).
  - `k3s_runtime_config` defined in `vars/` for validation and overwritten in
    tasks for control plane and workers.
  - Removed unused references to GitHub API.

* set_fact now uses FQCN

* re-pin molecule<3.2

* Command module now uses FQCN

* Added package checks for #72

* Reorder task files

  - Docker tasks moved into a separate directory for ease of removal #67
  - Bugfix: Control plane on alternate port didn't work.
  - Validation tasks grouped

* Fix Fedora tests

* Add optional documentation links to validations steps #76

* Removed jmespath requirement

* Fix issue with data collection

* Release candidate

CHANGELOG.md

@@ -14,6 +14,37 @@
 ---
 -->
 
+## 2020-12-21, v2.4.0
+
+### Notable changes
+
+  - `k3s_config_dir` derived from `k3s_config_file`, reused throughout the role
+    to allow for easy removal of "Rancher" references #73.
+  - `k3s_token_location` has moved to be in `k3s_config_dir`.
+  - Tasks for creating directories now looped to caputure configuration from
+    `k3s_server` and `k3s_agent` and ensure directories exist before k3s
+    starts, see #75.
+  - Server token collected directly from token file, not symlinked file
+    (node-token).
+  - `k3s_runtime_config` defined in `vars/` for validation and overwritten in
+    tasks for control plane and workers.
+  - Removed unused references to GitHub API.
+  - `set_fact` and `command` tasks now use FQCN.
+  - Check of `ansible_version` in environment check.
+  - Introduction of target environment checks for #72.
+  - Fixed bug with non-default listening port not being passed to workers.
+  - Added ability to put documentation links into validation checks #76.
+  - Removed the requirement for `jmespath` on the Ansible controller.
+  - Fixed bug with issue data collection tasks.
+
+### Breaking changes
+
+  - Ansible minimum version is hard set to v2.10.4
+  - `k3s_token_location` has moved to be in `k3s_config_dir` so re-running the
+    role will create a duplicate file here.
+
+---
+
 ## 2020-12-19, v2.3.0
 
 ### Notable changes

README.md

@@ -14,8 +14,7 @@ and [CHANGELOG.md](CHANGELOG.md).
 
 The host you're running Ansible from requires the following Python dependencies:
 
-  - `ansible >= 2.10`
-  - `jmespath >= 0.10.0`
+  - `ansible >= 2.10.4`
 
 You can install dependencies using the requirements.txt file in this repository:
 `pip3 install -r requirements.txt`.
@@ -70,7 +69,6 @@ consistency. These are generally cluster-level configuration.
 | `k3s_build_cluster`              | When multiple play hosts are available, attempt to cluster. Read notes below.   | `true`                         |
 | `k3s_control_node_address`       | Use a specific control node address. IP or FQDN.                                | NULL                           |
 | `k3s_github_url`                 | Set the GitHub URL to install k3s from.                                         | https://github.com/k3s-io/k3s  |
-| `k3s_skip_validation`            | Skip all tasks that validate configuration.                                     | `false`                        |
 | `k3s_install_dir`                | Installation directory for k3s.                                                 | `/usr/local/bin`               |
 | `k3s_install_hard_links`         | Install using hard links rather than symbolic links.                            | `false`                        |
 | `k3s_server_manifests_templates` | A list of Auto-Deploying Manifests Templates.                                   | []                             |
@@ -154,17 +152,18 @@ configuration.
 The below variables are used to change the way the role executes in Ansible,
 particularly with regards to privilege escalation.
 
-| Variable                         | Description                                                         | Default Value |
-|----------------------------------|---------------------------------------------------------------------|---------------|
-| `k3s_become_for_all`             | Escalate user privileges for all tasks. Overrides all of the below. | `false`       |
-| `k3s_become_for_systemd`         | Escalate user privileges for systemd tasks.                         | NULL          |
-| `k3s_become_for_install_dir`     | Escalate user privileges for creating installation directories.     | NULL          |
-| `k3s_become_for_manifests_dir`   | Escalate user privileges for creating manifests directory.          | NULL          |
-| `k3s_become_for_usr_local_bin`   | Escalate user privileges for writing to `/usr/local/bin`.           | NULL          |
-| `k3s_become_for_data_dir`        | Escalate user privileges for creating data directory.               | NULL          |
-| `k3s_become_for_package_install` | Escalate user privileges for installing k3s.                        | NULL          |
-| `k3s_become_for_kubectl`         | Escalate user privileges for running `kubectl`.                     | NULL          |
-| `k3s_become_for_uninstall`       | Escalate user privileges for uninstalling k3s.                      | NULL          |
+| Variable                            | Description                                                         | Default Value |
+|-------------------------------------|---------------------------------------------------------------------|---------------|
+| `k3s_skip_validation`               | Skip all tasks that validate configuration.                         | `false`       |
+| `k3s_skip_env_checks`               | Skill all tasks that check environment configuration.               | `false`       |
+| `k3s_become_for_all`                | Escalate user privileges for all tasks. Overrides all of the below. | `false`       |
+| `k3s_become_for_systemd`            | Escalate user privileges for systemd tasks.                         | NULL          |
+| `k3s_become_for_install_dir`        | Escalate user privileges for creating installation directories.     | NULL          |
+| `k3s_become_for_directory_creation` | Escalate user privileges for creating application directories.      | NULL          |
+| `k3s_become_for_usr_local_bin`      | Escalate user privileges for writing to `/usr/local/bin`.           | NULL          |
+| `k3s_become_for_package_install`    | Escalate user privileges for installing k3s.                        | NULL          |
+| `k3s_become_for_kubectl`            | Escalate user privileges for running `kubectl`.                     | NULL          |
+| `k3s_become_for_uninstall`          | Escalate user privileges for uninstalling k3s.                      | NULL          |
 
 #### Important note about `k3s_release_version`
 
@@ -303,7 +302,7 @@ stable release:
       datastore-endpoint: "postgres://postgres:verybadpass@database:5432/postgres?sslmode=disable"
   pre_tasks:
     - name: Set each node to be a control node
-      set_fact:
+      ansible.builtin.set_fact:
         k3s_control_node: true
       when: inventory_hostname in ['node2', 'node3']
   roles:

defaults/main.yml

@@ -26,6 +26,9 @@ k3s_github_url: https://github.com/k3s-io/k3s
 # Skip all tasks that validate configuration
 k3s_skip_validation: false
 
+# Skip all tasks that check environment configuration
+k3s_skip_env_checks: false
+
 # Installation directory for k3s
 k3s_install_dir: /usr/local/bin
 
@@ -71,9 +74,8 @@ k3s_agent: {}
 k3s_become_for_all: false
 k3s_become_for_systemd: null
 k3s_become_for_install_dir: null
-k3s_become_for_manifests_dir: null
+k3s_become_for_directory_creation: null
 k3s_become_for_usr_local_bin: null
-k3s_become_for_data_dir: null
 k3s_become_for_package_install: null
 k3s_become_for_kubectl: null
 k3s_become_for_uninstall: null

molecule/autodeploy/converge.yml

@@ -5,6 +5,11 @@
   vars:
     molecule_is_test: true
     k3s_build_cluster: false
+    k3s_install_dir: /opt/k3s/bin
+    k3s_config_file: /opt/k3s/etc/k3s.yaml
+    k3s_server:
+      data-dir: /var/lib/k3s-io
+      default-local-storage-path: /var/lib/k3s-io/local-storage
     k3s_server_manifests_templates:
       - "molecule/autodeploy/templates/00-ns-monitoring.yml.j2"
   roles:

molecule/autodeploy/prepare.yml

@@ -0,0 +1,9 @@
+---
+- name: Prepare
+  hosts: node*
+  become: true
+  tasks:
+    - name: Ensure apt cache is updated
+      ansible.builtin.apt:
+        update_cache: true
+      when: ansible_pkg_mgr == 'apt'

molecule/debug/converge.yml

@@ -6,7 +6,7 @@
     pyratlabs_issue_controller_dump: true
   pre_tasks:
     - name: Ensure k3s_debug is set
-      set_fact:
+      ansible.builtin.set_fact:
         k3s_debug: true
   roles:
     - xanmanning.k3s

molecule/debug/prepare.yml

@@ -0,0 +1,8 @@
+---
+- name: Prepare
+  hosts: all
+  tasks:
+    - name: Ensure apt cache is updated
+      ansible.builtin.apt:
+        update_cache: true
+      when: ansible_pkg_mgr == 'apt'

molecule/highavailabilitydb/converge.yml

@@ -10,7 +10,7 @@
       datastore-endpoint: "postgres://postgres:verybadpass@database:5432/postgres?sslmode=disable"
   pre_tasks:
     - name: Set each node to be a control node
-      set_fact:
+      ansible.builtin.set_fact:
         k3s_control_node: true
       when: inventory_hostname in ['node2', 'node3']
   roles:

molecule/highavailabilitydb/prepare.yml

@@ -1,5 +1,5 @@
 ---
-- name: Prepare
+- name: Prepare Load Balancer
   hosts: loadbalancer
   tasks:
     - name: Ensure apt cache is updated
@@ -28,3 +28,11 @@
       command: haproxy -D -f /usr/local/etc/haproxy/haproxy.cfg -p /var/run/haproxy.pid
       args:
         creates: /var/run/haproxy.pid
+
+- name: Prepare nodes
+  hosts: node*
+  tasks:
+    - name: Ensure apt cache is updated
+      ansible.builtin.apt:
+        update_cache: true
+      when: ansible_pkg_mgr == 'apt'

molecule/highavailabilityetcd/converge.yml

@@ -11,7 +11,7 @@
       secrets-encryption: true
   pre_tasks:
     - name: Set each node to be a control node
-      set_fact:
+      ansible.builtin.set_fact:
         k3s_control_node: true
   roles:
     - role: xanmanning.k3s

molecule/highavailabilityetcd/prepare.yml

@@ -1,5 +1,5 @@
 ---
-- name: Prepare
+- name: Prepare Load Balancer
   hosts: loadbalancer
   tasks:
     - name: Ensure apt cache is updated
@@ -28,3 +28,11 @@
       command: haproxy -D -f /usr/local/etc/haproxy/haproxy.cfg -p /var/run/haproxy.pid
       args:
         creates: /var/run/haproxy.pid
+
+- name: Prepare nodes
+  hosts: node*
+  tasks:
+    - name: Ensure apt cache is updated
+      ansible.builtin.apt:
+        update_cache: true
+      when: ansible_pkg_mgr == 'apt'

molecule/nodeploy/prepare.yml

@@ -0,0 +1,8 @@
+---
+- name: Prepare
+  hosts: all
+  tasks:
+    - name: Ensure apt cache is updated
+      ansible.builtin.apt:
+        update_cache: true
+      when: ansible_pkg_mgr == 'apt'

molecule/requirements.txt

@@ -1,6 +1,6 @@
 -r ../requirements.txt
 
-molecule[docker]<3.2.0
+molecule[docker]<3.2
 docker>=4.3.1
 yamllint>=1.25.0
 ansible-lint>=4.3.5

requirements.txt

@@ -1,2 +1 @@
-jmespath>=0.10.0
-ansible>=2.10.3
+ansible>=2.10.4

tasks/build/configure-k3s-cluster.yml

@@ -1,25 +1,25 @@
 ---
 
-- name: Ensure NODE_TOKEN is captured from control node
+- name: Ensure cluster token is captured from control node
   ansible.builtin.slurp:
-    path: "{{ k3s_runtime_config['data-dir'] | default(k3s_data_dir) }}/server/node-token"
-  register: k3s_slurped_control_token
+    path: "{{ k3s_runtime_config['data-dir'] | default(k3s_data_dir) }}/server/token"
+  register: k3s_slurped_cluster_token
   delegate_to: "{{ k3s_control_delegate }}"
   when: k3s_control_token is not defined and not ansible_check_mode
   become: "{{ k3s_become_for_kubectl | ternary(true, false, k3s_become_for_all) }}"
 
-- name: Ensure NODE_TOKEN is formatted correctly for use in templates
-  set_fact:
-    k3s_control_token: "{{ k3s_slurped_control_token.content | b64decode }}"
+- name: Ensure cluster token is formatted correctly for use in templates
+  ansible.builtin.set_fact:
+    k3s_control_token: "{{ k3s_slurped_cluster_token.content | b64decode }}"
   when: k3s_control_token is not defined and not ansible_check_mode
 
-- name: Ensure dummy NODE_TOKEN is defined for ansible_check_mode
-  set_fact:
+- name: Ensure dummy cluster token is defined for ansible_check_mode
+  ansible.builtin.set_fact:
     k3s_control_token: "{{ k3s_control_delegate | to_uuid }}"
   check_mode: false
   when: k3s_control_token is not defined and ansible_check_mode
 
-- name: Ensure the cluster NODE_TOKEN file location exists
+- name: Ensure the cluster token file location exists
   ansible.builtin.file:
     path: "{{ k3s_token_location | dirname }}"
     state: directory
@@ -69,10 +69,10 @@
   when: k3s_control_node and not k3s_primary_control_node
   become: "{{ k3s_become_for_systemd | ternary(true, false, k3s_become_for_all) }}"
 
-- import_tasks: ../validate/check-control-plane.yml
+- import_tasks: ../validate/state/control-plane.yml
   when: not k3s_skip_validation
 
 - meta: flush_handlers
 
-- import_tasks: ../validate/check-cluster-nodes-ready.yml
+- import_tasks: ../validate/state/nodes.yml
   when: not k3s_skip_validation

tasks/build/install-docker-amazon.yml → tasks/build/docker/amazon/install.yml

@@ -1,7 +1,7 @@
 ---
 
 - name: Ensure docker is installed using amazon-linux-extras
-  command: amazon-linux-extras install docker
+  ansible.builtin.command: amazon-linux-extras install docker
   args:
     creates: /etc/docker
   notify:

tasks/build/install-docker-prerequisites-redhat.yml → tasks/build/docker/redhat/install-prerequisites.yml

@@ -45,7 +45,7 @@
   become: "{{ k3s_become_for_package_install | ternary(true, false, k3s_become_for_all) }}"
 
 - name: Ensure Docker repository is installed and configured from file
-  command: yum-config-manager --add-repo=https://download.docker.com/linux/centos/docker-ce.repo
+  ansible.builtin.command: yum-config-manager --add-repo=https://download.docker.com/linux/centos/docker-ce.repo
   args:
     creates: /etc/yum.repos.d/docker-ce.repo
   when: ansible_distribution | lower not in ['amazon']

tasks/build/download-k3s.yml

@@ -1,19 +1,19 @@
 ---
 
 - name: Ensure target host architecture information is set as a fact
-  set_fact:
+  ansible.builtin.set_fact:
     k3s_arch: "{{ k3s_arch_lookup[ansible_architecture].arch }}"
     k3s_arch_suffix: "{{ k3s_arch_lookup[ansible_architecture].suffix }}"
   check_mode: false
 
 - name: Ensure URLs are set as facts for downloading binaries
-  set_fact:
+  ansible.builtin.set_fact:
     k3s_binary_url: "{{ k3s_github_download_url }}/{{ k3s_release_version }}/k3s{{ k3s_arch_suffix }}"
     k3s_hash_url: "{{ k3s_github_download_url }}/{{ k3s_release_version }}/sha256sum-{{ k3s_arch }}.txt"
   check_mode: false
 
 - name: Override k3s_binary_url and k3s_hash_url facts for testing specific commit
-  set_fact:
+  ansible.builtin.set_fact:
     k3s_binary_url: "https://storage.googleapis.com/k3s-ci-builds/k3s{{ k3s_arch_suffix }}-{{ k3s_release_version }}"
     k3s_hash_url: "https://storage.googleapis.com/k3s-ci-builds/k3s{{ k3s_arch_suffix }}-{{ k3s_release_version }}.sha256sum"
   when:
@@ -28,7 +28,7 @@
   check_mode: false
 
 - name: Ensure sha256sum is set from hashsum variable
-  set_fact:
+  ansible.builtin.set_fact:
     k3s_hash_sum: "{{ (k3s_hash_sum_raw.content.split('\n') |
       select('search', 'k3s' + k3s_arch_suffix) |
       reject('search', 'images') |

tasks/build/get-systemd-context.yml

@@ -1,7 +1,7 @@
 ---
 
 - name: Ensure systemd context is correct if we are running k3s rootless
-  set_fact:
+  ansible.builtin.set_fact:
     k3s_systemd_context: user
     k3s_systemd_unit_dir: "{{ ansible_user_dir }}/.config/systemd/user"
   when: k3s_runtime_config is defined

tasks/build/get-version.yml

@@ -1,13 +1,13 @@
 ---
 
 - name: Ensure k3s_release_version is set to default if false
-  set_fact:
+  ansible.builtin.set_fact:
     k3s_release_version: "{{ k3s_release_channel }}"
   check_mode: false
   when: k3s_release_version is defined and not k3s_release_version
 
 - name: Ensure the default release channel is set
-  set_fact:
+  ansible.builtin.set_fact:
     k3s_release_channel: "{{ k3s_release_version | default('stable') }}"
   check_mode: false
 
@@ -20,7 +20,7 @@
   check_mode: false
 
 - name: Ensure the release version is set as a fact
-  set_fact:
+  ansible.builtin.set_fact:
     k3s_release_version: "{{ item.latest }}"
   loop: "{{ k3s_latest_release.json.data }}"
   check_mode: false

tasks/build/install-k3s-directories.yml

@@ -0,0 +1,10 @@
+---
+
+- name: Ensure {{ directory.name }} exists
+  ansible.builtin.file:
+    path: "{{ directory.path }}"
+    state: directory
+    mode: "{{ directory.mode | default(0755) }}"
+  become: "{{ k3s_become_for_directory_creation | ternary(true, false, k3s_become_for_all) }}"
+  when: directory.path is defined
+        and directory.path != omit