In our last post we already made the case how important automated testing is in a transformation business case. It is one of the biggest parts when it comes to monetary efficiency savings.

But it comes with some big challenges:

the relevant artefacts need to be integrated into the SDLC
feedback from CI has to come within max 2h, demanding respective scalability from the application but also test infrastructure side
to unleash the full potential of your hybrid / multi-cloud setup the automated testing infrastructure must be part of the baseline delivery and this puts requirements to your automation level

Especially because of the last point we showed in the last article why infrastructure automation is a prerequisite to have the most out of an automated testing transformation.

Link: https://open011prod.wpengine.com/2022/06/27/why-test-automation-wont-work-without-infrastructure-automation/

In this article we would like to continue this discussion and show in more detail how test automation works with Ansible and Selenium ⁽¹⁾. In the second part of this story we’ll touch on another important piece: how to test your automated testing infrastructure. We are going to show some examples around Molecule ⁽²⁾.

So, without further ado, let’s get started …

Automated Application Testing

One of the biggest efforts in the application lifecycle is manual testing; Transforming to a primarily automated testing approach can unleash huge efficiency gains.

Another benefit of automated testing is it’s much more reliable than manual testing and much easier to standardize.

Red Hat doesn’t provide a full offering (like integrated case & incident mgmt.) for automated testing capabilities, except for the DevSecOps area.

But our Ansible Automation Platform offers a lot of the building blocks needed for automated testing frameworks. But on the other side you want to test the automation scripts you are creating for that purpose. Which leaves you with two main tasks:

I want to have an automation engine with has the potential to support me in my test automation mission
I need a way how to test my automation scripts

Let’s get away from the abstract world into a concrete use case:

Automated Testing using Selenium

The architecture we want to focus on is quite simple. We have a client which performs all kinds of online tests. And we want to have the test infrastructure running in a scalable environment via a selenium hub.

Selenium Client and Hub overview ⁽³⁾

Normally we would need to configure the Selenium Hub from scratch. In addition we would need to take care of the required scalability.

Why the automation of the automated test infrastructure is so important is shown in the next diagram:

Automated Testing Architecture in a multi-cloud environment

If you run a hybrid or multi-cloud approach you may have different infrastructures for the same application. Via containerization and OpenShift as the container platform we can already abstract from the different kinds of infrastructures. We also would have a repository to centrally store the relevant artefacts. Besides that we should not forget about test data. We need a central point to manage them as well.

But now comes the thing, maybe I am having my default development and my default CI running on Azure. And maybe I have excessive performance testing which runs on synthetic data and should go where the cheapest infrastructure is, maybe Alibaba. This example is not doable if the testing architecture isn’t coming with the baseline and is not automated. The manual effort would eat up all achieved efficiency gains. Besides that everything would get too complex and you quickly would lose control.

So, we have two parts:

PART I: Selenium support by Ansible shows how you can ramp up the test infrastructure via Ansible

and

PART II: How to test your automation scripts is shown in the second part via Molecule

Let’s see how we could achieve this setup through Ansible automation with a small example:

PART I – Selenium module for Ansible

Get it!

ansible-galaxy install SeleniumHQ.selenium Code language: CSS (css)

Use It

After using ansible-galaxy to install the module, take the library/selenium file, and put it into the respective repository.

Include the following examples within in your playbook/roles:

- name: Start a standalone server
  selenium:
    role: standalone
    state: running

- name: Start a standalone server (specific selenium version)
  selenium:
    version: 2.53.1 
    role: standalone
    state: running
    
- name: Start a basic grid that listens on port 4444
  selenium:
    role: hub
    state: running

- name: Start a grid that runs on port 4445
  selenium:
    role: hub
    state: running
    args:
      port: 4445

- name: Start the grid with greater memory
  selenium:
    role: hub
    state: running
    javaargs:
      - Xmx1024M
      - Xmy1024M

- name: Start a basic node that connects locally
  selenium:
    role: node
    state: running
    args:
      hubUrl: http://0.0.0.0:4444

- name: Restart a running node that was listening on a previous port
  selenium:
    role: node
    state: restarted
    args:
      hubUrl: http://0.0.0.0:4445

- name: Start a node that uses a JSON config and has a logfile
  roles:
    - name: output the config file
      template: src=config.json.j2 dest=config.json

    - name: start the node
      selenium:
        role: node
        state: running
        logfile: node.log
        args:
          nodeConfig: config.json

- name: Stop the running selenium standalone server
  selenium:
    state: stopped
Code language: JavaScript (javascript)

Develop

Here are some code examples how you would start tests on your selenium hub. You may integrate these parts into your CI/CD pipeline via CLI.

Before running tests:

alias test-module=/home/vagrant/ansible/hacking/test-moduleCode language: JavaScript (javascript)

To test the module manually, you can run:

cd library/
test-module -m ./selenium -a "role=hub state=running"
Code language: JavaScript (javascript)

To run all tests:

cd tests/
ansible-playbook -i localhost, test.yml

PART II – Testing of your Automation Infrastructure

In the chapter about Selenium we covered the use of Ansible to spin up the needed instances to layout the required platform to execute Selenium tests.

But test automation doesn’t just end on top of the automation infrastructure you’re using.

We also have to ensure this infrastructure is kept in good shape and changes or updates to it will as well be tested in an automated way beforehand to not disturb the test automation execution on top of it.

This is now the point where Molecule joins the scene.

Test Ansible Roles using Molecule and Podman

Extracted out of an article by Ilkka Tengvall ⁽²⁾

What we want to achieve: Lightweight and easy testing

Let’s have a look at how you can add testing to Ansible roles.

Meet Molecule with Podman plugin and Ansible as test language:

Let’s go through the process

We use RHEL8 in the guide, along with RHEL8 UBI container images. RHEL is set up with required repos, see ansible playbook.

Install Podman, Ansible and required tools for python

sudo dnf install -y podman ansible python3 python3-virtualenv gcc git

Install Molecule

Create a python virtual environment where we install all molecule tools. We use pip to install python modules, and python virtual environment to keep them separate from system modules.

python3 -m virtualenv molecule_ansible
source molecule_ansible/bin/activate
pip install ansible testinfra molecule podman python-vagrant ansible-lint 
  flake8 molecule[lint] molecule[podman]

Apply molecule to ansible role

Please note that the molecule test we are going to do requires you to be the root user!

Activate molecule virtual environment if not already done:

source molecule_ansible/bin/activate

Create a new role by using Molecule to get the basic molecule files in place. This will run ansible-galaxy init role in background, and will add the initial molecule config and test files.

molecule init role my-new-role --driver-name podmanCode language: JavaScript (javascript)

If you have an existing ansible role, and want to add molecule testing to that role, run:

molecule init scenario -r ftp --driver-name podman

If you are a bit lazy and just want to go on with testing molecule, git clone the below example role with added molecule testing:

git clone https://github.com/RedHatNordicsSA/molecule-podman-blog.gitCode language: PHP (php)

Read the following chapters and play with it by modifying the settings and test cases.

Molecule

Now let’s get going. Investigate how:

container gets prepared for the test in the converge.yml file. In this very simple case it includes the role to handle FTP.
Molecule is set up to use podman, and a container that has init in it. This is in the molecule.yml file.
tests are written in ansible in the verify.yml file.

Ansible test cases

Test cases are trivial to write with ansible. In this case it just runs install and deactivate commands for the service in check-only mode, and verifies if it would have needed to apply changes. In our case a change would show a failure, as everything should be already set up. So it’s ok if nothing would have needed a change.

   - name: check if vsftpd is installed
      package:
        name: vsftpd
        state: present
      check_mode: yes
      register: pkg

    - name: fail if package was not installed
      assert:
        that:
          - pkg.changed is false
        fail_msg: "Package vsftpd was not installed!"
        success_msg: "Package vsftpd was installed."
Code language: JavaScript (javascript)

Podman

You don’t need to know about podman in this case. I’d only add image pruning to be done once in a while to the test machine, if it’s a permanent VM. You could be running these tests in Ansible Tower or a Jenkins machine, in which case you want to prune the images. Remember, no root required, so podman gives you a lot of options where to run it.

Here’s how it looks when the container is started step by step. Podman has pulled the image, and is running a test container. You can do only those parts by the following commands:

(molecule_ansible) [cloud-user@localhost ftp]$ molecule converge
(molecule_ansible) [cloud-user@localhost ftp]$ podman ps
CONTAINER ID  IMAGE                                            COMMAND         CREATED         STATUS             PORTS  NAMES
712c7b4ac78c  registry.access.redhat.com/ubi8/ubi-init:latest  /usr/sbin/init  33 seconds ago  Up 33 seconds ago         instance
(molecule_ansible) [cloud-user@localhost ftp]$ molecule login
[root@712c7b4ac78c /]# whoami
root
[root@712c7b4ac78c /]# exit
(molecule_ansible) [cloud-user@localhost ftp]$ molecule destroy
Code language: PHP (php)

Executing the test

First and foremost, this is enough to run the tests:

(molecule_ansible) [cloud-user@localhost ftp]$ molecule test

In our case it has a happy ending:

 TASK [check if vsftpd is installed] ********************************************
    ok: [instance]

    TASK [fail if package was not installed] ***************************************
    ok: [instance] => {
        "changed": false,
        "msg": "Package vsftpd was installed."
    }

    TASK [check service is stopped] ************************************************
    ok: [instance]

    TASK [fail if service was activated] *******************************************
    ok: [instance] => {
        "changed": false,
        "msg": "Service vsftpd was disabled."
    }

    TASK [test result] *************************************************************
    ok: [instance] => {
        "msg": "FTP daemon was installed and disabled. Test OK"
    }

    PLAY RECAP *********************************************************************
    instance                   : ok=6    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0
Code language: PHP (php)

See Molecule command documentation ⁽⁵⁾ for details.

Special Tools

There are other tools out there which help you to keep your infrastructure in shape. Definitely worth mentioning is KICS ⁽⁶⁾.

KICS finds security vulnerabilities, compliance issues, and infrastructure misconfigurations in the following Infrastructure as Code solutions: Terraform, Kubernetes, Docker, AWS CloudFormation, Ansible, Microsoft ARM. In total over 2000 queries are available.

KICS is easy to install and run, easy to understand results, and easy to integrate into CI.

Definitely worth a look.

Key-takeaways

Ansible is a huge enabler and accelerator when it comes to automated testing
Automated testing in combination with infrastructure automation unleashes the full potential and in both stories Ansible is the right partner
Red Hats Advanced Cluster Security for Kubernetes offering fits perfectly into an automated testing story and applies to increasing security needs