This module allows you to run Cucumber tests against any system without having to start a current test. This means that the module is independent of the m8ty Server.

It is based on https://github.com/Ragin-LundF/bbd-cucumber-gherkin-lib.

Why using a separate module/application?

Normal application tests are always linked to the current sources of the application in the build process and the result is tested. However, in order to successfully test remote systems or deployments, for example, “external” tools are necessary or tests in the application would have to access external services. But this is not possible in the case of the m8ty server, because customer deployments and especially staging systems are often not accessible from the outside. Furthermore, the m8ty server is not designed to test specific customer systems.

The purpose of deployment testing is to ensure that:

• The deployment was successful

• The configuration was correct

• The connection to any backend systems works

• No unexpected errors occur with certain API calls with the live/staging backend active

Why not Postman or other tools?

The results of such tests should always be “transferable”. This means that they should be able to be read and understood by everyone.

This includes project managers and also people at the operations level, who often have little use for jUnit tests or highly technical Postman tests. The advantage of Cucumber is that it basically describes the process at the textual level, which is comprehensible and readable. Details such as request/response are included, but can be “skipped” by non-technical people. However, it is clear which flow was used and in which process something was tested.

In addition, the result is, among other things, an html file that can be easily opened in any browser and provides a good overview. This requires no technical know-how about formats and is easily exchangeable by mail or artifact.

Since this application can be integrated into the deployment process as a post-step, the resulting html file can also be delivered to the customer at any time to confirm a successful deployment.

Benefits:

• Simple, understandable language that can be read by anyone without technical knowledge

• Integrated request/response objects for more in-depth analysis of technical persons

• Simple, standardized and communicable format (html)

• Option to reuse test scenarios from the server tests itself

• Extensibility of the tests with your own sentences and matchers

Why testing the API and not the frontend?

In principle, you always want to test the entire system. However, it is much easier to test each individual level of the overall system separately.

Above all, structural tests of an API (post-deployment and contract tests) are much easier to implement than a test at UI level. At the API level, data can still be tested in a technically processable format, while at the frontend, this data is presented in a format that is primarily intended for human viewers and less for machines.

In addition, the stages should each have their own tests so that any errors can be quickly assigned. If, for example, an error is detected in a client-side frontend, this can have many different causes:

• API client may be incompatible

• API client was not properly implemented

• API client configuration was faulty

• Incorrect integration of API client in the service

• Any local storages did not work or worked incorrectly

• Interaction of components or security was faulty

• The models for the display were incorrectly painted

• A view has a problem

• .....

By testing at the API of a deployed system itself, issues at the technical interfaces can thus be excluded or found. In addition, the tests for the frontend can be built independently. These only have to test the presentation and mapping of the client response, but not the data delivery or data structure of the backend.

A well-tested frontend (if necessary, also with E2E tests on earlier stages with mocks) does not need to be tested again on live or pre-live staging systems. The difference to previous stages lies in the pure data connection to the final backends.

A test on a technical interface such as the API is therefore the most efficient way to ensure that the deployment was successful and that the backend systems can work together with the frontend.

Usage

Preparation

Cucumber feature files must be stored in a subfolder called features. This means that the directory structure should look like this:

|- features/
|— — — — - — /my_scenario1.feature
|— — — — - — /my_scenario2.feature
| m8ty-e2e-cucumber-<version>.jar
| application.yaml     # optional for external configuration

It is also possible to add subdirectories under features/

Execution

java -jar m8ty-e2e-test-cucumber-<version>.jar

or with external application.yaml configuration:

java -jar m8ty-e2e-test-cucumber-<version>.jar --spring.config.location=classpath:/,file:./

The --spring.config.location=classpath:/,file:./ part overwrites Spring's config location. Because of that the first element must be classpath:/, which loads the application.yaml file from the JAR-file, while the second part looks into the filesystem.

Configuration

Some parts can be configured with the cucumberTest section in the application.yaml. The possibilities can be found at https://github.com/Ragin-LundF/bbd-cucumber-gherkin-lib/wiki/Configuration.

The following shows the most important things.

Default server to be able to use relative paths

To be able to re-use tests of the application without adding always the full URL, the base-URL of the deployed configuration can be configured with:

cucumberTest:
    server:
      protocol: "https"
      host: "myserver.tld/api/v1"
      port: 443

Without this configuration, tests need to be set like this:

@PostDeployTests
Feature: [E2E] 0000 - Test that System is available
  Scenario: Check that myendpoint returns 200
    When executing a GET call to "https://myserver.tld/api/v1/myendpoint"
    Then I ensure that the status code of the response is 200

Having the configuration in place, the same test looks like this:

@PostDeployTests
Feature: [E2E] 0000 - Test that System is available
  Scenario: Check that myendpoint returns 200
    When executing a GET call to "/myendpoint"
    Then I ensure that the status code of the response is 200

Using Proxy

If a proxy is required, the following setting can be added:

cucumberTest:
  proxy:
    host: company.proxy
    port: 8866

Adding some defaults to the ScenarionContext

Under the scenario-context element static key/value pairs can be added, which are accessible later from the ScenarioContext:

cucumberTests:
  scenario-context:
    CTX_PRE_DEFINED_USER: Pre Defined
    CTX_PRE_DEFINED_FIRST_ID: abcdefg