test.guide user documentation

Test execution in development projects leads to the accumulation of various reports and data. Test automation intensifies this, producing very high numbers of test executions, which are distributed over several users and test environments.

test.guide is a centralized database application, flexibly deployable in different development and test environments. It has been developed specifically for use in the automotive area and provides the capability to manage, view and analyze test executions and results via an intuitive, web-based interface.

test.guide centrally stores test reports and related file-based test data such as log files or images. By capturing all relevant meta data for every single test execution and providing extensive filtering capabilities, test.guide makes analysis of trends or root cause analysis of test failures a breeze.

In order to import test reports of any test execution system, test.guide provides an ASAM ATX compliant interface and ready-to-use adapters for the most common test report formats.

The user interface is fully browser-based and supports using test.guide on various user devices such as notebooks, smartphones and tablets.

The test case coverage module provides features to analyze test case executions regarding scopes, variants and requirements.

The monitoring module continuously collects status and health data of the test benches. From this data, test.guide provides an overview about the complete test environment’s state and allows to relate test results with test bench data. Furthermore test benches observed by the monitoring can be used for automated test execution distribution.

Monitoring provides answers to the following questions:

On each test bench, a small software called the ResourceAdapter is executed. It transfers the nodes vital data, test executions and configurations to the central test.guide server, where the data is aggregated and related to test case executions.

The ResourceAdapter can also be extended via plug-ins, e.g. to capture specific test bench data.

The way in which the report information is contained in the ASAM ATX report format imposes some constraints on identifiers used for some report elements. In particular test case names, parameter set names, test constant keys and environment names must comply with the schema for identifiers defined in ATX. The valid pattern for those identifiers is:

So identifiers

When uploading a report from ecu.test, the identifiers will be adjusted automatically (for more information see Why is the name of my test case in test.guide different than in ecu.test?). If you upload your report by other means, you have to ensure compliance with this schema yourself. Reports that do not follow this definition will be rejected on upload.

There is the possibility to externally upload additional files in advance in order to reduce the size of data, that needs to be uploaded to test.guide. However, there are some prerequisites to use this feature:

If these requirements are satisfied, you can upload additional files independently to the report itself:

To upload ecu.test reports to test.guide, a connection to test.guide must be configured.

This can be achieved in the following ways:

By default, the required parameters for an upload to test.guide (marked orange) are part of the settings in ecu.test.

To allow for a more flexible configuration only those parameters can be overridden with an external JSON file. The configuration parameter uploadSettings must point to a file within the ecu.test workspace. The keys correspond to the general settings.

Reports are transfered to test.guide using the ASAM ATX format in which identifiers have to comply with a given schema (for more information see Limitations imposed by the ATX format). Identifiers that do not fit the schema are adjusted to allow upload to test.guide. Following changes are made on report generation:

These changes are not only applied to test case names but also to test plan names, constant keys and some other identifiers.

To export to another test.guide instance, click on the Test management tool button and select the target instance. After that, just submit the form.

To export to a test management tool, click on the Test management tool button and select the tool in the drop-down choice. After that, just submit the form.

You can export PDF documents by clicking the button PDF report. test.guide contains a default test summary report template, which is ready to be used.

test.guide can also export Excel files for your test report data. You can generate it by using the button Excel.

On the right side of report filter result toolbar, there is a Record release button and also a drop-down menu offering further options. Using them, you can add the selection to an existing release or create a new release.

Coverage is created with the usage of a so-called coverage filter definition.

This specifies the criteria to be checked in XML format (see also the corresponding XSD). Examples and further information can be found in the TraceTronic-Example filter definitions for Cartesian Product and Filter Tree General

The predicates that can be specified are always transformed to individual result filters, so it is important that the criteria to be checked can also be found in the uploaded report as information, e.g. in the form of a constant or an attribute such as the requirements id.

If it only distinguishes the coverage, for example, in regression tests, in the form of a build number or another identifier, a placeholder can be defined in the coverage, which must be specified before execution.

If a list or an extract of the requirements to be tested is available, e.g. in Excel, then a script can be used to easily read the requirements from Excel and convert them into a test.guide coverage filter definition.

See also chapter: X2Coverage (Just “cover” everything…​)

It is also possible to import test suites directly from ALM as coverage via the ALM import button in the filter definition section.

Therefore, an ALM connection needs to be configured over the Project settings  Management tools.

This has the advantage, for example, that for very large test suites that take several days to complete, you can use the coverage to check whether all the results are available, i.e. whether all the tests in the planned test suite have been executed.

Coverage data can be queried and processed via the REST API.

However over the Export to button in the Coverage filter it is also possible to export the data to Excel or to create a PDF for a coverage report.

Releases are mapped as tree structures. A release can contain multiple sub-releases.

Release folders (the nodes of the tree) can be created to structure releases, whereas the actual release information comes from the releases (the leaves).

Releases and Release folders can be created over the Release overview. To fill a release with existing test data you can open a report filter in the test report management and either add the results to an existing release or record a new release.

When creating a release, various metadata such as ticket numbers and release attributes can be specified. This helps in searches, but also offers additional possibilities.

For example, if a release is directly linked to a Jira ticket, all created defect Jira tickets are automatically linked to the release ticket during a review. This makes it easy to track in Jira which bug tickets were created for the release ticket.

If attributes are specified, it is possible that these attributes are automatically mapped to coverage placeholders. So if a placeholder is defined in a coverage, e.g. for a buildNumber, and the value is set via the release attribute buildNumber, the release attribute is automatically set for the coverage.

When a release (leaf) is created, coverage can be assigned. This is also possible at a later date and the progress of the release is then calculated based on the stored release data.

For each release, the result distribution and, if coverage is specified, the progress is also displayed. If the progress is 100%, this means that all reports to be tested have been uploaded. Now you can see from the result distribution how many faults still need to be fixed or whether the release can be released in this state.

When a release is completed, it should also be locked. Because if a release is locked, further changes to the release, e.g. through reviews, are no longer possible. Only a user with the necessary permissions can unlock the release again if a change is still necessary.

Release data can be queried and processed via the REST API.

However, it is also possible to create PDF for a test summary report, by clicking on the drop-down under the Actions button and selecting Export pdf report.

Staging involves creating a Q-gate plan that describes the process for releasing a test artifact, for example.

If the plan has been developed, it can be started. This is similar to an automation pipeline that is designed and activated. However, the difference with Q-gate plans is that each plan is created for a specific process execution. If you want to use a Q-gate plan to validate different versions of a test artifact, you have to create a new Q-gate plan for each version of the test artifact. The simplest approach is then to copy the old plan when the commit is triggered and, if necessary, only change the parameters for the new test artifact to be tested and start the plan for the release.

Q-gate states only change after the plan has been started. After starting the plan execution changing, adding or deleting plan elements is no longer possible, but it is possible to add attributes to the plan or to update these. When creating a new plan, the following states exist:

A new plan can be created over the plan overview with add plan.

There are currently six different Q-gates:

Q-gates switch depending on their state.

Q-gates can have 4 different states (in order of increasing state weight):

Q-gates can be connected to each other, which means that a Q-gate can define as an input condition that one or more other Q-gate results must be finished. This is how the different Q-gate states are achieved.

Q-gates can be shared across Q-gate plans, projects or even test.guide instances. The state of these shared Q-gates is then accessible via so called Pull Q-gates. Technically, this is achieved via a REST API. For authorization, every shared Q-gate specifies which project roles are allowed to access it. For authentication, a technical user is usually used who must have at least one of the specified project roles.

The following procedure is usually advisable to use this feature. At the sharing party:

The other party attempting to retrieve the status of the shared Q-Gate must be provided with the connection data of the sharing test.guide instance (URL and project ID) and the authentication key of the technical user:

To get a quick overview over all connected test resources, navigate to Monitoring  IT view. Here you can find all connected test resources and status information of these, e.g. vital data of a node such as CPU and RAM usage. By using thresholds vital data is evaluated. There are distinct thresholds for serious and critical values. Values can be configured in the ResourceAdapter setup. The state of the entire node is evaluated using the worst single value state.

All test resources are sorted by their location and the respective test resource name. The full location path is displayed at the top. By clicking on a lower location level (the row beneath) only test resources matching this location are displayed. The view of a test resource shows the health state (provided by the TestbenchHealth Plug-in) on the top left, the test resource name in the middle and the by thresholds interpreted vital data of the test resource on the right. Below that, the current action is shown. As long as there is no active test execution, in the lower line on the right you can find an icon indicating whether there is a manual user activity on the test resource.

As soon as a test was startet, different information about the current test run is displayed. This can be for example the test project, test suite and test case name. The test execution status is also shown at the executing test resource. Therefore, the icon in the bottom line is hidden to show the established time and the number of finished test cases.

By clicking on a test resource name you’ll get a detail view of the selected test resource with all information provided by the configured plug-ins.

Use Monitoring  Project view to get an overview of all configured test projects and the assigned test resources. By selecting the project on the top of the list, you will get an overview of the last test executions of the respective project arranged by the test resource on which they were executed.

Yes, by using the Activity view. Navigate to Test infrastructure  Activity where you can filter test resources by their location and after that by their name.

Beneath the test resource filter you can also use time filters to show only a desired period of time.

After filling in the filter parameters, click on Filter to see the filtered results. On the bottom you now can see the activity distribution.

test.guide can link monitoring details such as configuration information to test reports uploaded to test.guide automatically. Therefore, the ResourceAdapter has to be running during the test execution and a test execution info plug-in must be configured. For EXAM this only works, when the parameter createAtxTestReport is set to true configured in the EXAM test execution info plug-in.

After the activation you can find monitoring details in the detail view of test reports on the bottom. You can use the provided information e.g. for comparisons of test runs to find out whether a configuration change caused a changing test result.

The aggregated test resource state aggregates all state information of a test resource. The information of connection state (gathered by the Heart-Beat Plug-in), monitoring state (gathered by different monitoring plug-ins, e.g. CPU Load Plug-in) and the component state (provided by the TestbenchHealth Plug-in, the Jenkins Agent Plug-in and the Execution Plug-in) are taken into account for the aggregated state of a test resource.

There are six states, which are shown in ascending order of severity in the following table.

Details about the state-affecting values of the individual plug-ins can be found on the respective test resource detail page.

We distinguish between direct test execution with ecu.test, test execution via user defined Jenkins jobs and generic task execution.

The direct test execution with ecu.test supports multiple types of test bundles:

Test bundles which are executed with user defined Jenkins jobs are freely designable.The job has to take care of the execution itself, including starting and stopping ecu.test (or another test tool) and uploading test reports to test.guide.

For the generic execution of tasks an arbitrary workspace needs to be specified. Within this workspace specified commands will be executed.

Test bundles can either be managed outside test.guide and provided by reference (see "location" and "scm" in the next section). Or test bundles can be uploaded to the artifact management and used from there (see "artifact" in the next section).

Execution tasks can be created via "Upload execution tasks" under Test execution  Tasks. There you can upload a JSON file (*.json) to test.guide that describes the execution task.The file must at least contain the following information:

The test bundle source has to be one of the following options:

The scheme of a test bundle’s requirement corresponds to that of a configuration property of a test bench from the Monitoring section. A requirement is therefore a triple with:

Optionally, there can also be details:

In all of these fields you can use the wildcards * (matching multiple characters) and ? (matching exactly one character).

You can find additional setting options, such as workspace dependencies, user-defined parameters or playbooks, in the Execution API Docs.

After pressing the "Create execution task" button, an execution task is created and queued according to its priority.

When uploading execution tasks via "Upload execution tasks" under Test execution  Tasks, the type of the tasks can also be specified. In addition to immediate one-time execution, a task may be scheduled as delayed one-time execution or recurring execution.

If you select the delayed one-time execution, a project job will be created which generates one execution task at a certain time. You have to specify the time of the execution task creation.

If you select the recurring execution, a project job will be created which generates new execution tasks at times that match a certain cron expression in the UNIX format. Additionally, you may specify a time of first execution of this recurring task. However, note that this may not be the actual first time of creation of the task. The first execution time will be moved to the first timestamp matching the specified cron expression after the specified time. If Check last execution is enabled, the recurring job will check if the last execution tasks created by this job are already finished. Otherwise, no new execution tasks will be created.

You can find and manage all project jobs under User Profile  Manage jobs.

Yes, this is possible.If the test.guide ResourceAdapter of a test bench has been correctly configured for test execution, the test bench automatically gets the following configuration property:

You can reference this configuration in the execution task XIL configuration requirements:

When using the test execution with ecu.test, it is possible to create execution tasks through test bundle sources uploaded to the artifact management. All uploaded artifacts that have the attribute "ArtifactType" set to one of the following values can be used for creating execution tasks:

To assign the artifact to a certain test bench (or a group of test benches), an additional attribute "RequiredXilType" can be added. In this case, the test bundle will only be executed on test benches with the following configuration property:

You can reference this configuration in the execution task XIL configuration requirements:

Creating an execution task through an artifact can either be triggered by clicking the Rerun as execution task on an artifact’s details page or by clicking the corresponding menu entry on the page Artifact Management  Artifacts.

Doing so brings up a new window. After pressing the "Create" button, an execution task is created and queued according to its priority. Furthermore, for every of these execution tasks a release coverage will be created.

Execution tasks can contain a "dependencies" section. Workspace dependencies reference sources that are not part of the workspace source itself but are required for the execution.

As a source for a workspace dependency you can provide an artifact in the test.guide artifact management, an SCM repository or an HTTP link to a file (e.g. in the test.guide file storage). You can specify whether the artifact shall be additionally copied to a local path on the test bench or whether it shall be unzipped (if the dependency references a zipped folder).

In the JSON description dependencies could look like the following:

The dependency source has to be one of the following options:

If you want to use the same SCM repository with different commit hashes, branch names or tags in different dependencies you need to use the copyTo option in each dependency that uses this SCM repository. Furthermore, it is not allowed to specify the same SCM repository in the dependencies, that is already used as source SCM of the task or playbook.

You can find a more detailed description of the workspace dependencies schema in the Execution API Docs.

The page under Test execution  Tasks provides an overview of all created execution tasks.

Each task can be downloaded, deleted or rescheduled. Furthermore, not yet started execution tasks can be edited or re-prioritized. Finished execution tasks provide a link to their test results.

It is also possible to filter execution tasks by their properties. The filter form can be used to create the filter.

In addition to the user interface on the page, a filter may be provided using URL page parameters. The following table gives an overview of filterable properties and their respective parameter keys that can be used in the page parameters.

Example: Assume you want to filter for all execution tasks that have the name "My task" and the state "finished". The according URL looks like the following:

If you want to provide multiple entries for the same property, just specify the page parameter multiple times:

Using this query, the page will show all execution tasks that have the name "My task" or "Your task".

After creating an execution task, all currently suitable test benches will be calculated based on the tasks XIL configuration requirements, and a corresponding state will be set. The following states are possible:

If the configuration of a test bench changes, the states and suitable test benches of all execution tasks are recalculated automatically.

There are two different ways to abort an execution task.

During the execution of an execution task the ResourceAdapter offers a context menu entry to abort the execution task. Using the respective button leads to an immediate abort of the task. The currently running test step will be aborted, no teardown will be executed and ecu.test will not be shutdown. If the execution plugin of the ResourceAdapter uses an automation strategy that is not blocked by a running ecu.test, it might be helpful to first take the execution plugin offline and then aborting the execution task.

In the overview page of execution tasks as well as in the details page of an execution task there is an action that aborts the task. If the task is not yet running, the task is immediately set to the error state and assigned with an error message. If the task is currently executed on any test bench, the task is marked accordingly and will be aborted soon. Aborting a task via test.guide will abort the current test case and continue with the teardown block. If the teardown block is already executing the abort request will be ignored and the task will be finished normally.

Both abort strategies do only effect the execution if the executing dispatcher of the execution plugin is the ecu.test dispatcher or the generic dispatcher. Aborting tasks using the Jenkins dispatcher is not yet supported.

In the additionalSettings section of an execution task it is possible to specify the expirationTimeInMinutes field. If not specified the execution task will wait infinitely until its execution is finally started on a matching test bench. If the expiration time is set, the execution task will be set to the error state after the specified time if the task has not been started yet.

Example of an execution task with an expiration time of 6 hours:

In addition to the control via the graphical user interface, a REST API is provided for all important functionalities of the execution distribution. The documentation of the API and examples for the JSON files described above can be found at the following link: Execution API Docs.

You can execute your execution tasks directly with ecu.test. The ResourceAdapter then executes the task on the test bench using the configured ecu.test and automatically uploads the test results to test.guide.

The test bench needs to have an ecu.test installed and the test.guide ResourceAdapter started. An ecu.test process must not be running, otherwise the test bench will be considered busy and does not accept new tasks. The following ResourceAdapter plugins have to be configured:

Example configurations for each plugin can be found in ResourceAdapter example config.

Referring to the requirements for the test execution, it is necessary to configure a path to an ecu.test executable using the ecutest.executable property of the executable plugin.

If there are multiple versions of ecu.test installed on the test bench you can configure these versions in the ResourceAdapter config using labels (for example ecutest.executable.<myLabel> = path/to/executable). See the ResourceAdapter example config for an explicit example.

For each label configured in this way, the following configuration property is sent to test.guide:

If you want to use one specific of the configured ecu.test versions for task execution, you can add an entry to the XIL configuration requirements of the execution task (or playbook). You can use the * wildcard for the "value" field as the ecu.test executable may be installed in different locations on different test benches:

See XIL configuration requirements in test bundles for more information.

The execution via Jenkins is based on Jenkins jobs, which have to be preconfigured by the user. The connection to Jenkins can be configured in the ResourceAdapter of the respective test benches (see sample configuration of the ResourceAdapter). Each execution task is linked to an executable test bundle, which is located on a network drive, in the test.guide artifact management or in a source control management tool. The design of the test bundles is therefore completely arbitrary and depends only on their executability in the job. Following three parameters are passed to Jenkins job builds:

The test bench needs to have test.guide ResourceAdapter started. If you use ecu.test to run the test bundles in your Jenkins job, no ecu.test execution may run there, otherwise the test bench will be considered busy and does not accept new tasks. The following ResourceAdapter plugins have to be configured:

Example configurations for each plugin can be found in ResourceAdapter example config.

The ResourceAdapter allows the execution of arbitrary commands on the test bench. Therefore, the generic dispatcher can be configured in the ResourceAdapter configuration.

The execution task that shall be executed on the test bench needs to have a playbook that only contains steps of type "command". Steps of this type contain a list of strings that represent the command to be executed. Furthermore, additional environment variables can be specified.

Example: the following command step will output the value of the additional environment variable NEW_ENV through a python script:

For more information on the command playbook step, see the Execution API Docs.

If you want to create a report for your command step, you can create a report as a ZIP in JSON2ATX format. The path where the ResourceAdapter will search for a report is provided via the environment variable TT_TASK_REPORT_DESTINATION. If ResourceAdapter finds a report at the specified path, it will automatically upload the report to test.guide for you. See here for more information about the JSON2ATX format.

The test bench needs to have test.guide ResourceAdapter started. The following ResourceAdapter plugins have to be configured:

In order to use the generic task execution the generic dispatcher has to be activated:

More information on the configuration of the Execution-Plugin can be found in the ResourceAdapter example config.

A playbook describes a plan for a test execution. It references what shall be tested (the workspace and additional data), what shall be executed (setup steps, test cases and teardown steps) and where it shall be executed (XIL requirements).

For the actual execution you can create one or more execution tasks from a playbook. These tasks can then run independently on different test benches. This way you can parallelize your test executions.

Playbooks can be created via "Upload playbook" under Test execution  Playbooks. There you can upload a JSON file (*.json) to test.guide that describes the playbook. The file must at least contain the following information:

An example playbook could look like this:

You can find a more detailed description of the playbook schema in the Execution API Docs.

After uploading, each playbook gets a revision number. If you upload a playbook with a specific name for the first time, the playbook will get the revision number 1. If you upload a playbook with an already existing name, the playbook will get the next higher revision number. This way, different versions of a playbook can be tracked, and you can see the whole history of your playbook in test.guide.

Another possibility is to create playbooks via the test.guide user interface. Under Test execution  Playbooks there is a button called Add new playbook. You then have to provide the workspace source of the newly created playbook. After submitting the workspace source, you will find yourself in the playbook editor, with an empty playbook.

In addition to test cases ("execution"-section), playbooks can also contain a "setup" and a "teardown" section. Both contain steps for preparing or cleaning up the environment that is required for executing the test cases.

In the JSON description, the setup and teardown steps look quite similar to the test cases. A step type that exists only in the setup and teardown sections is the configuration step. This step contains relative paths to *.tbc and *.tcf files which should be loaded before the next package or project step. The keyword "KEEP" ensures that the previously loaded configuration is retained.

Starting and stopping ecu.test can be done explicitly by placing ecu.test start and stop steps anywhere in the setup or teardown section. If no start step is defined in setup or teardown, ecu.test will be startet by the ResourceAdapter right before the setup section begins. Accordingly, an ecu.test stop step will be automatically placed right after the teardown section if no such step has been defined explicitly. Therefore, it is possible to execute one or more command steps before starting or after shutting down ecu.test.

An example of using ecu.test start and stop steps could look like the following:

You can also configure the ecu.test start steps to reuse a running ecu.test instance instead of restarting it. For this purpose the field "reuse" must be set to the value "always". Also, ecu.test will not be closed at the end of the current execution, so that it can be reused for the following execution task (unless ecu.test stop steps are used). This way you can save the time for starting and stopping ecu.test over multiple playbook executions.

You can find a more detailed description of the playbook schema in the Execution API Docs.

You can find an overview of the latest revisions of all uploaded playbooks under Test execution  Playbooks. Click on the details button next to a playbook to view the playbook details and switch to older revisions.

In both views you can also download the originally uploaded playbook via the "Download playbook" button. The prerequisite for this is that a playbook archive has been configured under Test execution  Settings.

After pressing the play button next to a playbook under Test execution  Playbooks, a new window will open that shows some settings for creating execution tasks from the playbook. Here you can adjust the following information:

After pressing the "Create" button, one or multiple execution tasks are created and queued according to their priority. Depending on the chosen distribution strategy, the created execution tasks have different contents:

In the playbook editor as well as via the test.guide REST-API it is possible to define playbook parameters. Playbook parameters have a name, which then corresponds to the name of the parameter in the created execution task. Furthermore, a default value can be set and parameters can be set as required. In addition, a description can be stored.

If a default value is provided by the playbook parameter you may override it by providing another value while creating the execution task from the playbook. Parameters containing a default value can not be deleted and will always have at least their default value assigned to them after creating the execution task. If the parameter is set as required (while no default value is set), an execution task can only be created, if a parameter with this name is provided (see How to create execution tasks from playbooks for more information).

It is possible to use variables of the form ${VAR} in the xilConfigRequirements section of a playbook.

Example:

When creating execution tasks from the playbook you have to provide a value for each used variable as parameter of the playbook execution (a parameter named TEST-BENCH in the above example). In the resulting execution task, the variables will be replaced by the corresponding parameter values. The creation of the execution task will fail, if you use a variable without providing a corresponding parameter. However, you can escape a variable that should not be replaced by using the form $${VAR}. Variables may be used in every field of a test resource requirement and even in the details of the requirements.

For more information on test resource requirements, see here.

The test execution provides ecu.test with some special global constants that can be used during package or project execution. These are at least:

If the execution task was created from a playbook, the following global constants are additionally provided:

If the execution tasks source is an artifact from the test.guide artifact management, the following global constants are also provided:

Additional global constants for package and project steps can be defined at the playbook step definition itself with the keyword "constants" (see the Execution API Docs).

In a playbook you can use command steps for executing scripts (see generic dispatcher) environment>>). The test execution provides the following environment variables in all script executions:

Additional environment variables for command steps can be defined at the playbook step definition itself with the keyword "environmentVariables" (see the Execution API Docs).

You can use some special variables as placeholders in playbooks. At the beginning of an execution via the ResourceAdapter, these variables are automatically replaced by their resolved values.

The test execution provides and replaces the following variables:

All of these variables can be referenced with the notation ${VARIABLE_NAME} in the following places of a playbook:

Some variables are not available for substitution in specific parts of a playbook/task:

The following table describes which states a playbook step assumes for which test results. A general error in this context means that the step could not be executed, or the test report could not be created:

The state of the execution task results from the aggregated states of all playbook steps.

The following image and description gives a high-level overview of the elements and steps involved in the workflow automation.

The "Add flow trigger" button opens the following flow trigger dialog.

When the trigger is created, it is active and will create flow tasks for matching events.

Some event types have additional filter options. The following table contains all event types and their corresponding filter options.

*supports wildcards

As the flow bundle source of a flow trigger you define the flow bundle which runs if a matching event occurs.

You have two options to provide your flow bundle as flow bundle source:

Every flow task has a state which shows its position in the flow task execution process.

The following flow task states exist:

The flow bundles of flow tasks are executed by ResourceAdapters with configured Flow Automation plugin (see section Requirements for details on the configuration). As the flow tasks run on systems with an active ResourceAdapter, all resources available on these machines can be accessed while running the flow bundle, e.g. present network drives.

For every flow task execution the ResourceAdapter creates logs. These logs are located in the subdirectory Flow_Automation_Resources\FlowKitLog of the ResourceAdapter's workspace directory with a separate directory for each flow task execution.

Every flow task execution creates the following log and report files:

The flow report for a flow task can be viewed on the detail page of the flow task. It is available after the execution of a flow task. The following screenshot shows an example for the flow report.

The report shows information about the flow blocks of the flow definition for the flow bundle execution. The report includes a graphical visualisation of the run flow definition and the following information for every flow block:

Reported states of the flow blocks can be the following:

The flow task detail page shows a links section. It contains links to resources connected to the flow task. There are 4 categories of links:

In the Interaction categories a link to a single resource is only shown once. If a resource would fall into multiple categories it is only shown in the highest category. The order of the categories is the following: Created > Updated > Related. If that resource was triggering the flow task it is also shown as the Trigger event.

For flow tasks, the following actions are available.

You can select a depository for log file upload, where log files from flow task executions will be uploaded. If no depository is selected, the log files will not be uploaded.

The uploaded log files for each flow task are linked on the flow task detail page.

When utilizing Jira, test.guide supports different authentication methods:

When configuring the Jira connection, please select Authentication Method: Basic Authentication if one of the first two methods should be used. In case you want to make use of a Personal Access Token previously created in Jira, set Authentication Method to Personal Access Token.

If you want to use the test run <> defect linking of test.guide you have to provide the identification number for the relation type that you want to use. The easiest way to get this number is by using the Swagger UI of your Jama instance at <jama-url>/api-docs. Try out GET relationship types and choose the ID of your need.

When working with project roles, individual permissions and system groups, it can be hard to find out which permissions a certain user has in the end. Therefore, when editing a project user and their permissions, you can see their roles and groups. Additionally you can use the "Show effective permissions"-button at the end of the form to get an overview of all their accumulated permissions.

Every report design must contain a JDBC Data Source named TEST-GUIDE_DataSource.

In the root element of the report design (select it by clicking on the upper entry in the outline view of the Report Designer), by using the Property Editor, some settings must be made. These settings are used to manage the report designs in test.guide and ensure that the report design can be processed correctly.