Testing is crucial for SAP S/4HANA Cloud projects. You want to ensure that your business processes run as designed after go-live. In this blog post, I will explain key Test Management differences in SAP Cloud ALM and SAP Solution Manager. With this knowledge, you will have a better understanding of which of the ALM solutions fits better for you. This post is another part of my blog series ALM for SAP S/4HANA Cloud.
Foundation of Test Management is knowing what to test. You need a list of all test cases to check if all cases have been executed and if the results are OK.
In SAP Cloud ALM, you can assign test cases to the processes modeled in Process Management (see my corresponding blog). With this, you create test cases for these business processes. A test activity will be created for all process steps involved. You select which of them are in scope of your testing. With instructions and descriptions of expected results, you can explain to testers the actions to be taken while executing the test activities. Test cases, activities, and actions can be added manually or via Excel upload.
The SAP Best Practice content in SAP Solution Manager provides not only process flow diagrams to derive test cases from. As part of solution documentation, it contains Word documents with test scripts for the individual best practice processes. However, the test scripts are generic ones. You might need to adapt them to your process configuration, like you maintain instructions and expected results for the test actions in SAP Cloud ALM.
After collecting the relevant test cases, you plan the test execution. In SAP Cloud ALM, you set the preparation status to Test Case Prepared. This will make the test case visible for test execution. You can assign test cases to requirements or user stories if you want to focus on those test cases only that are assigned to an individual requirement or user story. SAP Cloud ALM does not provide the assignment of test cases to test plans and to testers yet.
SAP Solution Manager and the Focused Build add-on offer more options for test planning: Besides, the assignment of test cases to work packages and work items (which refer to requirements), you can collect multiple related test cases into test plans and test packages. This enables you to create individual test case collections for individual purposes, e.g., a specific work package. The status ‘Released for Test’ allows all test cases included in the test package to be executed.
In SAP Solution Manager, you can assign testers to the test cases. This allows users to get a better overview of which test cases are in their individual workload. And it is easier for you as test manager to trace if a delay is to be expected.
Test Scope Optimization
Extensive testing can cause huge efforts. Therefore, it is beneficial to reduce the test scope to those cases that are really impacted by changes. SAP Solution Manager enables this with the Business Process Change Analyzer (BPCA) and the Scope and Effort Analyzer (SEA). With BPCA, you can analyze the effects of a technical change to determine which business processes in your solution landscape are affected. You gain the option to focus your testing activities only on those business processes or process steps that are impacted by a change.
SEA predicts major cost and effort drivers of maintenance projects without the need to physically deploy software packages. You can use SEA in planning software changes in the initial stages of a project and use BPCA in later stages, during project implementation or when the project enters steady-state operations.
SAP Cloud ALM does not support such a detailed analysis for test scope optimization yet.
Automating testing activities leads to huge cost savings. You get rid of the cost for manual testers involved in recurring regression tests. This effect will be multiplied by the number of repeats of the tests. And you avoid higher costs by fixing errors in your production landscape as well as delays in projects due to errors detected late in the implementation project.
The public SAP S/4HANA Cloud contains a test automation tool. This tool comes with predefined automatic test cases that can be used and adapted. Content is delivered together with the software. No manual creation is required.
You would use the test automation tool for managing automated tests on SAP S/4HANA Cloud. SAP Cloud ALM integrates with this tool to ensure that you have access to the relevant test information (single source of truth). The interfaces used for the integration are public. Third-party test automation tools can use them to integrate with SAP Cloud ALM as well. These tools should be used for automated testing of other solutions, like SAP S/4HANA Cloud, private edition. I know that there are test automation tools preparing an integration with SAP Cloud ALM. However, I have not heard about the availability of such a scenario yet.
SAP Solution Manager provides Component-Based Test Automation (CBTA). It supports most of the SAP UI technologies. You can quickly create modular automated tests, which are easy to repair when damaged due to software changes. CBTA creates the script in the background with reusable components that are easy to repair and maintain.
You can extend the capabilities of the SAP Solution Manager test suite by using partner tools integrated with SAP Solution Manager via the Test Automation Framework. It provides you with an uncomplicated way to integrate certified test automation tools from partners and third-party providers with SAP Solution Manager. This enables you to easily test non-SAP applications and some SAP UI interfaces that are not supported by CBTA yet.
If you are a customer of SAP with a valid SAP Enterprise Support maintenance contract, you are entitled to use Tricentis Test Automation for SAP Solution Manager (TTA). This is not part of the SAP Solution Manager usage rights, but a separate content of the maintenance contract. But integration of TTA and SAP Solution Manager is available, such that you can use both in combination for automated testing of SAP S/4HANA Cloud and SAP S/4HANA Cloud, private edition.
After test execution, you want to gain an overview of the test results. This will indicate if the quality is well enough to start follow-up activities like a go live. In the project overview of SAP Cloud ALM, you find a Test Execution card that provides an overview of test case status. From there, you can drill down for details. Test execution information is also visible in the requirement traceability tool. In addition, the Test Execution Analysis can be used for analyzing the progress of test execution over time. If a test fails, you can create defects and trace their resolution.
In SAP Solution Manager and the Focused Build add-on, you can get more information regarding the status of testing in the Test Suite Dashboard. You can view the status and defects for individual test plans or waves. This gives you a more specific view than just watching the test case status of the entire project. Defect management is available as well. In Focused Build, there is a direct assignment between test cases and defects. This is not available in SAP Cloud ALM today.
Some companies require the validation of positive test results through electronic signatures for regulatory reasons. This is not possible in SAP Cloud ALM yet, only in SAP Solution Manager.
SAP Solution Manager provides more testing functionality than SAP Cloud ALM currently. But it is more expensive regarding hardware, installation, and operations. When deciding which ALM solution to use for your SAP S/4HANA Cloud, you should ask yourself regarding Test Management:
- Do we want to go for the pricier version, or is Test Management in SAP Cloud ALM good enough for us?
This is another entry for your decision table introduced in one of my previous blogs.
And if you do not need to start immediately, it is OK to postpone the answer. More SAP Cloud ALM functionality is planned for testing soon. Just watch the product roadmap.
The content of this blog post describes my personal view. Yours might be different. Please feel free to share your opinion in the comments below.