Functional Testing: Using ALM Releases Module

03 January 2022

TCT Computing Group is a leading functional and performance test consulting and training organization. With over 10 years as a Mercury/HP partner and reseller, TCT Computing has extensive experience implementing best practices in testing.


The increasing complexity of today’s applications often requires that they be tested on multiple platforms with multiple configurations. To add to this complexity, there may also be different versions of each application in the development, testing, and production phases. How in the world can anyone keep everything straight when dealing with so many moving pieces?

Something that can really help is ALM’s Releases module, which gives us a place to organize applications, application releases, milestones and testing cycles.

Basic Concepts and Terminology

In order for this to work most efficiently, it is important that we understand some basic concepts and terminology. First, it is important that everyone is working with the same ALM definitions. Remember that these definitions are specific to ALM. An ALM release represents a group of changes that are distributed to the end-users at the same time. Don’t define releases that contain too small of a group of changes. For example, don’t try to keep track of builds given to the testers by development every week. A milestone is a place in the release timeline that signifies the completion of the deliverable. One example of a milestone is when the database design has been completed. A cycle is a set of development and quality assurance efforts that achieve a goal within the release timeline. A release may have multiple cycles, and those cycles may overlap (and typically do). A scope item is a section within the release which defines requirements and their associated tests. For example, you could define a scope item for making sure you can successfully log into the application. Here you would define the requirements for logging into the application and then write the test cases for testing those requirements.

Setting up the Release Structure

With that understanding of the key terms and concepts, let’s walk through an example of using releases. The first step is to actually create the release structure. The easiest way to do this is to use the project plan, which will contain the start and end dates for the application releases and the different testing cycles. Start by navigating to the releases module that is found under the management bar. This module uses “smart” buttons that prevent you from making mistakes as you work. For example, because you can only place a release inside of an application folder, the release button is not enabled unless you have first selected a folder.

Let’s first create a folder for each of the applications in our project. Select the desired application folder and click the new release button. In the sample screenshot you can see that if your project is connected to a template project you can create releases from the template. You’ll be asked to provide a name for the release and to define its start and end dates. The dates will come from the project plan. Now you should highlight the release that you just created and click the new cycle button. We will give the cycle a name and define a start and an end date. Note that the cycle start and end dates must fall within the release start and end dates.

One final step is to create milestones, which are only available if you have ALM. First, you must give the milestone a name and a due date. Milestones are associated with scope items. Scope items are typically application features, to which we associate related requirements, tests, test sets and defects. Milestones are a component of Project Planning and Tracking and are used with key performance indicators to create a scorecard. This is how milestones can be tracked on a daily basis to determine the overall health of the project and its readiness for deployment.

Updating the Releases

Remember that project plans are living documents and will often change frequently. The release definition will now contain starting dates, ending dates, and due dates as taken from the project plan. If you find along the way that these dates need to change, please remember to make those changes in the release structure. For example, if a testing cycle is supposed to begin on October 14th but the testing group does not receive the application build until October 30th; we would need to change the defined starting date for that testing cycle. If this change is not made, it will appear that we had the full time available to complete that testing when in reality we were two weeks short.

Maximize the value
of your ALM

Govern quality and implement rigorous, auditable software lifecycle processes.

Assigning Requirements and Test Sets

The next step is to assign requirements to releases and cycles. Requirements describe in detail the application goals and needs, and are the basis for creating tests. All tests that are created should then be designed to ensure these requirements are achieved. Once the requirements have been defined and reviewed they are then assigned to releases or cycles. You don’t need to test every requirement in every cycle or even in every release. We may test a requirement for a new feature in release 2 and then not test it again until release 5. A requirement can be assigned to multiple releases and cycles because you may want it tested in multiple releases of the application. Requirements that are assigned to a cycle are automatically tracked within that cycle’s release. It is best to try to stay ahead of the game. In other words, try to assign requirements to a testing cycle prior to the beginning of that cycle. The ALM version control feature indicates what version of the requirement was tested in which cycle.

We have now used ALM to define the items to be tested and the associated dates. The next step is to assign the test sets to the cycles. A test set is a group of tests designed to achieve a specific testing goal. Doing this assignment will allow the testing group to determine how many tests need to be run during a testing cycle and allow them to handle any issues proactively. Test set folders are assigned to cycles, which indicates when the test sets need to be executed. As the tests are executed the execution progress is reported in the Releases module.

Tracking Defects

If any inconsistencies are found while running a test, they should be logged as ALM defects during the test run. Doing it this way ensures that defects are linked to the correct test step and all the way back to the cycle in the Releases module. As the tests are being run we can easily see the progress for that testing cycle and use those results to determine if the application goals for that cycle are being met. Results can be analyzed at both the release and cycle level. The best part is that everything is reported in real-time!

Organizing Assets

Having well-organized assets in ALM will make using Releases much easier. For requirements, we like grouping them by application and then by function area of the application. We also suggest trying to limit the folder depth to a maximum of three. Note that requirements need to be unique or it will invalidate the release metrics, so keep this in mind as you assign the requirements to cycles. Because tests are designed to test the requirements, typically the test structure will be close to the requirement structure. There may be a few “extra” buckets for basic things that are reused across features, but generally the subjects are similar to the requirement structure. The test lab folder structure is a little different from the others. We run tests in groups called test sets, with each one designed to accomplish a goal. The goal is to be achieved at that time because it is part of a specific release being tested when it is to be delivered. Because of this, the test lab folder structure should look exactly like the release structure.

Who's Involved?

Now that we know what the Release module can do for our project how do we implement it? But before doing this, we need to answer three questions:

  • Which project or projects should use the Releases module?
  • Who does all this work we have talked about?
  • Finally, who gets to reap all the benefits?

Which projects can benefit from using the Releases module? Although any project will benefit from using it, keep in mind that it involves extra work and depending on how often due dates change, it could be a lot of work! Although there is just one structure for each project, it does need to be kept accurate.

Who does the work? Usually this falls to the test lead. Although they will need the assistance of the requirement analysts to figure out which requirements are being delivered when, the actual ALM definition and maintenance work is usually done by the test lead.

And just who reaps the benefits of all this work? - the stakeholders. Remember that testing is all about providing accurate and timely information to the stakeholders so they can make knowledgeable release decisions for the application. The key is to be sure that it is accurate. Regardless of what application development model your project follows, remember that applications are not developed or tested using a “big bang” methodology – it is always done in pieces. In order to report whether application goals are being met on an ongoing basis, we need to know what exactly what should happen at what specific times. That’s what using the Releases module gives us.

Using the Releases module gives us a more exact way to use ALM and monitor the health of our applications over time. It provides a way to better exploit our testing assets by managing them efficiently over a specific time period