Many software development organizations using Jira find themselves looking for test case management integrations. These integrations allow developers to track and manage software testing directly from Jira issues, reducing silos and the confusion of having multiple tools. Understanding QA and testing as part of the development process is essential, and being able to report on that information even more so.
Xray Test Management for Jira is one of the most popular and trusted Jira marketplace apps for test case management, manual and automated software testing, and quality assurance in Jira. It won the award for fastest growing marketplace app in 2019 in the Atlassian Partner Awards. Incidentally, Custom Charts for Jira won the same award a year later. Sounds like a good pairing!
Well, that pairing’s happened. Custom Charts works directly with Xray custom fields such as Requirements Status, Test Type, Steps Count, and any Xray JQL options. Note that the Xray fields available will depend on whether you are using Cloud or Data Center, but regardless of deployment, you will be able to report on your Xray fields in Custom Charts.
Requirements Status
To chart by the “Requirements Status” of your test issues, simply select the field from the Chart By drop-down list.
These charts can show the status of your testing requirements, providing a high-level overview of progress made toward testing. Requirements status reporting like the above can identify if more test requirements need to be written, and whether the associated tests have been run or not.
Xray data pairs well with Simple Search, too. Connect your Custom Chart to Simple Search and you’ll be able to dynamically filter down when viewing the dashboard. Get granular insights such as the test requirement status for stories only in the current sprint, or larger insights like what percentage of all the testing requirements have been run.
Test Type
Selecting “Test Type” as the Chart By option breaks down the segments into the default test types of Manual, Cucumber, and Generic. The Calculation in the below example has been selected as “Step Count” to show the total number of steps for those tests. The ability to calculate test count instead of issues adds more depth to the information, which allows these charts to be more useful for data-based decision-making.
Understanding the Test Type breakdown can also be useful at a high level to understand how testing is completed in your organization. If you are looking to streamline your testing process or you’re having trouble pinpointing the cause of bottlenecks, then identifying how much of your testing is manual is essential.
Testing Dates
Use the Time Between calculation option to see how long tests are taking. Xray comes with Begin/End dates, and calculating the average time it takes tests to be completed can help understand if there are any bottlenecks or limitations that need to be addressed. In the example below, we’ve even renamed this column to Testing Duration to make it clear to users viewing the chart what this value represents. We’ve broken it down by issue type, but this chart could also be useful if you wanted to display testing duration by assignee, component, or even epic.
Using JQL
If you have more specific data that you want to see, you can always use JQL to pull in data in a more advanced way. Use Chart By – Custom JQL to create complex queries for your data. For example, combine search terms to show issues with a “Requirement Status” of UNCOVERED and NOTRUN for the unreleased versions, like in the table below. This blends your Xray-specific JQL with Jira’s release reporting to provide insight into testing status for upcoming releases.
Get creative!
There are lots of ways to report on your Xray testing data in Custom Charts, including simple charts based on your testing fields, custom JQL, or date comparisons.
Take advantage of the different chart types available to you – we’ve shown examples of pie charts, tables, bar charts, and funnels, but your testing data can be pulled into any chart type. You can even take advantage of 2-dimensional charts such as stacked bar charts to look at testing status by version, sprint, or assignee. You can color-code your charts, and change the display names so it is readable to your team.
Both Xray Test Management for Jira and Custom Charts for Jira are available on the Atlassian Marketplace. You can also trial Custom Charts for Jira in our app playground.
Morgan is a Seattle born and raised lover of rain and software, particularly software that isn’t a pain in the bum (like some Atlassian tools can be). This is why she’s a Custom Charts for Jira superfan and jumped at the chance to contribute to the solution herself. She specializes in Agile, Scaled Agile, and ITIL in the Atlassian app space, loves a cross-country road trip, and is on a mission to find the cutest coffee shop in every town she visits.