Agile Release Train Retrospective

Within ASML we are using the Scaled Agile Framework (SAFe) at several departments. At the end of a Program Increment (PI) the Agile Release Train (ART) organizes the Inspect & Adapt (I&A). Part of the I&A is the retrospective and problem solving workshop. Myself and two colleagues (Muriel WeinsteinFerdinand Veldmans) organized this last PI. In this post I will elaborate how we did this. Such that others can benefit from the experience and set-up.


Like a normal retrospective you have with the Scrum team the ART retrospective is to review their practices, and identify ways to improve, e.g., drive continuous improvement with a lot of good will! The biggest difference comes with the number of participants. With the team you are just with ~6-10 people. With the ART you could be anywhere from 50 to 100 people or even more!


When we started with brainstorming on how to organize this we had a problem statement of one of the teams. This came out of one of their retrospectives and they shared this with the Release Train Engineer. They wished this would be known to the management team. As such, we wanted to do something with this problem statement within the I&A. During brainstorming how to set everything up we also went through the Liberating Structures. From these we found the “What, So What, Now What?” quite appealing to use for the I&A.

What? So What? Now What? (Summary);
Invitation; After a shared experience, ask the three questions.
Space; chairs for groups of 5-7, paper to record lists, flip-chart.
Sequence of steps; for each of the steps below, use 1-2-4-all.

  1. What happened or did you notice? (15 minutes)
    collect the salient facts
  2. So what is important, which conclusions are emerging? (15 minutes)
    collect the conclusions
  3. Now what actions make sense? (15 minutes),
    collect the actions

However, due the corona-virus pandemic everyone is still working from home. So we had to do everything digital. So instead of arranging chairs we created 5 digital meetings upfront for the 5 teams. And instead of a flip-chart we used 5 digital whiteboards. In addition, since we really wanted to make this also a problem solving workshop, we changed the first step of the sequence a bit. We decided to use a fishbone. We didn’t mention that we were using a Liberating Structure, just called it “root-cause and solution finding analysis”. 

Microsoft Whiteboard setup (times 5 for every team)
Microsoft Whiteboard setup (times 5, for every team)

Now as mentioned we had one problem statement of a team. We asked the other teams within the ART to also come up with a problem statement during their next retrospective. As such we would have a couple of statements. The idea was that at the start of the event every team would pick one of the statements. It was proposed to give every team a different statement, as such, preventing that all the teams would by coincidence pick the same statement. However, we decided to let teams choose themselves, since when they would pick the same statement it would also be very clear that this is important to people. In addition, it would be interesting to see if teams who picked the same statement would come up with either the same solutions or with completely different ones. 

To make sure that everyone also stepped a bit out of his comfort zone and ensure cross-pollination we mixed up the teams. So the teams were setup randomly, upfront, with different people from the various teams within the ART.

In the last step of the What? So What? Now What? you collect the actions. We mentioned to Scrum Masters, who would facilitate and coordinate every team to steer to a minimum of 1 action and a maximum of 3 actions. This way we would end up with a maximum of 18 actions. At the end of the “root-cause and solution finding analysis” (after a 10 minute break) all participants would join the general meeting. There every Scrum Master would pitch the actions of the team. After all the actions were pitched people could vote for the actions. 

We did not want to give every person only 1 vote but rather a bunch of points. These points could then be distributed over the actions as the people saw fit. This is a kind of score voting, and has been shown to produce the lowest Bayesian regret among common voting methods, even when votes are strategic.

No alt text provided for this image
In the results one specific action clearly popped out above the rest.

Luckily, Mentimeter has a question which (sort of) facilitates score voting, it’s called 100 points [1,2]. So we used this to facilitate the voting. After the voting the Scrum Masters & RTE will reflect on the actions and the way forward.

Below you find the complete timeline of the event looked as follows;

  • 11:00 – 11:05 – Everybody joins the general meeting
  • 11:05 – 11:10 – You will join a designated team
  • 11:10 – 11:15 – As a team you chose one of the four statements that are prepared
  • 11:15 – 12:00 – As a team you go through the root-cause and solution finding analysis (15 minutes for every step)
  • 12:00 – 12:10 – We have a small break to prepare the pitches and voting
  • 12:10 – 12:30 – Pitches of the actions by Scrum Masters and voting

Hopefully this was interesting and fun to read for you! We got some beautiful and nice results out of this set-up!

Cross post of;

Iteration & Refinement Board

When working during a iteration (or referred as sprint) within the Scrum framework the team can use a Kanban board. “Kanban” is the Japanese word for “visual signal” and is like Scrum, a framework within the Agile model. A Kanban board is used to visualize the work to be done as cards on a board, in different states. This allows you to easily see the “big picture” of where the project currently stands. As well as identify potential bottlenecks that could affect productivity. When you are using Scrum and Kanban in combination this is also referred as Scrumban.

Kanban board elements
An Kanban board with its key elements. Cards are meant to go from the left to the right column which denotes that progress has been made. The columns denote the different state the cards are in. Work-In-Progress (WIP) limits can be set per column. Swimming lanes can be used to separate different types of activities, e.g., teams, persons, feature, et cetera. Additional markers, visualizations, colors can be used on cards to further identify the work. For instance, a marker to identify how much work the task is.

One can choose to use an physical board or a digital board, or even both. Both have pros and cons. One of the benefits of having a digital board is that most digital boards automatically add all kind of details on the cards of the board. Such as the person working on the card, the amount of story points, the priority, the status and more. In addition, digital boards are flexible. You can filter, search or change them very quickly depending on your needs at that moment.

I want to share with you how we setup our Kanban board for the iteration execution and the backlog refinement. Keeping in mind that we use the Scaled Agile Framework. Within our work environment we use JIRA as the issue-tracking and project-management software. As such we have a digital board. There are many alternatives for JIRA, such as Microsoft Planner, Trello, et cetera.

Iteration Board

During the daily stand-up the iteration board (Scrum board, storyboard, Kanban board) is used to check upon the progress of the iteration. The progress is defined using 4 separate columns and different statuses. Furthermore we use swimming lanes to categorize work per feature.

ToDon.a.Iteration backlog, issues which were planned during the iteration planning event. Currently in waiting to be picked-up by the team.
In Progress• In progress
• Testing
• Analysis
Issues which are in progress. Testing and analysis are added as status to give a little extra context.
Waiting• Review
• Blocked
Issues which are in waiting, such as a review or being blocked for any reason, e.g., due to a dependency to a different team or supplier.
Donen.a.Issues which are resolved.

Additionally one could add an extra column called “Verify”. Team members put the card to “Verify” when finished. The Product Owner of the team asses the card to determine if it is really done. If so, he moves it to done. This makes the handshake between the Product Owner and the team more vivid.

Backlog Refinement

The backlog refinement board is created to check and make the refinement of the backlog more visual and explicit. During the iteration the board is impromptu checked by the team. Likewise as the iteration board it has different columns to distinguish the status of refinement. Additionally “exit” criteria are defined. These elaborate on when a card is allowed to be moved to the next column.

ColumnWhatExit criteria
FunnelIssues which are created but still need further refinement. Anything can be put here.• Assigned to Program Increment
• Feature is known
RefinementIssues which need to be refined more. • Title gives context
• Assigned to a person
• Acceptance criteria known
• Story points known
• Team definition of ready is applied
• Assigned to iteration
ReviewIssues which are refined by the team and need to be reviewed by the Product Owner.• Reviewed by Product Owner
• Handshake between Product Owner and team
BacklogIssues which are agreed upon and are ready to be picked-up in an iteration.No exit criteria, issue is now assigned to iteration and can be picked-up by the team.
RejectedIssues which are rejected by the Product Owner and the team.Not applicable