Optimizing Offshore Software Development with Agile

This is a guest post by Prasad Chaudhari, freelance java consultant. He was appointed as a project manager for the project mentioned below and played a role of ScrumMaster.

mattwilde66

The first prerequisite to going agile offshore is a mature and realistic understanding of agile at home. We’ve been practicing scrum on-site for several years including trained Product Owners (POs) and ScrumMasters, both of whom employ and leverage scrum artifacts.

Locally, we have six scrum teams: five software development teams, and one IT operations team. Each team has a sprint backlog, scrum board and burn-down chart. Automated tests and continuous integration are part of the daily business.

Our company has a universal understanding and acceptance of the Lean and Agile Value Chain, beginning with rough user concepts and ending with customer delivery. Without this basic building block, as the theory of constraints suggests, there will either be many unfinished stories in progress (i.e., because operations cannot make regular releases), or developers will starve for stories (because the business is unable to prioritize).

Last year, we decided to grow our software development capability by adding another scrum team in India. After a few iterations, reflections and process adaptations, the remote team became productive. We’ve been successfully running agile offshore for over a year now, and I’d like to share our key learnings.

Can offshore teams really be agile?

If it’s important to have a true understanding of agile and scrum on-site, it is absolutely critical to ensure the offshore team members have internalized the concepts of agile software development. Just knowing what scrum artifacts are, doesn’t necessarily mean that the team understands agile methodology. Every team member must comprehend the purpose behind each artifact. During the initial sprints, we explained at-lengths the purpose of each artifact and encouraged feedback sessions before each meeting to spot misunderstandings early.

The offshore team’s organization structure is the next thing to check. Usually, offshore service providers are structured hierarchically. During project resource staffing, we ensured that the organizational roles would not conflict with the project roles. Our employees are placed within one of three levels of organizational hierarchy. After being hired, new colleagues are briefed about the importance of their scrum role within a project, and how it does not affect their career progression within the company.

During every scrum meeting, we encourage all team members to speak openly about the project rather than only few member speaking for everybody. This frees up the more senior resources for more important architecture work and improves productivity.
Finally, the entire team cannot deliver quality if QA isn’t properly integrated. Testers must be empowered to ensure the delivery as per the definition of done. At the same time, they need to know that their job is not only to report and retest bugs, but deliver quality software together with the team.

Do offshore teams need extra scrum artifacts?

We use the standard scrum artifacts. Initially, we omitted the story preview meeting due to the scarcity of time available for POs, but we soon had to hold it in order to get better prepared stories for the “planning meeting 2”. We observed a very positive impact of stakeholder review on the productivity and morale of the team. Especially for long running (10 months) migration projects like ours, where the offshore team is not co-located with the business, the stakeholder review is a powerful artifact. Stakeholder interest for this meeting can be generated by asking them for what they would like to see/hear, adopting the presentation to their interests, or by encouraging them to click around the software.

Finally, if multiples teams are working on a shared code base, a scrum of scrums is very useful. Everyday, a member from each team visits this common stand up with representatives from the other teams. Each participant explains what their team is doing and if the team’s current activity might block other teams from working.

The Scrum Master Location Dilemma: On-site or offshore?

Simply put, the ScrumMaster should be located where the most impediments are. For us, most of the project impediments were at on-site. As our infrastructure and processes have been tuned over years to suit our local needs, they did not work well with the special requirements of working with an offshore team. Consider the remote availability of important support tools like Jira, our wiki, test databases, and maven repositories.

As I mentioned before, around 70% of development was done on-site and everybody was working on the same code base so it was critical to integrate all development activities. In our case, the ScrumMaster was on-site and his main task was to ensure that all external impediments like unavailable code repositories, broken builds, and lost database connections were addressed and resolved as soon as possible.

However, as the offshore team grew to 9 developers and 2 testers, the ScrumMaster needed to do this job in two places at once. To solve this dilemma, we created a hybrid system for fulfilling ScrumMaster responsibilities with a shared task list. Assigning these tasks to the offshore team, we made sure that the responsibilities are distinct and did not conflict with each other. This sharing of responsibilities will be expanded upon in the next section.

Offshore Planning Meetings slow and painful?

At project kickoff, “planning meeting 2” was an incredible test of patience for the on-site participant. A new team, poor story quality and a complex domain made it difficult for offshore team members to make any sure estimates. The root cause of the problem was lack of well-groomed product backlog and story estimates in hours. Additionally, the offshore company management had set KPIs for the team that it should finish all stories to which the team commits during the planning meeting. This measure, taken to improve the productivity of the team, was surely in the back developers’ heads while estimating stories. We solved this problem in multiple ways. At first, by enforcing story preview meetings and making sure the stories are well defined and small. Although it helped considerably, meeting times were still not reasonable.

Initially, we did planning meeting 2 immediately after the planning meeting 1, but later we introduced a gap of few hours between these two meetings. In this gap, the offshore team was given a chance to research, brainstorm and come up with the design. This reduced the time of planning meeting-2, and everybody in the offshore team had a chance to actively take part in the discussion.
We split the offshore team (9 developer and 2 testers) in half and appointed a team lead. To avoid the trap of hierarchy, we called them “story responsible” and rotated this role.

Now, planning meeting 1 is done with the entire team, but planning meeting 2 (the detail planning) is done with two teams. This has removed the wasteful pain of an entire offshore team trying (and failing) to plan all stories. With a team lead sharing the ScrumMaster responsibilities, we have also achieved linear scalability. The two team leads take over the moderation of meetings, retrospectives and solving offshore impediments.

Are you communicating well?

As we all know, communication is key to any process. Making use of the predefined, scrum artifacts ensures that communication takes place, but it does not ensure the quality of communication. Quality of communication is directly proportional to the quality of interpretation by the receivers. Ambiguous communication is differently interpreted by the individual project team members. We continuously improved our quality of communication by trying different communication channels and feedback loops.

After months of experimenting, we settled upon these three communication channels: Skype for video, telephone line for voice, and “live meeting” for desktop sharing. Having an uninterrupted voice connection was vital; especially when 11 people are all taking part in a meeting. Hence, we decided to use a land-line for voice communication instead of Skype or Google talk.

Although telephone calls are more effective than emails, certain documents (like guidelines) need to be written down. Most of the on-site communication was implicit, based on situational awareness and simple co-location – definitely not enough for an offshore team to start implementation.

Therefore, we discussed the requirements together with the offshore team and asked them to write the final specifications. The on-site team reviewed these during development and after completion to suggest improvements. To provide a strict framework for the offshore team, we wrote the guidelines for clean-code, pre-commit checklists, and operational wiki notes.

Creating Situational Awareness

One of the powerful tools scrum provides is the storyboard. Of course, for distributed teams, use of common physical board is not possible, so we use Jira with the excellent Greenhopper plug-in. As all 6 teams are working on the same code base, we use Teamcity for continuous integration and keep the build green by using pre-commit hooks. Individual developer commits are staged on the integration server, compiled with the latest revision of code, and, finally, run against all automated tests. Only after passing all of these quality gates, is the commit pushed to the source code repository for the rest of the team.

To summarize, adopting agile software development processes was instrumental to the success of this project. Developing working software in each sprint provided us, as well as company management, with confidence in the early phase of the project, and retrospectives gave valuable input for fine-tuning the process.

Having a fully functional PO role would have definitely helped in further improving the team’s productivity. But, we successfully achieved linear scalability by splitting offshore team into two and letting both team leads share some of the ScrumMaster responsibilities. For the next project, we are working on a more well-groomed product backlog, and better story point estimations.

Looking back, I believe the most important learning was continually improving every aspect of our development. After all, being agile means ‘readiness for change’, doesn’t it?