A question that comes up often in forums, and in planning sessions is, “How can we do Agile with Offshore developers?”
The answer is “very carefully.”
Back in 2001, Jason was a development manager at DeCenCorp, an office automation software company who was looking to push into the web. The executives were very interested in this effort, since they had seen the web go from interesting niche to lucrative business opportunity. They did not want to be last to the party. In fact, since their primary competitors were pulling back and/or shutting down, they saw this as the perfect opportunity to capture more market share. The Director of IT was given a mandate to get it done, so he turned to his first lieutenant (and development manager), Jason.
“Jason, the executive team is really hot and heavy on this new initiative. I need you to take a look at it and let me know how we can get it done”
“Um… ok. How about now?”
“Sounds good, we will start now and continue through lunch, my treat”
As Jason and Walter looked over the vision, Jason began to get more and more concerned. “Walter, this is big… we’re gonna need to staff up for this, at least 5 more heads.” Walter nearly spit out his panini, “What? Can’t the guys just carve out some time for this?”
“No Walter, they are already at full capacity. Besides, this is so big we’re gonna need dedicated folks on this, I’m thinking we need a headcount budget of at least another 400k… per year”
“Ok, I’ll send the message upstairs, but I don’t think they are going to like hearing this”
“Doesn’t matter if they like it or not, its the truth. Let me know what they want to do. If there is no budget, we can’t do it… period.”
A few weeks later Walter steps into Jason’s office, “Hey J, got time for a chat?”
“Sure Walter. Is this about the budget?”
“Yes. There is good news and bad news. The good news is you got your $400k. The bad news is that its going to be dispersed over the course of 3 years. They want this to be finished by 2004, and we only have $400k to do it.”
“But Walter, that’s only like, $133k/year. I can hire what… a dev and a half. 1.5 developers is not going to cut it, even if its 3 years of development time. Tell them to up the budget or we can’t do it”
“Well what Walter? I don’t like that look.”
“Look, I already committed to the project. I don’t know if this money is gonna be around next budgeting cycle. It could go from $400k to $200k, so I had to take what was in front of me. I committed to the project on your behalf.”
“You WHAT? Why would you do such a thing? Did you have an extra million in your back pocket or something?? Walter, there is no way in hell we can commit to this, you’re gonna have to go back and tell them…”
“J, chill out. Lets think it over for a week or so before we decide to pull the plug. There might be a creative solution for this”
“Why oh why did they put a sales guy in charge of engineering…..?”
“Because I have a degree in computer science, that’s why”
“Yea, but you haven’t built software in 15 years. You did sales engineering then sales then biz dev now engineering director”
“They wanted someone who could see both sides. Anyway, think over the budget thing and we will talk next week.”
Jason was livid as he left the office. He went straight from the job to the gym, hoping he could sweat and grunt the problem away. When he arrived at home an hour and a half later, his fiance could tell there was a problem, “Jason, why is your brow furrowed? Is everything ok?”
“Walter put me in a bad spot yet again. He committed to the executive team that we would finish a project with only 1/4th of the actual budget we need. I don’t know how he expects us to do this. My guys are already at full capacity.”
“Well, you know I don’t really understand the ins and outs of the software business and what you do, but I don’t like seeing my love all crinkled up like this. How can I help?”
“I’m sorry honey. I don’t like bringing work problems home. I’m just gonna let it soak for awhile and deal with it next week. Let’s watch a movie.”
The following week, Walter comes bounding into Jason’s office, “I got it! I have solved all our problems!” Jason was not eager to see Walter so upbeat, especially given the conversations from the week prior. “What is it now Walter?”
“How many testers do we have?”
“Can they code?”
“Yea, probably”, Jason hesitated, “But its not what they do. They wouldn’t be as good or as fast as the guys who do it day in and day out… what are you getting at Walter?”
“I was talking to a buddy of mine who works at BacCenter as a consultant, and he said companies are going gangbusters by sending all their testing to India!”
“Yea, India! Look at it like this.. those guys all speak English. A lot of them have engineering degrees, and they are cheap. I was thinking we outsource all the testing to India and turn our testers into developers for the new initiative. In that way, we get 3 testers in India at a fully loaded cost of about 60k per year. That means our budget REALLY buys us 3 testers in India for 2 years for 120k, and we get 3 more developers, our own guys, for FREE! I mean, we are going to have to spend some money to train them up, but that’s small in comparison. We can take the overage from what we save on outsourcing and use it as a training budget!”
Jason thought about it, trying to find holes in the logic as engineers are prone to do. He knew the test plans the testers used were detailed step by step directions that nearly anyone could do. As long as the spec was well documented, anyone could follow those plans and eventually write them given the proper guidance. “I’m thinking about this and I can’t find any fatal flaws”, said Jason. “But that doesn’t mean I’m comfortable with it. When did you think you wanted to start?”
“I was thinking we could take the rest of this quarter and have QA ramp down. Make sure everything is tested thus far, and make sure the test plans are rock solid…. solid enough to be handed off.”
For the next quarter, the testers worked diligently to ensure the orderly handoff off the manual testing. They were excited to embark upon a change in their careers, and even more excited to work in web development. Thus far, all the experienced developers worked on desktop software, but they didn’t have any web experience anyway. By mid August, the process was already in place. The development team created builds every week on Monday, and every night while the developers were sleeping, the testers in India were testing and creating bug reports that were available by the following morning. The process fulfilled the long-sought solution of “follow the sun engineering”. The executive team was impressed, and Walter soon got a promotion to Sr. Director.
Meanwhile, the new web initiative was in full swing, and while the QA turned WebDev team was making progress, it wasn’t as quick as Jason would have liked. “Look, Walter, I know we got creative with the last budget, but we still need 5 guys on the new web project. We only have 3 and they are junior, plus this is a new initiative so there is a learning curve to deal with. Now that you got your promotion, can you please look into getting us more budget?”
“We are coming up near the end of the quarter, Ill see what I can get us allocated starting in October. Maybe there is some money we will find when we close the quarter.”
Little did Jason and Walter know that not only would they not get the budget, things would actually turn for the worse.
On September 11, 2001, Jason called Walter as soon as he found out. Walter was actually in New York visiting family, but was unharmed. “This shit is crazy Walt. What is this gonna do to our business?”
“I’m not sure J. We had some good clients on Wall Street and were planning for more. We have to wait until the dust settles… literally, to see what happens next. New York is crazy right now. I wanna get back home, but all flights are grounded. I’m just gonna lay low for the week and try to work remote.”
When the October budget allocation rolled around, the news was worse than expected. Walter shuffled into Jason’s office to deliver the sad news. “Jay, I got some bad news..”
“Let me guess, I didn’t get my budget…”
“Even worse than that… your budget was cut”
“What? ! Ok…. so the initiative is off the table?”
“No actually they want to move forward with it”
“Walt, that doesn’t make any mathmatical sense… “
“They think the offshoring is working so well, they want to offshore the development of the new initiative. So cut the 3 jr engineers who are working on it. Hire 1 project manager and do the development and qa offshore. You will write the spec, it will all be developed in India, and the new PM will manage the thing.”
“Will the PM be here or in India?”
“Here, he will report to me and will also need to give status updates to the executive team”
“I really just don’t see how this is gonna work Walt”
“Just make sure the spec is rock solid and we will be fine.”
So over the next 3 months, Jason went about the work of writing the specifications and architecture. It was exhaustive but strangely rewarding. It was heartbreaking to have to let go of 3 good guys, but after the September 11 tragedy, budgets needed to be shifted. Peter, the new project manager, was hired in the interim and started putting together the project plan. By February, the project was in flight with the development and testing happening in India, and the project management and architecture happening in the US.
Fast forward to 2013 and over the past 12 years the company has amassed 40 offshore developers and testers. Things are ok, but could be better. Specifically, it takes too long to release software. Jason’s department is lucky to get out one solid release per year. There is an army of 5 business analysts who are 100% dedicated to writing specs reporting to Peter, but no matter how solid and detailed the specs, they cant get over the hump of one release every 12 months. Walter, who is now Sr. VP of Software Development, enters into the office of Jason, who is now Sr. Director of Software Development. “We need to do Agile. It will make us faster!”
“Have you talked to Peter about this?” Peter had since been promoted to Director of PMO. “Walt, we can’t do Agile, our specs, documentation, and compliance requirements are too high. Those cowboy Agile guys on the West Coast don’t document anything. Fine for a web startup but not for us. We do real stuff, and its too complex for Agile. Agile won’t work with us.”
“Well, we’re getting creamed in the market. It takes us a year to come out with new versions. That’s not going to work. Our customers are complaining that we need to be faster. I heard agile can make us faster. Plus I have heard from our customers that a couple of our competitors are already doing it. Why don’t you go to a certification class and let us know what its about “
A week later Jason comes back electrified. “Walt, this Agile stuff is gonna change everything. We have to rethink a few things, but its going to make us work so much smarter”
“Ok tell me about it… high level”
“Remember back in the 90’s when we were all in the same office, and we could get out updates every 30 days? We were nimble and would routinely get releases out 4… 5 times a year! Agile is going to get us back to that?”
“Yea, but how are we gonna do that when most of the development is offshore these days? We still have to do a lot of documentation and specs. How are we gonna do our specs and documentation with Agile? How are we gonna do Agile with our offshore teams? How are we gonna make all this work together? You need to figure this out. I’m not saying no, but I am saying I’m not convinced.”
The Trap of Optimizing for Cost
How can we do agile with offshore teams? This is a very very common question that comes up a lot. Both in online forums and with in-person client visits. First, let’s explore why companies outsource work. The number one reason, from my experience, is cost. Just like the situation with Walt and Jason, companies find themselves having to do more with smaller budgets, and offshoring held a promise of a skilled, English speaking workforce at a low cost. Since cost was typically the key driver for offshoring, and “strong processes and documentation” was seen as an antidote to the geography and time differences, many many companies over the last 20 years have invested heavily in both the offshore model and heavier process and documentation controls.
Since agile depends on fast feedback, high throughput communication, and a “leaning out” of the value stream, we rely on systemwide optimizations in order to increase the flow of value. However, having offshore teams means we have a constraint on communication, both geographically (location) and temporily (time). As a result, we are not able to optimize the whole system in order to increase system-wide throughput.
The reason most companies went to an offshore model is because of cost. Lots of companies know how to quantify cost but have no idea how to quantify value, and the tendency is to optimize what you can measure. As a result, we end up with a system that has a localized optimization (cost). So when we attempt to use a framework that depends on system wide optimizations such as lean or scrum, we can easily see the mismatch. This doesn’t mean agile is impossible with the offshore model, it just means we have to be mindful of the constraints in which we have to work.
Stay tuned for the next segment of Offshore Agile where we continue our story and talk more about implementing Agile with offshore teams.