the agile engineer

Tom Gilb's Conference on Culture Change

Fri, 26 Jun 2009 07:40:20 -0400

I’m on the way home from London - flying high over the Atlantic in the relaxing confines of BA’s Club World section. I’m not sure how I got upgraded to business class, but after a long week I certainly can’t complain about a seat that lays down, red wine, some classic live Dead on iTunes and a gourmet meal on the way.

This year’s conference was on “Culture Change”. It was my third trip to London to visit Tom and the merry cast of thinkers who have somehow crossed paths with him along the way. I’m always amazed and impressed by the caliber of folks who attend Tom’s conference - an intersection of technologists from around the world who have some connection with the principles and practices of Evo and measurable value.

My talk was on Changing Organizational Culture in software development teams and included a mix of proven techniques (those I’ve used effectively) and emerging techniques (those I’m exploring now). The reception was good but I worked and refined it up until the last moment - something I’d like not to repeat next year. I received a good response - from a crowd that can be testy at times - but I wasn’t completely happy with the way I delivered the information. Definitely need to finish earlier and have more prep time next year.

In addition to getting to spend time with Tom and Kai Gilb, I also got to have some in-depth conversations with Rolf Goetz, Allan Kelly, Giovanni Asproni, Niels Malotaux, Clifford Shelley, Lorne Mitchell, Gio Wiederold, Marilyn Bush, David Stoughton and Yazdi Bankwala amongst others.

It was a special treat to meet Jeff Sutherland, who was in town teaching a Scrum class. I was able to join him, Tom and Solveig (Tom’s wife) for dinner on Sunday evening at a Thai restaurant in Tom’s neighborhood.

I think next year’s talk is on Value Delivery, this time I’ll try and prepare earlier!

Reflections on being World Class

Thu, 28 May 2009 21:55:16 -0400

For the last few months I haven’t been working hands-on on software development projects. Its been a nice break that has forced me to get outside my comfort zone in some respects, but at the same time I do miss working with software teams. Everything changed three weeks ago when I started two new projects in the same week. This is why my blog posts have fallen off as well as my reading!

In the first project I’m an engagement manager for a .NET development of a SaaS for a non-profit client. I’m taking over for someone who recently left our company and in doing so have inherited a system and development team. I’m coming up to speed on the application and working to secure some work building new features the client has requested. This project is a bit of a departure for me in that I’m purposely not hands-on with the code.

[Aside: After eight years of Java and the last few dabbling with Ruby, I made a personal decision two years ago to not learn .NET. Although much of the software my company develops for clients is .NET, and I think that .NET is a decent-enough technology, for me I saw little value in ramping up in a new language and platform that was, IMO, nothing really new or interesting. I know plenty of smart people inside and outside my company who are passionate about .NET, but it’s just not for me. I think this stems in part from the early years of the Internet when I saw Microsoft eschew standards, open-source and take an our-way-or-the-highway approach to working with others.]

The second project is far different - I’m leading a small team developing a high-level plan and business case for a client who wants to make enhancements to their real-time marketing platform (ironically, also developed with .NET). I’ve been involved with this company for over two years, on and off, and they’ve been extremely successful. I think I’m most proud of the fact that we introduced Scrum to them when they were a small start-up and now after two years of growth they’re still succeeding with Scrum. They still use cards on the wall, daily stand-ups, burndown charts and the works. Even their marketing and analytics teams use Scrum.

After initial success with their platform they want to grow revenues by $150M by 2015 and are looking for new capabilities that will enable them to do this (along with go-to-market costs). Our charter, from the CEO, is to “get from 30,000 feet down to 10,000 feet” in 4 weeks. Our team consists of myself, who knows the domain and can assess the technical impacts, and two colleagues who have expertise in finance and operational processes.

After our last interview today we headed out to grab a few beers and my friend Jimmy asked Ryan and myself, “What do you think you could be world class at?” It’s an interesting question that I must admit I’ve thought about before, but never discussed with others. My answer was that I thought I could be world class with solving complex problems, especially those involving teams designing and building software systems. Jimmy followed up and said, “So what are you doing now to be world class with this?” I didn’t have a good answer.

On the drive home and later this evening I find myself coming back to this question again and again. I’m not sure if my answer is right, but I am sure that I don’t ask myself this question enough nor do I share my thoughts with others about it.

While most of my blog posts are informative on some topic, this one is more of a reminder for you to spend some time reflecting on what could you be world class at and what are you doing now to make this happen? Give it some thought and if you’re not on the right path, figure out what it’ll take to get there.

The Agile PMO: Prioritizing by Business Value

Wed, 13 May 2009 06:21:19 -0400

One of the consistent themes in my work is this notion of making sure you’re doing the right thing before doing the thing right. While the word juxtaposition is slight, their implications are anything but.

Last week Sanjiv Augustine came to town to give a talk at Agile Richmond (photo above). In addition to being an author, entrepreneur and agile practitioner, he’s also a heck of a nice guy. We enjoyed chatting over dinner on a variety of topics and I came away thinking it was nice to connect with an agile thought leader who lives in the area. Sanjiv’s talk was on the The Agile PMO - From Process Police to Adaptive Governance.

The main takeaways for me were the concepts of optimizing your project portfolio for throughput not utilization and the governance models for supporting an Agile PMO. Sanjiv presented a very tangible way to make this happen, including ideas for how to prioritize all projects by business value and then to limit the number of active projects to the capacity of the delivery team. This typically means canceling lower-value projects and re-distributing those resources to higher-value projects so they can be completed quicker. The end goal is faster time to market for high-value projects - not optimization of resources across all projects.

One surprising point I learned was that, according to one study, 53% of project prioritization is driven by politics. In this climate, adapting more open and transparent project prioritization activities can be extremely difficult. There are powerful people who like the way things are and may have “pet projects” that are near and dear. But that doesn’t mean we shouldn’t try and prioritize by value - it just means bringing change may be harder than we think.

While Sanjiv’s points were all familiar to me, the way he presented the information was new. He helped me understand the concepts and how I can do them in practice. During his talk, I couldn’t help but think of a few Evo concepts that could be included in the adaption of his Lean-Agile PMO framework.

The next day over lunch my colleague Jimmy and I kicked these ideas around and agreed to try and put them into practice at our next opportunity. Since Jimmy leads up our Operations and Technology Strategy solution that helps clients manage their project portfolios, he’s in a good position to try them.

Specifically, we discussed two “plugins” that could be used with the Lean-Agile PMO based on Evo:

Quantify Business Objectives
Project Prioritization using Impact Estimation

Quantify Business Objectives
Sanjiv presented a simple framework for prioritizing projects based on weighted values for certain criteria. Projects get weighted scores of 10, 20, 30, etc. based on criteria and when added up, the project with the highest score is the most important - resources should be focused here. He said this is one example, and there are others, for how to prioritize projects in the Lean-Agile PMO. His explanation reminded me of the plugin architecture I’m familiar with in software - how different prioritization methods could be used to extend the Lean-Agile PMO approach.

With my Evo background I recognized there was, I’d argue, a more robust way to define business value that would allow a Lean-Agile PMO to prioritized by business value. This plugin would use Planguage to quantify the business objectives first and then prioritize the projects through the use of Impact Estimation. Doing so will show us how well we think the project will meet our business objectives, and at what cost.

I can use examples from my Value Delivery article to illustrate the concept. Let’s pretend we want to prioritize our portfolio of projects based on business value so that we can ensure our resources are focused on the highest priority initiatives.

Step 1 is defining business value - which means defining the business objectives (aka results) of the business unit our PMO is working with. Instead of putting weighted scores next to a business unit objectives such as Increase Market Share or Increase Monetary Donations, we’ll go a step further and define them using a scale (what to measure) and meter (method to measure):

Step 2, is to understand where we’re starting from (benchmark), what’s success (target) and what’s failure (constraint):

Charting the results helps us quickly grasp which objectives we are doing ok with and which ones need our attention now.

So why this approach over a weighted scoring system? Yes, it is more work, but I’d argue not that much more once you understand the concepts. So is it worth it? I’d argue that with project portfolios in excess of millions of dollars, the decisions around which projects to fund is a critically important one that has an enormous impacts. So in this case, a little more effort can be justified.

If you want some additional arguments, see “What is Wrong with the Weighting Process for Determining Priority” in this paper.

Project Prioritization using Impact Estimation
With our objectives defined, we can use Impact Estimation for evaluating how well our proposed projects will help us meet our objectives, and what resources are required to make this happen. Here’s an example that illustrates which of the three proposed projects (Recurring Payments, Facebook Integration, Image & Video Uploads) will deliver the greatest business value:

The important point is that projects aren’t evaluated on just the expected value (Total Objective Impact), but the ratio of value to resources (Benefit to Cost Ratio).

For example, in Figure 5, the Facebook Integration project isn’t selected because it has the highest expected value. At 110%, it’s actually 3rd with respect to value delivered. But, it does this with 40% of the resources - compared with the other projects that use 70% and 100% of the resources to deliver value. Thus it represents the best “bang for the buck” and should be prioritized high in our portfolio.

Summary
For those in charge with prioritizing the projects in your portfolio, next time consider using a more robust method than simple weighting based upon arbitrary values. If you need help, email me and I’ll provide you guidance.

Remember: Don’t just sort your portfolio, engineer it to delivery business results!

Agile Engineering @ RJUG April meeting

Fri, 24 Apr 2009 20:24:32 -0400

Thanks to everyone who came out to the Richmond Java User Group meeting last night. I saw mostly new faces in the crowd but as always it was good to re-connect with old friends and colleagues.

I felt the dynamics were good and everyone was getting the main concepts. People seemed to enjoy the group exercises and I enjoyed walking around and talking with people (and snapping photos on my iPhone). I overheard lots of interesting discussions about different was to define response time, availability and throughput qualities.

I wish one of my points about Impact Estimation had worked out better with the examples. It’s the point that the design idea with the most performance isn’t necessarily the best one - it’s the ratio of performance-to-cost that matters most. Well, I picked 3 design ideas at random from what the group’s identified, but it turned out the design idea with the most performance was also the one with the best performance-to-cost ratio! Doh! Oh well, it just worked out this way.

I’ve uploaded the slides and Impact Estimation spreadsheet in case you want to download. As I mentioned in the meeting, contact me if you’d like support implementing these ideas on your project. I’m happy to help anyone trying them in practice and would enjoy the feedback based on your experience.

After the success of the talk last night I was thinking how bummed I was this talk got turned down for the Agile 2009 conference. I think the agile community could benefit from new approaches to defining and measuring system qualities, especially approaches that integrate with Scrum. But I guess not. I submitted it for the Developer Jam stage and the rejection email said they had 100 submissions for 26 spots. Tough odds I know, but still never good news to hear.

The Fed: Going Agile out of Necessity?

Wed, 15 Apr 2009 08:39:21 -0400

During some recent time off I was reading an interesting article in the Washington Post about “How Bernanke Staged a Revolution” at the Federal Reserve with his guidance during our shaky economic times. It turns out Ben Bernanke, the Federal Reserve Chairman, is a scholarly expert on the Great Depression. He contends the major lesson learned from the Great Depression was that during a financial crisis like the one we are facing now, the government has no choice but to act quickly. After all, it was only World War II that brought the US out of the Great Depression, 12 years after the stock market crash of 1929.

This time around, Ben’s not about to let that happen on his watch, thus the government intervention into AIG, FreddieMac, FannieMae, CitiGroup, General Motors and a list of others. The argument goes the price of action is expensive, but history tells us the price of inaction could be much, much more expensive.

So what does this have to do with agile? Here’s the quote from the article that caught my attention: 
“A decade ago, when the Fed wanted to know how it might deal with technical issues created by the government's need for fewer Treasury bonds, a study of the issue took 18 months and involved 73 economists across the Fed system. The result was a 165-page report.

This year, the Fed has made decisions of similar complexity and importance over a single weekend.

The pressure to act fast has, by all accounts, come from Bernanke himself. His relationships with staff members are warm, dating to his days as a Fed governor when he ran the equivalent of faculty seminars to help young economists develop their research. But sources who have been in contact with Fed staffers also say that he has prodded economists and lawyers to move faster and think more creatively to execute new programs being enacted.”

In reading this, I can’t help but think, “A decade ago, the Fed was waterfall. Now they’re Agile. Out of necessity.”

The article goes on to describe Bernanke’s leadership style and his emphasis on setting goals and challenging his teams to “Find a way to do this.” He breaks down hierarchical barriers and fosters open and honest debate from a diverse set of team members. He challenges his team to find creative solutions to problems quickly.

Reminds me of a well-run Agile project!

With the Fed making swift decisions in days and weeks, not months and years, I’m curious if their IT organization as adapting as well? If Federal Reserve IT projects are traditionally waterfall, perhaps the new economic realities are just the impetus for going Agile?

Master Craftsman as Leaders

Thu, 9 Apr 2009 08:03:57 -0400

I’m catching up on my reading during a little time off and one of Bob Martin’s recent posts on Master Craftsman Teams caught my attention. To be honest, I’m a bit behind on my reading and have only heard second-hand all the brew-ha-ha over SOLID, but I’ve been a longtime fan of Bob and generally agree with most everything he writes.

In Bob’s post he argues for having a team centered around one Master Craftsman with a few Journeymen and many more Apprentices. He concludes that using this tiered configuration while using Agile principles and practices is a more economical option and will yield better results. As I read the article though, one quote really captured my interest:

“The master also sets technical direction and overall architecture, and makes all critical technical decisions. However, the chief job of the master is to relentlessly drive the journeymen towards higher quality and productivity by establishing and enforcing the appropriate disciplines and practices. From the master’s point of view, it’s “my way or the highway.”

Amen.

Master Craftsman must be more than technologists, they must also be leaders of men and women. They must foster collaboration and drive teams towards delivering working, tested software continually. They must be adept at working with people, teams and organizations, not just code.

In my career, I like to think I have done this on a large enough project for a long enough time to appreciate what leadership is about. I have also witnessed leaders like my late friend Steve Metsker guide a successful team using a style that was his own. With seemingly no effort Steve would proactively identify and diffuse situations and keep everything running smoothly - from the system architecture to the team’s biweekly deliveries. From writing just the right amount of code to helping the executives understand complex technical details using simple analogies. Steve was a true Master Craftsman.

I have also seen projects where there was no Master Craftsman and “design by committee” was used. Team members floundered, nobody took responsibility and work limped along. Not surprisingly, morale wasn’t great either. The leadership vacuum that existed without a Master Craftsman was obvious.

In reflecting on the contrasts between the two and on what I thought made some team’s successful, it was as much about the Master Craftsman’s ability to organize and lead people as it was their pure technical chops. Don’t get me wrong, slinging quality code is important. But a Master Craftsman is not only about their technical decision making, they must also learn to:

- Gain the team’s respect and confidence by demonstrating knowledge and experience in ways that are engaging and inclusive, not arrogant and dismissive.

Put teams ahead of individuals and reinforcing daily the idea that we all succeed or we all fail, together. 
Hire the right talent at the right time and ensure they fit with your team culture. Personally, I won’t hire “cowboy coders”, no matter how bright. I hire for attitude, aptitude, then skills. 
Lead by example and demonstrate you can walk the walk, not just talk the talk. Architects write code. No exceptions.

Know when to roll up your sleeves and do it yourself, when to step back and let someone else on the team take the lead and when to engage an industry expert. 
Understand everyone’s role at a detailed-enough level to identify new collaboration, automation and growth opportunities within the team. 
Define standard work that includes simple policies and easy-to-follow, repeatable processes for developing and integrating code and tests (ideally automated). 
Set the quality bar high from day one and setting team expectations that drops in quality will not be tolerated.

Carry a sharp saw with a technical prowess in the latest tools, designs and technologies that’s backed by experience.

Foster modest competition amongst your team to ensure star performers and teams are recognized and others know how they too can be a star. 
Simplify, simplify, simplify by designing risk out, not testing it out. Simple designs with fewer moving parts are more reliable and less prone to failure. They’re also easier to understand and explain to others. 
Have a positive attitude and a will to succeed, even when times are difficult and things break in unexpected ways.

What do you think? Suggestions for inclusion?

Value over Velocity

Wed, 1 Apr 2009 14:43:21 -0400

There’s been a recent discussion on the Scrum Development list about velocity - the Agile term that represents ”the sum of the estimates of the stories [points] that were completed in an iteration.” Within the Agile community, velocity is the primary metric used to report progress and success. There’s no other single Agile metric that gets as much attention, discussion and scrutiny as velocity.

While following the velocity discussion, I wondered, “Is velocity really the best metric for Agile teams and organizations (including Product Owners and ScrumMasters) to focus on?”

After some thought I concluded No.

Why, you may ask?

Answer this question: What’s most important for the people financing your project-product-service? Is it the benefits they expect from the solution, such as increased revenue, market share or operational efficiencies (we’ll call this business value)?

Or is it the estimation accuracy of the team working on the solution (velocity)?

Let’s try another: What’s most important to your customer, the person exchanging money for your project-product-service? Is it the benefits they expect from the solution, such as ease of use, reduced manual data entry or improved forecasting accuracy (we’ll call this customer value)?

Or is it the estimation accuracy of the team working on the solution?

See where I’m going here...

I’m not saying velocity is useless and we shouldn’t track it. I track it on my projects and agree it helps with capacity planning future iterations and investigating “what happened” for particularly good or bad iterations. It can also help manage expectations about what’s realistic in a defined time period for a team.

Unfortunately though, velocity has become the primary metric used by Agile teams for measuring success. I think this is a shame. Instead of focusing teams and organizations on estimation accuracy we should be focusing them on value delivered.

There are ways to measure value, as illustrated in my recent article. But this requires focusing on the real stakeholder objectives that represent business and customer value. It requires augmenting story-centric requirements approaches with value-centric requirements approaches. It requires measuring success in terms of value delivered.

Kai Gilb’s recent work on the role of the Value Product Owner illustrates nicely how Stakeholder and Product Values can be used as the primary measures of success and integrated into the Scrum process. Instead of just planning and prioritizing at the user story level, the Product Owner plans, prioritizes and tracks results at the Stakeholder and Product value levels, thus getting a more holistic picture of success (business and customer value delivered).

I’ve also recently discovered James Shore’s term “value velocity”, which uses customer-defined “value points” at the story level to estimate the value before a story is scheduled for a release/iteration. Value velocity is therefore reported as the number of value points completed in the release/iteration.

While I think this is an improvement over not attempting to track value at all, I would argue 1) actuals at the stakeholder/product level are better than estimates at the story level and 2) actuals can be obtained without too much difficulty - if you quantify success and failure using scales and meters that are relevant and affordable (based on your budget). It’s doable with a little effort and I would argue is well worth the limited investment required.

So the next time you find yourself using velocity, think for a second, “Do I mean estimation accuracy or the value of what was delivered?”. If it’s the latter, then perhaps velocity isn’t the right metric.

Put the Hatchet Away!

Mon, 23 Mar 2009 12:25:22 -0400

The first of four columns for gantthead.com, the online community for IT project managers, went live today. Here’s my article “Value Thinking in Lean Times: Put the Hatchet Away” (free registration required).

The article challenges IT leaders to avoid across-the-board “hatchet cuts” and instead use a “scalpel approach” that focuses on delivering bottom-line results. Implementing this can be done with simple-yet-effective Value Delivery Policies for IT:

Top Critical Objectives
Deliver Value Early and Often
Objectives Focused Contracting
Value-based ROI

Implementing these policies can be done by progressive IT leaders following simple practices:

Help the business define their critical objectives
Prioritize critical objectives and get focus
Align teams on delivering value incrementally
Measure ROI often
Continually reduce waste and improve flow

While simple in concept, mastering these policies and practices can take years. But the effort will yield results, often substantial results very quickly, and more importantly these are changes that will position IT leaders well as the economy turns around.

So, think these Value Delivery policies could be implemented in your organization?