Time is valuable, Project to Product, reducing waste and Software Development Process

🎼Time is ticking away, tick, tick, ticking away 🎸

DC Talk Time is ticking away

"Lost time is never found again." 

Benjamin Franklin

"So teach us to number our days that we may get a heart of wisdom." 

Moses - Psalms 90:12

As I’m getting older (so are you ;-)), I’m realizing the value of time more and more. I have less extra time and have to constantly make choices (Should I watch a movie or go to bed or read a book. Should we sign up the kids for an activity or are we overwhelmed already? the list goes on). I’ve also been apart of many months of software development where the work never got used and was a waste for the company. I’m not the only one, our industry has a huge amount of waste!

I’ve been reading and thinking about How do we have more feedback, flow, continuous learning and joy as Mik Kirsten talks about in IdealCast episode 2? “how do I become more efficient and create more value?”, “What can I do as a engineer, when I’m not running the company, to help my company and other companies to thrive?”, “How do I apply the knowledge I’ve gained from books and articles over the years?”.

The problem with Software Development Processes

"Peter Moore (00:11:39): And what McKinsey did was they didn't study in 2018, and they documented it around the globe there was approximately $1.3 trillion spent on some version of, of digital transformation, 70% of which did not achieve its desired goals, which means $900 billion, ( laughs) did not get much of a return." ~[IdealCast episode 1](https://itrevolution.com/idealcast-episode-1/)

I’ve also spent some frantic days fixing and adding features to find out a few days or weeks later that the customer wants it to behave a different way. I put in a configuration value and added the new feature, hoping that someone will want it the other way. Really, we’re just guessing until customers actually use this.

I began reading Product to Project by Mik Kirsten while holding Ephraim (our 4th kid) in the NICU (he was born 5 weeks early in August, 2020. We were there for 7 days, but now he’s doubled his weight!).

Mik quotes work from Carlota Perez that we are in the middle of a disruptive age, moving from the Age of Manufacturing to the Age of Software. We are entering the “Deployment Period” 1 - pg 18 and no sector will go un-touched. Perez equates this to the Industrial Revolution and the “Age of Steam & Railways” [1 - pg 21]. We can already see this with Blockbuster, the rise of the tech giants and Tesla. Companies are becoming software and tech companies to survive.

One problem is that the tech giants have found ways to create a lot of value with huge teams and others like Sears, Kohls, etc are failing behind. Some like Nokia invested a lot in Agile training, but couldn’t right the ship.

The book points out these and many other problems with current management approaches for large scale. Many times IT is labeled as a cost center. We can learn from Lean Manufacturing, but software isn’t exactly the same. Software development isn’t connected to the “value stream”.

In his 2018 DOES [3] talk, he gives an overview of the book. He says that most of thrashing is where disconnection of code to the business. I highly recommend listening to this one, followed by Episode 1 of IdealCast 2.

3 Epiphanies from Project to Product

In the book [1] and his 2018 DOES talk [3], Mik shares about visiting the BMW plant and seeing how they approach manufacturing. He has three epiphanies that formulate his approaches to fixing the problems

#1 "Productivity declines and waste increases as software scales due to disconnects between the architecture and value stream" [1 pg 23]

#2 "Disconnected software value streams are the bottleneck to software productivity at scale. These value stream disconnects are caused by the misapplication of the project management model" [1 pg 23]

#3 "Software value streams are not linear manufacturing processes by complex collaboration networks that need to be aligned to products" [1 pg 24] (don't manage to cost savings). "More like an airline network" - See the graphic on [1 - pg 183]

Many readings, videos and thoughts

I’ve read many books and articles and watched videos over the last 14 years that informed and shaped my thinking. I’ve put these into the order that I came across them and added some comments

Agile and XP Programming

Thank you to my University of Minnesota, Morris professors for teaching this approach and unit testing!

Continuous Delivery - Jez Humble

An earlier book, before DevOps and deploying more often was a normal thing, on setting up tools to integrate code continuously. He also showed how you can continuously deliver changes to test and production.

Software Gardening articles on DNC - Craig Berntson - 2014

Building software is not like building physical buildings, it’s like gardening.

I read this in the DNC magazines (I miss getting the MSDN magazine mailed to me… I can’t remember if DNC was in paper for awhile now.)

Culture Videos from Spotify

These videos from 2017, continued to open my eyes to what a development culture and process could be. Feature Teams, feature flags and releasing on a normal schedule. These are still well worth the time to watch.

Spotify Engineering Culture part 1 Agile Enterprise Transition with Scrum and Kanban 1

Spotify Engineering Culture part 2

The Phoenix Project - Gene Kim

Gene Kim shares a story of a company going from bad culture, releases going wrong and payroll not working. The “guru” or “Yoda” shows up and shares the 4 types of work and three ways. A lot is focused on lean manufacturing, identifying the constraints and making work visible.

Omnitech’s book study write up

DevOps Handbook - Gene Kim

After I read the Phoenix Project, I wondered, “how can we actually do this?”. The DevOps Handbook provided learnings from others and topics to dig into.

Accelerate - Nicole Forsgren, Jez Humble, Gene Kim

The research and statistics show that the software systems we build and how we do them affect employees lives (measured by eNPS). There are direct correlations to higher performing teams and automated testing, continuous integration and continuous delivery.

Lean Startup - Eric Ries

Experiment, measure and pivot. Learn from Lean Manufacturing. Define your users with customer archetypes and use that for a guide.

Clean Architecture - Bob Martin

Architecture should allow easier changes and adapting. take into account cohesion and coupling. The business rules should be a the core. As developers we should fight for time to fix technical debt and improve architecture. Architecture isn’t something that can be all planned up front (building software is not the same as construction) and evolves based on trade-offs. John and I gave a presentation at SD Code Camp and Omnitech’s book study write up.

Working with Legacy Code - Michael C. Feathers

Omnitech’s book study write up

The Unicorn Project - Gene Kim

Gene Kim shares a newer story, closely following the heroine Maxine as she navigates getting a build to run to joining the “rebellion” to actually create value for the company. Companies can win in the marketplace and create value when the focus on creating value instead of cutting costs.

Mr. Kim introduces “The Five Ideals” as guiding principles:

• Locality and Simplicity
• Focus, Flow, and Joy
• Improvement of Daily Work
• Psychological Safety
• Customer Focus

Omnitech’s book study write up

Art of Value - Mark Schwartz

This was a free book on Kindle for awhile. He had some great insights and ideas. I’ve been meaning to write a full article about it, but this section will have to suffice.

Did you know that no one really has a handle on what “value” is for the business? We expect product owners to know, but even they are guessing. Often those we are building the software for don’t completely know what they need either. It’s also hard to measure it before we start. There are many metrics that people use, but none cover everything.

Value is different for profit, non-profits and government. Even bureaucracy rules can have value.

How do we measure Value?

We need to spend time and experiment to learn what value is for our businesses. Value could be features, technical debt reduction, learning, business does not always mean customers, agility.

He promotes experimenting, feature teams, Lean Startup ideas, Agile, DevOps, incremental learning and adding business people to tech or tech to business (think Google/Facebook/Microsoft that have CEOs that are also engineers).

Weekly Dev Tips podcast - Steve Smith aka @Ardalis

This podcast has been very helpful for development tips, to approaching problems and process. Start at the beginning and listen through.

Domain Driven Development (DDD) - Eric Evans

Ok, I haven’t gotten very far in this book. Ubiquitous language (same terminology in code as the business) and Bounded contexts stuck in my mind. We can use ideas from this book to mimic real life in code and discover how things should work. We need to work directly with people in the business to learn together.

At the end of Domain Driven Design: The Good Parts - Jimmy Bogard, he says to skip the DDD book (or focus on Bounded contexts) and get the Building Microservices by Sam Newman.

Idealcast - Gene Kim podcast

Gene Kim has a very interesting and insightful podcast [https://itrevolution.com/idealcast-episode-1/] talking to many different people, replying presentations while adding comments. I first heard about Project to Product in Episode 2

See my resources page for more links.

Some of my experiences

I’ve been a part of a year long project, we were rebuilding a system and seemed to be making good progress. They had attempted to estimate it ahead of time, but a 1000 hours is really hard to get right. Some things took longer. At the end, they brought in a new leader, my contract ended and I don’t think anyone used our software. The developers thought we were really close. How did we fail?

I was a part of another company for a 6 year span. They went from multiple software platforms that did similar things. They started a re-write, but realized they couldn’t just copy the functionality that was before. Over the years we learned about automated testing (unit and UI), moved to feature teams (members from multiple horizontal layers (server, UI, QA engineers, hardware and a PO)), with once a week component days (UI, server meeting on their own) and Scaled Agile Framework (SaFE) with combined planning.

Recently, I rushed to get a piece of a service working in a certain way. I made a mess, but got it working. Thankfully, I had some time to clean up the “technical debt” and got it into a more readable/maintainable state. A few weeks later, we found the customer didn’t even need this functionality. Could we have avoided the rush if we had the time and had connected these decisions to a value stream?

What is value and how do we create it?

This is a really hard question. It’s vague and depends on each situation. Experimenting, learning from those results and pivoting when needed seem to be the way to find this.

Project to Product

Many things are different between Project-Oriented Management vs. Product-Oriented Management Office. See the table on [1 pg 54]. Here are few examples:

• milestones and budget vs funding value, on demand with incentives.
• Time Frames (1 year vs multiple years)
• Success (cost reduction vs profit center)
• Risk (up front vs spread across time frame)
• Teams (people to work (Taylorism) vs work to people with feature/cross functional teams assigned to a value stream (Fordism))
• Priority (waterfall, project plan vs roadmap and hypothesis)
• Visibility (black box vs direct mapping to business outcomes and transparency)

This sounds like a great shift of thinking for software systems and matches the books and videos above.

The Flow Framework

The Flow Framework is Mik’s answer to the problem of the disconnect of software to the business.

“The role of the Value Stream Network is to provide an infrastructure for innovation for software delivery at scale.” [1 - pg 201]

The Flow Frameworks is a layer above Agile, DevOps and SAFe. It helps give visibility to the business leaders, align software development and puts the focus on the value streams.

There are 4 flow items (types of work): features, defects, risk (security/compliance), and technical debt. With 4 corresponding business results: Value, Cost, Quality, Happiness.

We need the Flow Framework to help answer these questions:

• Who is the customer?
• What value is the customer pulling?
• What are the value streams?
• Where is the bottleneck?
• What is the level of technical debt? (added from a presentation)

~[1 - pg 66]

Start with the customer. What are we delivering to the customers?

The Value Stream Networks allows “measuring the four flow metrics in real time across all value streams” Flow Velocity [1 - pg 98], Flow Time [1 - pg 102], Flow Load [1 - pg 106], Flow Efficiency [1 - pg 107]. Use data to help decision makers see where investments need to be made.

~ Screenshot from https://flowframework.org/about/

I’m not really doing this justice. There was a lot more to the flow framework and some I didn’t get all of. Get the book and read it, listen to the IdealCast [2], checkout the TaskTop website.

TaskTop’s Implementation - Integrating the tools

TaskTop Viz Demo is impressive and shows all the parts of the Flow Framework in dashboard. I grabbed a screenshot from the demo.

TaskTop available integrations

TaskTop Customers and Case Studies

TaskTop’s YouTube channel

TaskTop Hub Demo: Sophisticated cross-tool integration for ServiceNow Agile Development

There are some on-demand trainings available. Maybe it’d be worth paying for a workshop?

How do you define a Value Stream?

An Example/Attempt

Let’s say we are working for a company that sells it software system to other companies (we either host and manage it for them or they can install it in their servers). These companies sell services directly to customers. They like how our software is integrated and let’s them setup offerings, track sales opportunities, make the sale and provide customer support and integrate with billing and accounting.

Currently, there are several software UIs that support this. People from different timezones work on different UIs and backend services to support the software suite. This company often follows “Taylorism” (bring people to work) and team members feel disconnected. Developers will complete their work, wait for QA to test, then create an installer or drop for activations and support to move into the production environments. It often feels that the teams are competing against each other, waiting for each other and sometimes pointing fingers.

Can this be split up into value streams and cross-functional teams be created? Would this improve team work and change the focus to creating value and lead to better eNPS (Net Promoter Score and happiness at work)?

The business leaders and clients see the full suite as the value. Everyone would have to start to think differently.

We should start by thinking of the customer, how will this feature create value for them or how does this bug affect them.

TaskTop has some examples.

Can we move towards a flow framework without going all to TaskTop?

We’d have to:

• Move the Azure DevOps collection and teams to the same collection.
• We’d probably need to move code from several repositories, into one or at least in the same project in Az DO.
  • This is an opportunity to migrate from TFVC to Git.
• Change organization of teams, sprint ceremonies (planning, retros, demos)
• Need a Team of Teams or SAFe or something to keep the teams coordinated (Delivery Plans may be an option).
• Move incrementally
• Get the business leader buy-in to use TaskTop’s Value Stream dashboards and software
  • What tools are we using as a business to track issues, dev. Are they disconnected?

Can we apply ideas to smaller teams?

One take away I have so far, is Mik’s idea on how to setup feature teams. He says to focus on the “value stream” or the role of the customer. The team would focus on that section. Code and data could be organized and stored that way. Some day, deployments could be done by value stream instead of all as one big chunk.

Could we track the value stream items in Azure DevOps by applying tags and running queries?

What can I do as a developer?

• Keep your head up: Identify constraints, wait states, and technical debt, raise the need to address them
  • Don’t just stay in your lane (like Maxine in the Unicorn Project)
• Improve daily work
• Write automated tests
• Propose Safe to tries
• Create searchable knowledge centers, document knowledge, reduce the need to have to go to that one person (OneNote works really well)
• Move from TFVC to Git
• Learn about process and share what you learned
• Learn how to use the Azure DevOps tool better
• Share what you are learning: Teach others, talk things, share in code reviews
  • Teach, Train, Model
  • Spread the word to business leaders with books from IT Revolution ~ Mik Kirsten - min 58
• Communicate openly and honestly. The managers and leaders need to know the information (like the Andon cord in Lean Manufacturing)
  • information needs to flow from the edge to the center

Overall Takeaways

• the Value Stream Network sounds really great for large enterprises, but ideas can translate to smaller teams
  • DevOps and Agile isn’t enough, we need to optimize and align the entire business
• Start with the customers, what is value, measure
  • focus on delivering business value and not just the time frame. Ask why? and How does this add value?
• Lessons from Lean Manufacturing are helpful, but software is not the same
• Strive for Feedback, Flow, Continual Learning (the Three Ways of DevOps) and the 5 ideals (Unicorn Project)
• avoid waste
• may still need estimates and products (people want quotes for budgeting, especially when they are hiring you as an outside software developer or team), but realize that the risk of failure increases.
• software is “eating the world”
• We don’t want the tech giants to take over everything
• Look for the constraint. Attack that first, not just in development
• Make work visible
• “We need to scale the ways of DevOps beyond IT” [1 - pg 66]

Next Steps

It’s a big shift, so would require more thinking, but is very intriguing. I’m open to your thoughts, feedback and comments. Have you actually used TaskTop’s system? What am I missing?

For me/you

1. Listen to the IdealCast podcasts [1 & 2]. Keep listening, the rest of them are really good as well.
2. Get a copy of Project to Product by Mik Kirsten and read it.
3. Think about value and applying value streams at your company.
4. Look for constraints and attack them.
5. If you’ve read The Unicorn Project, watch the presentation from Mik Kirsten
6. Watch

References

---