Programmers, Managers, Agile, and Failures: Software's Long Crisis

A UCLA assistant professor of Information Studies just published a short history of software engineering in Logic magazine — titled "Agile and the Long Crisis of Software."

It begins by describing Agile's history as "a long-running wrestling match between what managers want software development to be and what it really is, as practiced by the workers who write the code."

When software engineering failed to discipline the unwieldiness of development, businesses turned to Agile, which married the autonomy that developers demanded with a single-minded focus on an organization's goals. That autonomy is limited, however, as developers are increasingly pointing out. When applied in a corporate context, the methods and values that Agile esteems are invariably oriented to the imperatives of the corporation. No matter how flexible the workplace or how casual the meetings, the bottom line has to be the organization's profits.

But this has major implications, the essay's conclusion argues:

Could Agile even have played a role in some of the more infamous failures of the tech industry...? If a company sets a goal of boosting user engagement, Agile is designed to get developers working single-mindedly toward that goal — not arguing with managers about whether, for example, it's a good idea to show people content that inflames their prejudices. Such ethical arguments are incompatible with Agile's avowed dedication to keeping developers working feverishly on the project, whatever it might be.

This issue becomes especially pressing when one considers that contemporary software is likely to involve things like machine learning, large datasets, or artificial intelligence — technologies that have shown themselves to be potentially destructive, particularly for minoritized people. The digital theorist Ian Bogost argues that this move-fast-and-break-things approach is precisely why software developers should stop calling themselves "engineers": engineering, he points out, is a set of disciplines with codes of ethics and recognized commitments to civil society. Agile promises no such loyalty, except to the product under construction.

Agile is good at compartmentalizing features, neatly packaging them into sprints and deliverables. Really, that's a tendency of software engineering at large — modularity, or "information hiding," is a critical way for humans to manage systems that are too complex for any one person to grasp. But by turning features into "user stories" on a whiteboard, Agile has the potential to create what [software engineer] Yvonne Lam calls a "chain of deniability": an assembly line in which no one, at any point, takes full responsibility for what the team has created.

Other observations from the article:

• "Daily standups, billed as lightweight, low key check-ins, have become, for some workers, exercises in surveillance. "

• "The warts-and-all breakdown of Agile 'retrospectives' seems healthy, but I've watched them descend into a structureless series of accusations; everything depends on who's leading the team."

• One freelance developer in the article even argues that "As developers, IT professionals, we like to think of ourselves as knowledge workers, whose work can't be rationalized or commodified. But I think Agile tries to accomplish the exact opposite approach."

• "Some people I talked to pointed out that Agile has the potential to foster solidarity among workers. If teams truly self-organize, share concerns, and speak openly, perhaps Agile could actually lend itself to worker organization.
  
  "Maybe management, through Agile, is producing its own gravediggers. Maybe the next crisis of software development will come from the workers themselves."