It is estimated that up to 65% of large software projects fail disastrously. In 2013 the BBC wrote off £100 on an overambitious IT project. In 2016 an incorrectly configured software update destroyed the $286m Japanese satellite Hitomi, and the Scottish Police Authority scrapped £60m on an IT project because of “insurmountable flaws”. Also in 2016 the NHS revealed that its IT call handling and IT project is four years late and £40m over budget.

Failures occur not common for lack of skill, but because technology changes at a breakneck speed; hardware improves exponentially; software libraries, environments, and even programming languages are replaced before reaching maturity; and because markets dictate unrealistic deadlines. In this climate design and reengineering seem a luxury and there is only time for ‘firefighting’. Before you know it, your software is increasingly a hopeless mess.

Decades of studying government, health, open-source, and financial projects have taught us that successful projects have dedicated effort to reduce complexity. How? Unfortunately, there is no silver bullet (yet). In reality, each project has its own needs. A stable architecture can emerge from research-informed application of cutting-edge software tools and modelling techniques (not only UML). Bottlenecks can be diagnosed using software visualization and analysis techniques for reengineering and evolution. Flexibility can be built in using techniques such as design patterns. And the performance of critical functions such as security can be enforced using industrial formal methods. The knowledge required comes from experimenting with research-informed software engineering tools and practices and keeping up with the scientific literature. This is where we could help.

There is little point in writing lengthy documents. Nobody reads them — usually they’re obsolete anyway.

But some documentation must record clearly and ambiguously the following: What exactly can the user expect from the application you’re developing? Are these expectations realistic? Which changes in the requirements can be expected within 6 months from release? Within two or five years? Which strategic design decisions will ensure smooth transition to the next version? How should programmers see and understand these decisions? How can we ensure that future changes will not violate these decisions accidentally?

Instead of lengthy descriptions we recommend using one or more experimental tools and methodologies that communicate the answers explicitly, visually, and precisely. They may not easy to adopt but small investments can yield very large returns. Alternative forms of specifications can describe software function and structure clearly and precisely, and automated verification can effectively prevent the majority of bugs.