A Tuner Day

The next sentence is a lie.

I don’t believe in performance testing.

It’s a lie for two reasons:

  • We recently gave an apprentice a task that was essentially learn how to performance test the product
  • I do a lot of performance testing as part of my development

Measuring Performance Is Not An Outcome

Notwithstanding the task that has learn how to performance test as its outcome, the main principle I operate is that delivering performance test results is not, in itself, an outcome.

The general principle I have is that a test report that shows us thinks are not good enough is a waste of paper.

The objective must be to make things become good enough.

In the case of performance, that means deciding how good good enough is, ideally with the aim of the Go Down The Pub method, then make the software achieve that performance.

Achieving Performance Is An Outcome

This is why I favour stories that have the word tuning in them. Tune the software to make it fast enough. We generally do this in stages:

  • Coarse grained – make the software get to the right order of magnitude – approximately in the 1000’s, say
  • Fine grained – make the software fast enough against much more rigrous acceptance criteria
  • Rethink – if the software’s nowhere near fast enough, then renegotiate practical acceptance criteria, or rearchitect

Ideally we’re performing these tests close enough to the creation of software for early warning, and ideally performance is fundamentally negotiable. In that a car that can only reach 50mph maximum is still a car you could take to the shops, even if you plan to do a longer road trip in the future.

If a system must be able to achieve very high throughput from the get go, then you need to include continuous performance testing in the project, from the very get go to prove it will go fast enough as a definition of done on each significant increment.

If All Else Fails

If the architecture is right, and yet the throughput is not, then you have three tricks up your sleeve:

  • Throw bigger hardware at it
  • Change multi-threading patterns to use the hardware better
  • Change the low-level algorithms or function calls

Recent shocks with exception handling have shown there’s a lot to achieve with this last one, but it should be a last resort, since it shouldn’t usually offer such significant performance benefits.

TL;DR

Don’t make slow software. And don’t pay people to tell you it’s too slow.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s