Summary of Is TDD Dead Part 3 of 6: Tradeoffs in testing

Kent introduces part 3 explaining this video is going to be looking at what the tradeoffs are.

Almost no questions in computer science can be answered absolutely, with the answer to most questions “It depends on…”

There are a number of important questions around TDD such as

  • How frequently do we need feedback about our code?
  • He explains there are differences of opinion on how quickly we need feedback on whether our tests are passing, with some people demanding sub second times and others saying several minutes is okay.

  • Fidelity: Quality of the feedback
  • Different personalities require different levels of fidelity

  • Overhead: Cost of increasing fidelity
  • Lifespan: how long is the software going to live? Probabilities of lifetime
  • Kent asks if he’s missed anything

    Martin says there’s more than one thing that you’re looking for feedback on.

    Is it doing what the user wants? – Acceptance tests
    Have I broken anything? – Regression tests
    Is my code-base healthy? – More tricky to test

    Green bar / red bar functionality is more important to Martin for Regression tests.

    David argues that many developers are too young to know the days of large Quality Assurance teams, and that the fall in the numbers of Q.A. roles is largely due to the popularity of Test Driven Development and other automated testing. He says the pendulum has swung too far in the direction of developers testing their own code and believes that lots of green tests make developers overconfident.

    David complains that the rhetoric from the TDD community has promoted faster and faster unit tests at a cost that isn’t well understood by may developers. He talks about the uptime at Basecamp, going from 99% to 99.999% was exponentially more expensive. He says the criticallity of your code affects the approach you should take. If you’re making pace-maker software or NASA space launch module the emphasis needs to be on higher quality and more exhaustive testing that for other software. The 100% code coverage mentality ignores the actual costs involved.

    Kent says he has a poster that says “Nothing at Facebook is somebody else’s problem” which he likes because taking responsibility is very important as a developer.

    Martin says Thoughtworks use Quality Assurance testers. He thinks relations have massively improved between developers and Q.A. staff over the years.

    David stresses the need to be aware of the existence of trade-offs and the value of exploratory testing. He thinks many developers become naive because of all there passing tests when real users will use their software in many ways that the developers never considered. He says some developers even think 100% code coverage will mean they will be no bugs in their code.

    Kent says customer feedback is really important. Unit testing feedback is not a substitute for real customer feedback.

    “As soon as you’re not willing to accept that you’re make mistakes anymore, then you’re making mistakes”

    Additional Reading

    The Costs and Benefits of a Unit Testing Culture

    Leave a Reply

    Fill in your details below or click an icon to log in: Logo

    You are commenting using your account. Log Out /  Change )

    Facebook photo

    You are commenting using your Facebook account. Log Out /  Change )

    Connecting to %s