Bad Tests, Good Tests

Bad Tests, Good Tests

Tomek Kaczanowski

Bad Tests, Good Tests

Bad Tests, Good Tests

Tomek Kaczanowski

Copyright @ 2013 kaczanowscy.pl Tomasz Kaczanowski

All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written

permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.

Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book

is sold without warranty, either express or implied. Neither the author, nor the publisher, and its dealers and distributors will be held liable for any damages

caused or alleged to be caused directly or indirectly by this book.

The author has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals.

However, the author cannot guarantee the accuracy of this information.

Visit us on the web: http://practicalunittesting.com

Published by kaczanowscy.pl Tomasz Kaczanowski

Cover design by Agata Wajer-Gadecka, http://poleznaku.pl

ISBN: 978-83-938471-0-5

First printing, December 2013

Version pdf_a4_20131229_2323

i

Dedication

To all, who develop open-source projects.

May the Force be with you! :)

ii

Table of Contents

Acknowledgments ................................................................................................................................ iii

Foreword .............................................................................................................................................. iv

Preface .................................................................................................................................................. vi

About the Author ............................................................................................................................... viii

1. Why Bother? ..................................................................................................................................... 1

2. Breaking all the Rules ...................................................................................................................... 2

2.1. No Assertions ......................................................................................................................... 2

2.2. Autogeneration ....................................................................................................................... 2

2.3. It is a Full Time Job .............................................................................................................. 3

2.4. Conclusions ............................................................................................................................ 5

3. Strength ............................................................................................................................................. 6

3.1. Use Smart Values .................................................................................................................. 6

3.2. Self-Test ................................................................................................................................. 8

3.3. Happy Path ........................................................................................................................... 10

3.4. Expecting Exceptions Anywhere ......................................................................................... 14

3.5. Evolve or … ........................................................................................................................ 16

3.6. Assertions should be Merciless ........................................................................................... 18

3.7. Is Mockito Working Fine? .................................................................................................. 18

3.8. Expected Exceptions and Verification ................................................................................. 19

3.9. Mockito any() vs. isA() ....................................................................................................... 21

3.10. Be Generic! ........................................................................................................................ 22

3.11. Write the Right Tests ........................................................................................................ 25

4. Maintainability ................................................................................................................................ 27

4.1. Mock’em All! ...................................................................................................................... 27

4.2. Control the Environment ..................................................................................................... 31

4.3. Time Means Trouble (Always!) .......................................................................................... 34

4.4. Waste of Time ..................................................................................................................... 36

4.5. Overspecified Tests .............................................................................................................. 37

4.6. SRP for Tests ....................................................................................................................... 48

5. Readability ...................................................................................................................................... 54

5.1. Why formatting helps .......................................................................................................... 54

5.2. Ceremony ............................................................................................................................. 55

5.3. Creation of Objects .............................................................................................................. 56

5.4. Naming is King ................................................................................................................... 61

5.5. Mocks Are Good ................................................................................................................. 69

5.6. Assertions ............................................................................................................................. 70

6. Things to Remember ...................................................................................................................... 79

iii

Acknowledgments

This book would never have happened, had it not been for the help and encouragement of a number

of people.

Bartosz Ocytko, Martin Skurla and Bartek Zdanowski shared some examples of bad tests with me

(which doesn’t mean they wrote them in the first place!). My colleagues from the various teams I’ve

worked with also contributed in this way; however, their contributions may not always have been made

knowingly… ;)

Jakub Nabrdalik and Tomasz Borek delivered three complete sections that are now included within the

book.

Krzysztof Koziol and Marcin Michalak read early versions of this book and provided some valuable

feedback.

My special thanks go to Petri Kainulainen, who helped me a great deal by carrying out several exhaustive

reviews of early draft versions of the book.

Petri not only found many weak spots and suggested improvements, but also complimented me on the

things he liked and encouraged me to go on (which is something I really needed!). Thank you!

Peter Kofler - yes, the Code Cop himself! - agreed to write a foreword and… well, I couldn’t imagine

anything closer to the spirit of this book. Thank you!

Carl Humphries has done a splendid job by translating this book from my version of English into a

standard one. You would not enjoy reading this book at all, were it not for his hard work!

Even though many people have contributed to this book and have carried out reviews, all

bugs still belong to me!

iv

Foreword

I still remember when I came across the first bad tests. Early in my career I had joined a new team and

I discovered JUnit tests hiding throughout the main source tree. Although they were JUnit tests, they

were neither unit tests, nor could they be run standalone. The team’s architect had created a special test

runner that depended on the whole application being initialized inside the application server container.

After the architect had left, nobody ran these tests any more. The real problem was that the application

made heavy use of singletons internally, preventing any isolated execution of the business logic. The

bad design of the system prohibited us from writing proper unit tests. It was no surprise that there were

so few of them. Back then I was already fanatic about code quality which also meant fanatic about unit

tests, and I immediately attacked the problem. It took me several years to push back the singletons and

enable unit testing, so that the whole team would be able to add unit tests as they went along.

In the years since then some things have changed in our industry. Eventually the singleton pattern was

considered an anti-pattern and the rise of Test Driven Development put a new focus on unit tests. But the

problems with tests remain the same. I still encounter bad tests regularly. As we have more automated

tests than we used to have – which is without doubt a great thing – the problems of bad tests sum up and

have a higher impact as well. These tests slow down ongoing development and increase the maintenance

cost without adding much benefit. As today’s code bases grow in size and complexity each year, we

need to add and maintain more and more tests. Some projects are facing testing-hell already, where bad

tests actually grind whole development teams to a halt. To avoid this every software developer needs

to understand what defines a good test, to recognize a bad one and to know ways to transform bad tests

into proper ones. And this is exactly where Tomek and his book come in.

I met Tomek at GeeCON, a Polish Java conference. He delivered a talk about bad and good tests, where

he shared some of the advice given in this book. He had been aware of bad unit testing habits since

years like me, but unlike me had taken the time to collect and categorize them. I was excited. There was

finally someone who cared for the quality of automated tests and even talked about it. I just had to meet

this guy. Fortunately GeeCON is a community conference where you can still meet with speakers if you

want to, and so we met. After chatting a bit about tests and our industry in general I knew that we would

be friends. Tomek is genuinely concerned about code quality in general and bad tests in particular and

I wish there were more people as dedicated to quality in our industry.

Tomek is a developer and he talks code. This book shows roughly as much code as plain text. For each

category of bad tests, Tomek shows the original code and then an improved version of it. This is a

good mixture, usually the example of code gives more insights than long explanations of coding rules

or design guidelines. After all a picture is worth a thousand words. Of course the underlying principles

that bad test might violate are manifold and explaining them all in detail would exceed the scope of

this book. This is a practical book showing real world code. Some of the code examples are really ugly,

some contain intricate puzzles. The number of bad tests Tomek has collected over the years is really

depressing but Tomek’s humorous comments and the sheer amount of WTFs make this book a hilarious

read - hilarious and scary at the same time.

This is a small book, but it is full of great advice how to avoid bad tests. Every developer should read

it. The fact that it is short really helps. Go ahead, read it, you really need to know all about good and

bad tests. Please do not be the person that adds another testing hack like the architect in the beginning

of my career. After all I am sure you do not want your tests to appear in the next edition of this book

as bad example, do you?

Peter Kofler, Code Cop

Foreword

v

Austria, 2013

vi

Preface

So you want to write great tests? Ah, so do I! :) I have some experience in writing tests, and I would

like to share it with you.

Stay focused, stay sharp, distill what you read through your own experiences, and choose what seems

right and valuable for you.

The Idea

The idea behind this book is to present test code snippets and discuss ways of making them better.

All examples discussed in this book come from real code of real applications. I haven’t made them up.

They are real. From thousands of tests I have seen I have selected these which illustrate some typical

errors or imperfections which I frequently encounter while doing code reviews (or pondering over my

own code!).

Some of the code examples were obfuscated in order to "protect the innocents". ;) Many were

significantly truncated, so only their "essence" was left, and all the cluttering was removed. This makes

the examples easier to understand but at the same time it makes some of the discussed issues look not

so important. However they are important in real life scenarios, where you have to deal with numerous

much more complex tests (or the same issues were repeated in numerous tests).

SUT and Assertions

From time to time I use term SUT which denotes System Under Test, that is the thing being tested. The

SUT can vary in size. For unit tests it is usually a class, for integration tests it might be a class, a layer

or a module, and for end-to-end tests it is the whole system.

Throughout the book I often use AssertJ assertions1

(e.g. assertThat(…)) instead of those provided by

JUnit or TestNG (e.g. assertEquals(…)). After having written thousands of tests (of each kind), I’ve

learned that they make my tests more readable by allowing me to better express my intentions.

Icons

The following icons appear within the text:

A helpful tip.

An additional note.

Warning, danger ahead!

1

http://joel-costigliola.github.io/assertj/