I just finished listening to a discussion between Jim Coplien and Bob Martin:
http://www.infoq.com/interviews/coplien-martin-tdd
and there were some really interesting things in that conversation.
The most stunning was that Jim Coplien works with systems that are hundreds of millions of lines of code. That is a massive amount of code. Shouldn't a higher paradigm like Object-Oriented save one from such a difficult fate? How long does it take to assemble that much code? Why hasn't millions of it been refactored?
The next is that even though Jim was dead-on about his observations of having to start close to the underlying understanding of the actual domain in his example of writing software for a bank -- otherwise you'll spend 40 years reinventing a well-known wheel -- that center of the conversation just died. If you know how you need to structure your system, why wouldn't you go straight there with the minimum amount of work, especially if you have huge time constraints. Approaches such as CDD or TDD seem to be taking the long way around the block.
Jim's point about developers without an architecture painting themselves into a corner is true irregardless of any discussion on Object-Oriented or Agile issues. It is a long standing tradition in Computer Science for aimless meanderings to finish off a project badly, that hasn't changed in 40 years. It didn't work before OO, YAGNI, TDD or any other acronym and it still won't work after. If you don't start close to where you need to be, you probably won't ever get there unless you are really lucky.
Bob's description of TDD seems more like a 'game' where you are pitting two different representations of the same thing against itself. All the more so when you realize that Jim is correct about unit testing just being a game of 'sink the battleship' where you randomly fire specific tests over a large surface hoping to hit something, before running out of time and shipping the code. Testing is always about getting lucky. And doesn't context switching between tests and code tired one out more quickly? For speed, I've always liked to batch up the work as much as possible.
Bob's thesis that it was unprofessional to 'not' unit-test every line of code before a release was absolutely crazy. And not possible. Mass buckets of code get released by 'professionals' all of the time where the 'coverage' of the testing is probably way, way less than 30%. That's just a guess, but an easy one. TDD doesn't even stop that; given all of the areas were you can't use TDD (GUIs and databases?), huge amounts of untested code will get loose. Untested code will always get loose.
Even if you unit-tested each and every line to 110%, the combination all of the pieces will open up unexpected and undesirable results; which if you didn't have some overall architecture to tie it all together would be a particularly huge and likely fatal mess. You can't skip integration testing, it isn't safe. If you have to do a lot of it at the end, then wouldn't it be better to minimize the stuff in the middle or at the start?
Further, TDD is not a main-stream process right now, even if it is getting lots of coverage from the 'Agile Press'. Most code is still written using waterfall methods, for most industries. Agile itself isn't even main-stream, and TDD is only a tiny piece of it. Saying it is so, doesn't make it any truer, and it leaves a bad taste in the mouth.
The weirdest part for me is that both of these guys are arguing about how to make code more stable, or at least that's what I hope they are arguing about, otherwise what is the point? All this low-level testing is nice, but it doesn't invalidate the higher level full-system testing that 'must' be done to end-to-end test the various pathways through the code. When is low-level testing effective and when is it just doubling up the workload for no apparent reason?
So, I add a process to my house-building technique that makes sure each and every brick is super-accurate. Now they are all exactly the same size given some huge level of precision. That's nice, but does that really help me build a better house?
I have not listened to the discussion, but I scanned the transcript.
ReplyDeleteI think you're selling TDD short. It is a legitimate tool for tackling complexity and improving the day-to-day development experience. Perhaps you're seeing the 'T' and thinking its all about testing. It's not (which is why there is a group trying very hard to put the "T" word back in the bottle and rename it Behavior Driven Development).
Bob's views on coverage are extreme, but you have to take it in the context that he is fighting to raise the standards of the profession. The message that I hear is "delivering software without automated tests is unprofessional". I agree with the spirit of what he is saying, but not the words he uses - I like to think he uses those words because they are shocking, and provoke discussion.
Finally, I have a different analogy than the brick one: Every condom is tested before shipping. It is not tested to be perfect, but it is tested to ensure that it meets the its most important criteria (no leaks). To do otherwise would be irresponsible. TDD unit tests are part of that sort of coverage. Automated Acceptance tests are another.
Regarding whether TDD is mainstream, a study was just released showing the latest Agile adoption rates. Since TDD is a part of almost every Agile method, we can use the numbers as a basis.
ReplyDeletehttp://www.methodsandtools.com/dynpoll/oldpoll.php?Agile2
So in 2008, somewhere between 17% and 56% of projects use Agile. My own experience is that full TDD adoption is much lower, maybe 5%, but partial adoption much higher (I'd guess around 15%).
Hi Steve,
ReplyDeleteThanks for the comments. TDD is interesting, to be sure, but beyond using it for extremely complicated coding bits or to help newbie programs learn to structure their code, I just see it as extra work. It may help to focus some effort on getting less small errors, but if it is taking away from design and integration-testing it is not a reasonable use of resources for bread-and-butter code.
I think that promoting TDD as an all-around solution to be used by everybody all of the time is actually hurting professionalism. It is these types of extreme positions that have cause the rank-and-file programmers to tune out any these discussions; thus protecting the status quo.
Buggy code is a huge problem, but it causes are both sloppiness and huge code bases. If you are going to write a hundred million lines of code, how you write it is not significant (50% less bugs won't save you).
The condom in your analogy is the 'final' product, thus the test is the final one, not a unit-test. A better example would be testing it once before it has lubrication added, and then again after. In that case, the first test is a unit one, while the second is the integration. Do we really need two tests?
Paul.
Hi Steve,
ReplyDeleteI was hoping someone would ask me about what is mainstream :-)
The study is nice, but at only 512 the sample set is way too low, and the collection method was likely biased.
If we consider all programming, including mainframes, PDAs and embedded devices, Open Source, commercial, consulting and domain specific coding there is a huge amount of coding going on.
Somethings I can guess at:
- The bulk of the world's mission critical data is probably still stored on mainframes or AS400s.
- The most popular set of technologies based on development effort must be COBOL, COBOL-derviates and RPG.
- Less than 50% of all new code written is actually object-oriented. Possibly less than 25%.
- Most professional programmers have barely heard of Agile.
Certainly in some areas, such as the big Internet web applications the newer technologies and techniques are heavily used, but there are still massive groups of 'professional' programmers using well-established technologies like mainframes. I'd be interesting in know what percentage of the Open Source code is object-oriented, that would be quite telling as well.
It is easy to forget that all of the other technologies are still out there and still being actively used. There are even active APL programmers still working away.
Paul.
I concede your points on what is mainstream, but I will add:
ReplyDeleteIn many ways, the legacy COBOL & AS400 stuff does not matter. In terms of software development, it is an evolutionary dead end. Once companies figure out how to get rid of it, they will. And it seems they have, in that many are making their last big investments in that area, and it is focused on SOA - abstracting the problem away. Once that is done, the next logical step will be to replace the abstracted pieces as needed.
None of this affects the Agile or TDD discussion in any way. But it does increase the proportion of OO code, because lots of new SOA stuff that would historically have been in the mainframe space is now being written in Java.
In terms of open-source, my quick peek at sf.net says 51% of 117,709 projects are coded in an OO language (C#, C++, Java, ObjectPascal). You could probably filter in other ways to get a different answer though.
Hi Steve,
ReplyDeleteThanks for the sf.net stat, I guessed that it would be around there, but it is good to get it confirmed.
Right out of school, nearly twenty years ago the big thing was that we were going to replace all of that 'legacy' mainframe stuff with our nifty new technologies.
In a lot of ways, that is the source of my frustrations that drove me to writing about software development. In twenty years, so little has actually changed. Every five years we get a new set of techniques and technologies that ignore the lessons of the past. We are clearly stuck in a loop.
Paul.
Paul,
ReplyDeleteI have a more optimistic viewpoint on the state of software development. I've been at it for (a little) less than 20 years, but am of the opinion that we are in a renaissance period.
I am really excited about both Agile and Lean. They are not silver bullets, but they are really well-thought out approaches, and they are continuing to evolve.
Agile, specifically, talks about Values. This is an excellent starting point, because Values give rise to Principles, and those give rise to Practices.
Many people look at Agile methods and all they see is the practices, and they discount them because they are just practices, after all. What they miss is the underlying principles, and the fact that the principles are supporting real values.
For example - TDD is just a practice. BUT...it directly supports principles of cohesion, coupling, testability, and more. The above principles support the Value of Maintainability - the idea that software should be open to extensibility and bug fixes.
Another example - OO is a practice that enables the principles of high cohesion, low coupling. Again, it is just a practice. You cannot code anything in OO that you could not using a procedural language. But, as a practice it does improve our odds of supporting the Value of Maintainability.
Excellent! I think it's good with a healthy skepticism against something that's almost become a religious matter. The lack of criticism against TDD is somewhat disturbing, and that's why I find your post refreshing.
ReplyDeleteI myself use TDD whenever I can. I have felt its strength and personally don't want to go back, so to speak, to my old way of programming. But, I am not a fanatic. I try to keep a pragmatic attitude towards it. If it doesn't feel right, I drop it. I don't demand people around me to use TDD, although I recommend them to.
The danger, though, lies in an extreme over-usage. How to develop must never be more important than what. So, forget about code coverage as a leading metric.
Hi Hans-Eric,
ReplyDeleteYes, we seem as an industry to want to go to extremes all of the time. I've always figured it's because if you spend all day playing with binary or black & white things it skews your perspective on life to being only black & white. In a world of many shades of grey, becoming too rigid can lead to problems.
A great deal of what I am coding these days is similar to things I have written in the past. The technologies change, the goals do not.
While I am always trying to push the technologies to the next level, after years of coding it becomes hard to find truly novel things for me to build. For the most part, I am looking for the fastest way to get my 'designs' into action, without losing too much in translation. Getting a team to faithfully implement one of my designs (as I would have on my own) is the big challenge.
We definitely need to explore more development techniques like TDD, but we have to be careful that the hype doesn't outshine the reality by too much.
Paul.
About "development" effort, I'd say this measure is intrinsically dangerously biased. If anyone finds a Better Way(tm), it will require less effort, thus hurting it's "popularity".
ReplyDeleteSimply put, if a given project costs 50 man-years in C++, and a similar competing project costs only 5 in Haskell, that's makes Haskell 10 times better, and 10 times less popular. Oops. Real (and less exagerated) example: Window managers.
We tend to correlate "popularity" with "good". It mostly work, but in this case it doesn't.