Following a discussion on testing and architecture I thought I’d write a post. The statement was: When architecture is not informed by tests, a mess ensues. That’s of course nonsense. Versaille was never tested but is still recognized for it’s architecture.
The statement got me started. Rhetorically that’s a clever statement. It uses an old trick most salesmen has under their skin. The statement associates the item being sold (in this case that item is testing) with something with objective positive value. That would be information in this case. The statement is of course also logically incorrect but marketing never worries about mathematical correctness as long as the statement is either mathematically incomplete or ambiguous. However the statement was not taken from a marketing campaign but a discussion of engineering practice between J “Cope” Coplien and Uncle Bob (the latter wrote it) and in that context it’s incorrect. The key is not the test but the information. How the information came to be is irrelevant to the value of the information. So a more correct version of the statement would be “Without information mess would ensue” and that is of course correct but also tautological.
The real value of information is in it’s use. If you don’t use the information but disregard it all together it had no value. Since testing is retroactive you get the information after you are done with your work, you can’t improve anything retroactively. Unless of course you are the secret inventor of the time machine in which case retroactively becomes entirely fussy.
If you do not intent to create another version, the information you gained from testing has no value in respect to the architecture. So using test as a means to produce value in the context of architecture requires an upfront commitment to produce at least one more version and take the cost of a potential re-implementation. If your are not making this commitment the value you gain from the information produced by your tests might be zero.
In short you have a cost to acquire some information, the value of which is potentially zero.
It’s time to revisit the statement that got it all stated and to try and formulate it somewhat more helpful than a tautology.
“If you do not invest in the acquisition of information your architecture will become messy”
You can try and asses the cost of acquiring information using different approaches and the choose the one that yields the most valuable information the cheapest.
There are a lot of tools to use to acquire information. One such tool is prototyping (or even pretotyping). Prototyping is the act of building something you know doesn’t work and then build another version that does. In other words prototyping is when you commit to implement a version, learn from it by (user) testing and then build a new version. Might that be the best approach? at some stage for some projects, sure. Always? of course not. Prototyping and pretotyping are good for figuring out what you want to build. So if you do not know what you (or your business) wants then use the appropriate tool. In innovative shops pretotyping might be the way to go. When you have figured out what to build, then you need to figure out how to build it. The act of figuring out how to solve a concrete task is call analysis. Analysis is good at producing information of how to do something in the best possible way.
Bottom line: There’s no silver bullet, you will always have to think and choose the right tool for the job
Fine print: You can’t improve anything with testing. You can improve the next version with what you learn by testing but only if there’s a next version