Engineering is much more like plumbing than math.
As a programmer, being productive rarely involves finding new O(log(n)) algorithms. Adding new features and even optimizing for performance has much more to do with trying to figure out how to turn a vector of scoped_ptr<FooFactory>& into const ThreadableFooInstanceWrapper* tuples, regardless of whether the change actually affects the underlying algorithm in a nontrivial way.
My coworker pointed out that this tendency isn't limited to computer science. Modern digital devices come with slick programming interfaces, accessed over buses like I2C, SPI and CAN. All the datasheets have timing diagrams down to the individual bit levels for these standard, widely used interfaces, and at some point you'll get to hook up a logic analyzer to the lines and pull out the diagram to see why your commands don't seem to be affecting the chip. Oh, also, it needs 3 different power supply rails, and the following 6 very specific capacitors placed within 1cm of the power pins, before it'll do anything at all.
The semantic thing you're trying to get the chip to do -- turn off output B, enable automatic black level correction, or set the refresh rate -- is utterly straightforward. But hooking up the plumbing correctly so you can tell it that takes up a huge amount of the development time.
Mechanical engineering has that nature too: brackets, matching up bolt patterns between standard parts, sourcing a flange button head cap screw instead of the ordinary button head cap screw because the head is 2mm shorter and clears the bolts on the other leg of the L bracket.
When faced with a big project, an engineer's first impulse is to throw away what looks like a big mess of plumbing and start reinventing the core machine. That's almost always the wrong thing to do. Expect to spend at least half your time on plumbing by the time the system ships.
I suspect that engineering would benefit from a more explicit focus on plumbing skills. Yes, it's good if a programmer can break quicksort down into little pieces, but how many times have you had to implement quicksort? Okay, now how many times have you spent an hour trying to figure out how to get a new C++ library to work because the author didn't give any example code, and you didn't know which kinds of objects had to be initialized and in what order just to get the library initialized properly?
How much time have you spent pulling data from one source into a data structure so you can stuff it into a different data source in a slightly different format? Maybe CS departments need a Software Plumbing Research Group figuring out how to cut down on all the manual plumbing we have to do, and teaching classes on how to quickly rig up conduits between AJAX data sources and STL data structures used by C++ libraries.
And if you're on the designing end of a system, it suggests that before you write the detailed document for all the clever algorithms at the core of your system, you should write up some very straightforward recipes, with lots of ready-to-compile, complete examples (no pseudocode!) for how to build, use, and safely change what you've built. Make sure to copy and paste the examples into a clean environment and make sure it builds as advertised.
Stop assuming that algorithms are the important part and treating the plumbing as an afterthought, and you'll spend less time being frustrated at how hard it is to reach the semantics to make your tweaks. The plumbing's there; deal with it. And realize that your ability to hack the plumbing efficiently (or simplify it without crippling the system) is a big part of your overall productivity as an engineer.
4 comments:
So true.
Another salient fact about plumbing is that plumbers need to deal with a fair amount of sewage. This too is the case for engineering.
The best plumbing tool I've ever used is quasiliterals. In Caja we needed to do a bunch of rewriting of Javascript. We took a Javascript parser and added a new pattern to the identifier matcher, allowing identifiers that start with @ and optionally end with *. Now we can write patterns like
Replace
for (@key in @x) { @body*; }
with
for (@key in @x) { if (!@key.match(/___$/){ @body*; }}
[This is a hack that makes trusted code skip over Caja's private security variables in the absence of an ability to mark properties as DontEnum, which will be fixed in ECMAScript 5.]
Contrast this with the corresponding Rhino AST manipulation code:
// for (key in x) { body; }
// => for (key in x) { if (!key.match(/___$/)) { body; } }
if (n.getType() == Token.FOR && n.getChildCount() == 3) {
Node body = n.getLastChild();
n.removeChild(body);
Node key = n.getFirstChild().cloneTree();
Node match = new Node.StringNode(Token.STRING, "match");
Node keyDotMatch = new Node(Token.GETPROP, key, match);
Node cajaPrivate = new Node.StringNode(Token.STRING, "___$");
Node re = new Node(Token.REGEXP, cajaPrivate);
Node keyDotMatchRE = new Node(Token.CALL, keyDotMatch, re);
Node notKeyDotMatchRE = new Node(Token.NOT, keyDotMatchRE);
Node ifNotKeyDotMatchRE = new Node(Token.IF, notKeyDotMatchRE, body);
Node newBody = new Node(Token.BLOCK, ifNotKeyDotMatchRE);
n.addChildToBack(newBody);
The great part is that they're not templates: you're manipulating ASTs with quasiliterals, so there's no chance for injection attacks. When you're rendering the output, you know the context of the pattern and can do the proper escaping automatically, if necessary.
As far as comments are concerned, Updoc is a documentation tool that allows executable comments for the purposes of testing as well as for exposition:
http://www.erights.org/elang/tools/updoc.html
For sure plumbing increases the productivity of an engineer but without the math engineering miracles and branches are impossible. Interesting post !
Post a Comment