The tragedy of computer science.
by Vinay Gupta • April 16, 2008 • Personal • 2 Comments
Faster computers do not make writing programs easier, any more than printing presses made it easier to write poetry.
Vinay Gupta
I’ve been trying to find a way of saying that for a while. Now, of course, conversely
Computers which are too slow or too small for the problem at hand make programming harder because you must write code to solve the problem and make your computers act as if they are faster and bigger than they really are. These are two separate problems, and the second one is often very hard indeed.
What happened is that people saw faster computers make programming easier in the early days because it was simply dealing with the second problem – machines that were too slow and too small.
The fascinating problem is this: can you turn excess computing power into more code that solves the problem?
Answer? NO!
That’s AI. Expecting faster computers to make programs easier to write is AI-complete. There might be a few “hacks” which are equivalent to teaching computers to play chess. For example, Python is fairly widely acknowledged to make many programmers feel far, far more productive. And it is pretty slow. So a problem solved in Python might be solved a little more easily than solving it in assembler, but the improvement is perhaps log(n) of the difference in speed between the computer which can solve the problem in assembler, and the computer which can solve it in Python.
Maybe every 10x improvement in computing power doubles the speed with which programs can be written.
But the reason for that is that, at the end of the day, we’re sending a computer to do a person’s job. The conversion of computer cycles into intelligence is very, very inefficient in deeply non-linear ways as we know from fields like chess and, say, natural language processing in areas like machine translation.
What’s the answer? Frankly, almost no work has been done on programming as a human activity. If we trained programmers the same way we trained athletes – monitored performance, charted data across large groups, experimented with training regimes… if we had coaches who’s job it was to get the most out of teams – not managers but honest-to-god coaches…
A good programmer can make a company tens or hundreds of millions of dollars over a career. The investment of companies in optimizing the performance of their programmers seldom goes further than a decent office environment. We’ve been barking up the wrong tree for a long time, trying to make computers into better programmers, rather than focussing all that research into how to make programmers better at programming.
I think what you are articulating here is what Fred Brooks described as the difference between essential complexity and incidental complexity:
http://en.wikipedia.org/wiki/No_Silver_Bullet
That is, when all incidental complexity has been removed, then the solution is as simple as possible, but not simpler – to paraphrase Einstein.
Thus I would restate your self-referential law as follows:
Faster computers make programming easier by removing incidental complexity, in the same way as a word processor makes it easier to write poetry by removing incidental complexity. But there is no silver bullet for the essential complexity of the problem.
As you point out, faster computers equal less incidental complexity in programming. They allow us to use higher-level languages, thus graduating, say, from machine code to assembly code to imperative programming to functional programming. They also allow us some illusions orthogonal to the language used, in the form powerful abstractions – consider how garbage collection allows us to treat memory as effectively unlimited for a large number of mundane problems, greatly simplifying programming.
The comparison to athletes is flawed, however. Programming is fundamentally a creative effort. If you were to restate what you said in terms of, say, painting, it would no longer make sense. Yes, we could still state that instead of coaches, you need teachers and mentors. However, the idea of monitoring performance becomes absurd – what do you measure? The number of pencil strokes and the lines of code per day? This, of course, is the classic dilemma of software project managers – there’s no way to reliably determine how far along a project really is. This problem also found perfect articulation in Fred Brooks’s writings of many decades ago.
So here’s a question then…”if programming is fundamentally a creative effort, then why can’t you find a creative method to enhance both the technical skills and productivity of said programmers?” Isn’t redefining the reality of both the instruction and the environment both a mesh of structured techniques learned and the inherent creative ability of the student?