Sergey Barskiy, a colleague at Magenic, likes to say that there is no magic in software development. There’s only hard work.
Every few months, another software management process is promoted, a new tool is developed, or a new snowclone, "X-driven development", is coined to make software developers more productive, and in general they all promote themselves as the magic that will radically change the way we deliver software, and in general they don’t really pan out. Instead we just end up with different schools of software development. Physiology, metoposcopy, chiromancy, theurgia, goetia, necromancy, cabalie, geomancy, hydromancy, aeromancy, prymancy and suffumigations — does one method really provide a better way to deliver software than another? Or should we simply pick the techniques that work best for us and stick with them? What is ultimately disappointing, and this is at the heart of Sergey’s rule, is that once one immerses oneself in any of these techniques, one discovers, like a teenage goth working through Alaister Crowley’s Magick, Liber ABA, that things don’t ever go quite quite according to plan.
But does this mean there is no magic? Perhaps we are looking in the wrong place.
Last Friday, on June 6th, Microsoft released the beta 2 of Silverlight 2. Almost immediately, several prominent bloggers published entries not only about the release, but also full code samples demonstrating how to use the new release. Scott Guthrie, Jeff Wilcox, Kirupa Chinnathambi, Brad Abrams and Ashish Shetty all had immediate posts (Mr. Shetty’s was actually a day early), but this is to be expected as they are all Microsoft employees closely associated with Silverlight.
More impressively, non-Microsoft employees like Shawn Wildermuth, Peter McGrattan, Walt Ritscher and several others had immediate code to publish around this release. No amount of hard work can make that possible.
What is the occult, after all, but something hidden? Even for people who once believed in such things, metoposcopy, geomancy and chiromancy were simply techniques for dealing with the hidden world not commonly understood. Along with alchemy and astrology, cryptography was once considered one of the areas of expertise of a renaissance magus. Both Johannes Trithemius and Giambattista Della Porta wrote about it. What made cryptography go so well with other fields such as necromancy and hydromancy is that its secrets were possessed only by the few, and knowledge of it helped preserve one’s monopoly on secret knowledge.
Software development is full of secrets. Developers call what they do "coding", for no obvious reason other than that it is generally incomprehensible to anyone but a fellow initiate of a particular coding language. The code, in turn, is a set of instructions which must be translated into another code, assembly, the mystical language of all our virtual worlds, which is actually incomprehensible to nearly everyone.
Dame Francis Yates called this kind of magic "practical" magic. It is simply a way of getting things done. Whether one instructs a demon to sour one’s neighbor’s milk, or uses chemicals to acidify it, the effect is basically the same — all that differs is the particular technique one employs to accomplish one’s goal. One is clearly going to be more effective than the other, but the difference between the occult and the mundane surely does not turn on mere efficacy.
The other kind of magic is a "spiritual" magic, which is a different sort of secret. In a chapter entitled "113" in Umberto Eco’s Foucault’s Pendulum, Eco quotes from Ja?far al-?adiq, the sixth Imam:
"Our cause is a secret within a secret, a secret that only another secret can explain; it is a secret about a secret that is veiled by a secret."
Spiritual magic, in this case, is the way one gains influence by either having special access to secrets, or by appearing to have such access. One begins to be an initiate into its mysteries simply by recognizing that it exists. In his history of Secret Societies, Arkon Daraul describes an actual case of a young man becoming a Sufi:
"His first contact with a Sufi was when he was working as a part-time assistant in a restaurant. Here he noticed a man among the customers who always seemed ‘on top of every situation. His methods of discussion with the people who came into the place were so controlled, and his perception, especially of atmosphere, so profound, that I plucked up enough courage to ask him how one did it.’"
The initiate is then tested for suitability, and finally takes the oaths required of him to learn more of the Sufi way. Ahmad Yasawi, a thirteenth century Sufi, laid down rules for initiates, of which the seventh is possibly the most important.
"Utter silence of secrets is my oath; and I will show respect for those who are set up over me, without quibble. I am the friend of the friends of the Order and the Murshid who exemplifies it; the enemy of the enemies of the same."
Today this would perhaps be called an NDA. The rituals change over time, but the patterns are always recognizable.
The patterns of success are imprinted upon the human mind and its shape appears again and again throughout history. Secret societies exist in every field, whether we recognize them as such or not. I do not claim that Microsoft has such a structure, nor do I deny it. I only suggest that if there is any magic in software development, this is where you will find it.