Friday, May 22, 2009

Driven by Work

Recently I returned from spending several weeks backpacking in China. I stomped on the Great Wall, gawked at the Terracotta warriors and even managed to plunge lightly into some of the mysteries of Tibet. All great locations in a land filled with a long and complex history.

Traveling always renews my excitement for life and reawakens my sense of curiosity. I find that if I'm stuck too long without some type of grand adventure, I tend towards pessimism. I think getting caught in a rut just clouds my mood.

Every so often I need to break free of the constraints of modern living and lead a more nomadic existence. I need to get away from the routine, and react more dynamically to the world around me. I just need to break free of all those bits of life that just keep piling up around us. I need to get back into the real world, not just my little corner of it.


LANGUAGE AND OCCUPATION

There are a lot of theories, such as the Sapir-Whorf hypothesis, that suggest that languages influence how we think and see the world around us. For many modern languages, it may seem to some to be a stretch, but I seem to remember a reference to an ancient Chinese "water" language with only 400 written characters (although I couldn't find an Internet reference, so you'll have to trust my memory). With such a limited vocabulary it would certainly be difficult, if not impossible, to craft an effective rant on most subjects. The words just aren't available. If you don't have the words, you can't express things easily. Even if you could create long complex sentences; the lack of brevity starts to be become an impediment to the ideas.

That's probably why so many professions end up with their own huge technical dictionaries. Short precise terms that have large significant meaning. Words that encapsulate complex ideas. We always need to be able to communicate larger ideas with less bandwidth.

More obvious than language, one's occupation surely has an effect on how we perceive the world around us. You can't spend 40 to 60 hours a week engaged in something, year after year and not expect it to affect you in some way.

Lawyers probably argue more than most, doctors seem preoccupied with their health (although I know several that smoke and drink) and accountants tend to pinch pennies. If you keep looking at the same problems in the same ways, it's hard to prevent that from spilling over into other aspects of your life.

I'm sure everyone will have counter examples where they know of someone that breaks the mold -- there are always exceptions -- but I think it's still pretty likely that the way we live our lives directly affects how we see the world around us.

It certainly shows up heavily with programmers and other techies. You see it clearly in their ideas and interactions. The web is plagued with examples.


THREE CHANGES

Over the years I've noticed several changes in the way I see things. Some changes are purely a result of age or education, but there are definitely some that have come directly from how I am spending huge chunks of my time. Influences from a lifetime of pounding out software.

Three changes in specific, appear over and over again. I've become more detail-oriented, I tend to view the world more often in a black and white perspective, and I'm often more disappointed when things are not easily deterministic.

It makes sense. Each of these attributes helps significantly in developing software.

The key to getting any big project designed, coded and into production is to make sure all of the details are strictly and carefully attend to. No matter how you look at the big picture, the details always either act as time sinks or have to be carefully managed.

Also, as a software developer I always have to tightly constrain the world around me into a strict, yet limited black and white perspective. Dynamic shades of grey make for horribly inconsistent software.

And I've learned to stick to the things that I know are strictly deterministic in nature. If it is going to work, it needs to work every time, consistently. A program that works sometimes is all but useless.

Spending my days and often nights in pursuit of big functioning systems gradually takes a toll on the way I see the world. On the way I want to interact with it. For each new domain I enter, and each new product I create, I have to break everything down into relatively simple, straight-forward chunks.

While these changes have helped me build bigger and better things in my career, I've often mistakenly applied the same back to the world around me, to a negative effect.

Software straddles the fence between the purity of mathematics and the messiness of the real world. The real world doesn't work the same way that a computable one does. You can't debug a personal relationship for example, and some things such as weather are just too chaotic in nature to be predicable.

While often we can understand the underlying rules, that doesn't mean we understand the results. You can see how the stock market works for example, but still be unable to profit from it. Those attributes that help me create programs also set me at odds with the world around me. The harder I work at it, the more they seem to influence my perspective.

I think that's why traveling restores my sense of balance, and reminds me that not everything should be simple, consistent, rational or even clear. It's often a breath of fresh air, in an otherwise stuffy environment. A subtle reminder to not get too caught up in myself.

I do notice that besides myself, often others in the community are highly afflicted by a computationally-constrained perspective as well. One that probably makes their lives more difficult. There are a huge number of examples, but the web itself becomes a great historian for studying people who have become far too rigid.


THE DEVILS IN THE DETAILS

While the small details are very important in getting large complex projects to work, being too attentive to them can lead to missing the big picture. Given the choice, getting the broad strokes wrong is far worse than missing any of the details. In the later case, the distance to fix the problem is often far shorter.

Focusing too hard on too many small things leads to a tunnel vision. A perspective of a world were the importance of nearly trivial details are often thrown out of proportion.

It's not uncommon, for example, to see techies highly irritated by spelling mistakes and typos.

Language is the container for ideas, and while good spelling and grammar make reading easier, it is rare that they introduce real ambiguities into the text. As such, they are really minor annoyances, yet some people seem so put off by even the simplest of them.

A well-edited piece, quite truly, is easier to read, yet often it is symptomatic of the work being less raw and more contrived. A polished effort. You'd expect good writing in a magazine for example, but you always know that the polishing 'twists' the content. You are not getting the real thoughts of the author, but instead you're getting their packaged version.

Ultimately, what's important is the substance of what people say, not how they say it. Missing that, leads to dismissing possibly correct ideas for the wrong reasons. The truth is true no matter how badly it is packaged, while falsehoods tend towards better packaging to make them more appetizing.


BLACK AND WHITE

You can't just toss anything into a computer and expect it too work. Computers help users compile data, which can be used to automate work, but ultimately everything in a computer is a shadow of the real world. A symbolic projection into a mathematical space.

We can strive for a lot of complexity, but we can never match the real world in depth, nor would we want to. To make our systems work, we have to reduce a world full of ambiguity and grey into a simplified black and white version. If it's not easily explainable, then it's hardly usable.

This can be a huge problem.

Too often you see programmers or developers trying to pound the real world into overtly simplistic boxes of good and bad. Right or wrong. Left or right.

That type of poor deconstruction, driven by a Star Wars mentality, leads to many of the stupid flame wars that are so popular were two sides pound on each other assuming the other is wrong.

You know: my programming language is good, yours is bad. My OS is better than yours. My hardware is right, yours is inferior. That type of nonsense.

Clearly, all technologies have their good and bad points, but that quickly becomes ignored in a black and white world. Everything get reduced into two overly simplified categories, whether or not it makes any sense. A stiff rigid viewpoint fits well inside of a machine, but isn't even close to working with the world outside.

If you're oriented, for example, to see all of your fellow employees as only either good or bad, then because of that limited perspective you are missing out on a broad (and ultimately more successful) view of the world around you. People have such a wide range of strengths and weaknesses, that assigning them to one list or the other misses out on their potential. You'll end up relying on weak people at the wrong times, while passing up some well-suited resources.

Black and white works well for Hollywood or comic book plots, but it forces us to miss the depth of the world around us.


A LACK OF DETERMINISM

A programmer lays out a pre-determined series of steps for a computer to follow. Even with today's overly complex technologies, the computer still precisely follows these instructions. It does exactly what it is told to do. It does so in a deterministic and predictable manner. It is logical, and rational in it's behavior.

We get used to this behavior from the machines, so it's not uncommon that programmers start to expect this from other parts of the real world as well.

This becomes most obvious when you see technical people discussing business.

Management consultants lay out huge numbers of theories for business to follow, but hopefully they don't really believe that it works out that simply. Business, like weather, has some simple driving factors underneath, but at the practical level it is chaotic enough that it should be considered irrational.

If there existed some simple rules on how to succeed in business, eventually enough smart people would figure them out, to a significant enough degree that their exploiting the behavior would change the system. That is, if to many people know how to exceed the average, then they become the new average. If everybody wins, then nobody wins.

The markets are intrisically irrational (from a distant perspective) yet that doesn't stop techies from apply bad logic to predict or explain their behavior. It's epidemic, examples of people explaining why a superior technology will dominate the market, how being nice to the customers will increase business, how careful listening will get the requirements or any other assumption that presumes that the underlying behavior is deterministic, logical or predictable.

This works well for dealing with the internals of software systems, but for business and politics most programmers would do well to except that they are just irrational and unpredictable.


SUMMARY

If you know that you're biased, it is far easier to compensate for it. However, if you're walking around in an imaginary world, you tend to find that it's an uphill fight in the real one. You keep bumping into invisible walls.

The best way around this is to try very hard to separate these two worlds into two distinct perspectives. We don't have to unify our working perspective with the real one, but we do have to be aware when one is corrupting the other. We can keep them separate, and have separate expectations for each. They don't need to be combined.

When I travel, it reminds me of the greyness, the people and the irrationality of the world around me. Although I can often break things down into simple bits for the system development work, I always need to be reminded that that is just a mere subset of the reality surrounding me.

Computers should be predictable devices that are easily explainable, at least at a high usability level. The real world, outside of the machines, on the other hand, is an infinitely deep messy collection of exceptions to each and every rule. If you think you've figured it out, then you haven't gone deep enough yet.

It brings nothing but pain and frustration to expect that the real world will work on the same principles as a computational one, but still it's very common to see people caught in this mental trap. It certainly is something worth avoiding. Even if you have to occasionally trek around on the other side of the planet to do it.