The Science of Information

My last couple of blog entries have -- once again -- been procrastinations from finishing off a difficult post. Most often, my difficulties in completing a piece come from uncertainties about the essence of what I am saying. Probably a good thing, as I am often trying to stretch my understanding to a new level.

In this case, since what I'm talking about is what I think can be, there is no foundation for me to fall back on. I can't check my memories, or facts, or even look to other people for help. I have a vague outline of what I really want to say, but it's not as entirely as certain as the words I used to describe it.

Because of that, I would suggest reading this one lightly, it may prove to be substantial, but then again it may not. Of course, comments are welcome, and any addition thoughts are always appreciated.


CHANGING TIMES

The 20th century tore a hole right through Newton's deterministic world. On impact, his simple discrete model of cause and effect in a linear existence seemed to shatter into a million pieces.

Our advancements were amazing. The special theory of relativity redefined how we saw geometry. We no long live in a flat space, instead it is curving and bending around us. Quantum physics shifted our foundations, basing them on probability rather than actual matter. Things are now so little that we can only guess where they are. Chaos theory showed us that relationships aren't necessarily linear, small things can have huge unexpected effects. This gave us reasonable explanations for phenomenon like weather, but left us unable to predict it. And fractals revealed the naivete of our calculus by opening it up to allow recursive, constantly morphing functions that are infinitely deep. Now we can relate the complexity of detail back to our mathematics, as more than just a simple subset of common shapes.

Our models of the world around us have taken huge leaps. In this, for all that we lost in realizing the world is not a simple deterministic place, we gained it back again in understanding complex behaviors like fluid motion, weather patterns or other previously unexplainable material events. The world around us may not be simple, but it is still possible to understand it.

In one of my earlier blog entries, "The Age of Clarity", I predicted that the we would pass through a new Renaissance driven by our increased understanding of the information around us. To some degree, I suspect that many people will assume that I am falling back into thinking that the world is simple and deterministic; that this new age will allow us to model it conclusively with sciences similar to classic physics.

From our vast increases in knowledge I am certainly aware of the dynamic nature of the world around us, but that does not stop me from envisioning a discipline that can fully identify and integrate these behaviors into our world. Just not deterministically. All things, as we know them, leave an imprint on the fabric of space and time around us. We are surround by vast amounts of information, we can record it and understand its structure, its inter-relationships, even if we cannot entirely project how it changes in the future. We can project some changes, particularly large ones, even if we cannot aligned to them over time. We know more than we realize, but we understand less than we know.


MONITORING CHANGE

At some point in our not-to-distant future, I would expect that we could easily be able to collect any relevant information for real-life things or events. With this information we would be able to process it in a way that would allow us to model the effect of at least large changes, with some extraordinary degree of precision. With this type of model, we will be able to understand how changes have positive and negative effects, we will also be able to determine whether or not things are getting better or worse.

More specifically, I mean that we should be able to monitor an organization like a government, measuring its important aspects. This we should be able to accomplish quickly, possibly spending only a couple of days setting up the process. From there we would be able to model the key variables, with a full and complete understanding that any 'non-monitored' variables are 'non-essential' and that their impact on the overall model is insignificant.

By virtue of our knowledge, we will know absolutely all of the variables that are significant for this type of model and all of the variables that are not significant. And we will be right, because what we are doing has rigour. We will able to understand and collect the significant pieces of information that relate back to our real world. With this type of model, we can then exploratively make changes and see their effects.

As the government considers new laws and regulations, we should be able to alter the model to get an estimate of the effect of the changes. As the changes are made, we should be able to get a quantitative feedback that proves that the changes are having a positive effect. In the end, we should know, conclusively that we have in fact improved the government; not a guess or a gut feel, but solid evidence that things are better to some degree.

Now, that type of statement is easy in a deterministic world, but in ours, modeling an organization for instance and introducing changes, then recalculating a new direction to any accurate degree is nearly impossible using today's understandings. We need to guess at what information to capture, and then guess at how changes influence it. We don't have enough of an understanding of either, but that won't always be true.

Our goal then is too be able to measure the correct things, model them completely and then project these models forward to some degree of previously unobtainable level of accuracy. All extremely hard tasks. With our current understandings they are nearly impossible and definitely not rigorously defined.

Our current abilities, when we computerize things is to spend years flailing away at the design of large systems. Often we guess about the structure of the data, and spend a great deal of our effort trying to slowly fix the serious problems caused by our poor initial guesses. Even when we have collected huge piles of data, we barely know how to go through and mine it for simple things. If we know what we are looking for it is easier, but still difficult. It takes years, often fails and is never particularly rigorous.

There are lots of available technologies, books, theories, etc. but none of them are fully objective. We kinda know how to do it, and it kinda works, but often fails.

Another aspect I expect for the future is that in assembling large piles of data, we will be able to decompose them into their elementary influences. Thus new and unexpected relationships will get discovered, just because of the amount of data we have collected, and the process of finding this relationships will be rudimentary. Once we understand the structure of data, we will also understand how it relates to other data in the world around us. Mining those relationships will be easy.

Our systems in the future will just present to us all of the causal relationships based on the increasing data pile size. We will no longer have to guess, and then go in search of the answers to back up our assumptions. Instead we will assemble the data, and the answers will be plainly visible (although it may still take us lots of time to accept them).


THE MISSING SCIENCE

To get there, we'll need at least one new science, if not many. For these goals we need a way to really understand information, including both its structure and its relationships.

One of our truly powerhouse accomplishments as an intellectual species, has been the creation and refinement of mathematics. In a very real sense we can see it as being a rigorous way of relating together numerical and symbolic qualities in an abstract container that are always universally correct. It's the absoluteness that is key.

We have a huge number of different 'branches' of mathematics, dealing with a wide range of axioms and theorems. Some are quite disparate from each other, but they all hang together because they are abstract, they are rigorous and they deal with numerical and symbolic entities.

If we stick our head directly into the abstract plane where mathematics hangs out and squint a bit, we can see other 'things' there as well.

We know 'information' and using information theory we know for instance that there is some undeniable minimum size for any piece of data. Claude Shannon proved that 'information' can only be compacted so far down, and no more. In that 'sense' information has a physical properly in our world, even if we can't see it. Information is numbers and symbols, but it is also so much more. Some branches of mathematics do 'indirectly' deal with the relationships of data, but their focus is on the numerical or on the symbolic.

We know we need a new science to apply rigour on top of our data, and we know that we want it to be an abstraction like mathematics, but dealing with information instead. I like the term 'informatics' as a name for this new science, but there is already an existing definition. The dictionary definition for informatics relates this term to Computer Science and to the study, storage, retrieval and classifying of information, however I would really like to dis-connect this term from any underlying hardware, software or even reality.

I'd prefer to recast this definition as the 'abstract' study of the structure and relationships of information, focusing on the shape and interrelationships between datum. Like numbers and symbols in mathematics, data may be markers for the real-world, but if we contain them within a rigorous abstract foundation, we can fully understand their nature prior to it becoming clouded by reality.


STRUCTURE AND RELATIONSHIPS

For everything we need a foundation; for this we want to study the abstract structure and relationships of data that mirror our real-world.

All information in our real world has a placeholder in any abstract one. We want to be able to conclusively say interesting things about the structure and relationships of the various placeholders. We want to be able to know how they are organized and see how they change over time.

Right now for instance, we may know some of the elements of how to model complex some real world entities, such as bank accounts or financial instruments. A lot of work over the last 50 years has gone into gradually producing more reliable models. Interesting, virtually none of it has been formal in anyway. We may know a lot about creating abstract models for financial instruments for example, but this is experimental propriety custom knowledge that is not shared.

The next step is to put structure and rigour to this knowledge. We don't just want a huge number of custom models, we want some unifying theory of how to reliably model the world around us. We need to dig into the foundations of information.

All information has a name. In fact most of it has many names. The names usually belong to complex taxonomies, and often have instance-related elements. For example, $128.02 is the price in CAD on June 5th, 2007 of a slice of a 1995 Canadian Govt Bond as traded in Toronto. BondPrice is a shorter name, and relative to things like market (Toronto) and date (June 5th).

Beyond that, a Bond is a type of financial instrument which is a type of contract, which is an agreement between two parties. So, we have both instance information and type information. For our purpose in this example we can mix the instance, type and categorical information, not distinguishing between the different varieties.

In an abstract sense, we can see the various breakdowns as being structural information about the underlying data. From some point, relative or absolute, we can construct a long-name, as a series of labels, that leads us through the structure and down to a specific datum.

The various parts of the long-name (price, date, instrument name, market) all form a pathway through the structure to describe the data. A collection of related pathways for similar data outlines the complete structure. An instance of price/yield information for a specific day, in a specific market is a data-structure. The static properties of the bond, its name, type, creation date, etc. all form a data-structure. These structures, plus many others form the structure of financial instruments. All data has some underlying structure that relates it various parts to each other.

While the pathways may be relative, or change with language, science or other categorical process, the underlying type structure of information never changes. Its structure is its structure. The idea is that underneath the same basic thing exists whether we use French or English to describe it. It exists whether or not we describe it relative to ourselves or to some fixed point like the universe. Given two different descriptions of data, if they are close to complete, then they are close to having the same structure and inter-relationships.

That is a huge idea, the notion that although our way of representing data at the high level changes, the underlying structure of data is immutable. Fixed. It does not change, it exists in the world with a specific static structure. The world changes around it, but it is the same in any language, with a relative or absolute point of reference.

Oddly, the static-nature of the information is there by definition. If the structure of the data is different, then the thing being described is different. They might be related, and similar but from a structural standpoint differences mean things are not the same. Not that that implies that instances cannot vary from each other, that will also always be the case. Dogs always have legs, usually four, and they come in a variety of colors and sizes. Variations are normal, changes in structure are not.

To utilize this, we need a way to objectively start with one datum, and use it to understand rest of the structure of the information we want to capture. One way, for now is to start assembling meta-models of all of the different data types. If we assemble models of financial instruments, social organizations, etc. We will be able to understand their existing structure. For every industry were we can model, we can be objective about the structure.

At some point however, understanding the structure should get beyond observation, particularly as we build up larger and larger abstractions for containing our world. We'd like a deterministic way of walking the information that by definition proved that the structure was actually correct. An algorithm for reliably mapping any point of data in a bigger picture.


MAPPINGS AND DIMENSIONS

All things are different to varying degrees. The strength in understanding any structure is being able to find isomorphisms and other mappings between the data at different levels. There are rather obvious mappings such as changes in language or terminology, and there are more exotic ones such as the similarity in structure between the events for different objects, or common patterns of data structures. Whatever type of mapping, it provides a way in which we can get a higher-level understanding of the underlying information. We always generalize to build up our knowledge, based on lots of individual bits.

For some types of information, the decomposition at some level is arbitrary. In that sense, it falls into some set of primitives at a specific level in the structure. As we seen, the underlying structure is immutable, then for some higher-level set of primitives where there are multiple ways of expressing them, there must be a mapping between the different sets.

An easier way of saying this, is that if we express something in terms of AND, OR and NOT primitives, at the same level, NAND will work as well. Anything expressible in the one set of primitives is also expressible in the other. Because they are the same, there is a mapping between them (although not necessarily one-to-one or onto).

If in a different language for example, there might be 50 words that describe types of snow. The mapping from English to the other language will have to include descriptive verbs as well as the noun 'snow'. Because the primitives decompose differently, that doesn't make the underlying thing different, it is just an alternative geometry for expressing it.

Different primitive sets will have different sizes, but still represent morphisms from one of set to another. We can show the expressiveness, and the coverage of each of the primitives and prove that the coverage of one set is identical to the coverage of another, thus the two are essentially the same. Thus information broken down into one set of primitives can be mapped to some other set of equivalent primitives.

Besides primitives, there are some interesting mappings that occur because all of the information in our real world exists in four dimensions. In our language we use nouns and verbs to describe the things in our space.

Nouns tend to be tied to objects sitting our our 3D space of geometry, while verbs, being actions are more often tied to our 4th time dimension. We also relate our 'business' problems in terms of nouns and verbs. We can by definition discussion all aspects of business using our verbal language. Putting structure directly to our communication decreases the likelihood of misinterpreting it.

Our language itself underneath is tied to the structure of our physical existence. That is an expected consequence because language is in a very real sense is just another space on which to bind real-world things onto abstract concepts such as words and sentences. Languages are loosely-structured, ambiguous, non-rigorous models of the world around us, but they are still models.

Similar mappings are important, but they are not the only class of mappings that have interest or value. Knowing as we do, that much of our world is 'fractal-based', we need some less rigid concept for mappings that are similar but not truly isomorphic, possibly a term like 'fractal-morphic', with a meaning of a twisted or skewed mapping. The scope of the mappings is complete, e.g. both sides are the same size, but the relationships are transformed with respect to one or more dimensions. A sense of degree would help redefine the concept, so we could say something was 2 degree fractal-morphic to something else, meaning that it was skewed in no more than 2 dimensions.

A corner-stone of any understanding of information will be creating rigorous definitions for these different types of similarities. If we could for instance boil the world down into a few dozen or so major meta-structures, then it becomes very easy to determine the properties of the data. What we know in general can help us to deal with the specific.


A SIMPLE HIERARCHY

Stepping back up to the higher-level, we have two abstract bases of theory. One specializes in numerical and symbolic qualities: mathematics, and the other specializes in structural and relationship attributes of information. As pieces, these two then fit into each other quite nicely.

With the various branches of mathematics we manipulate numbers or symbols in numerical ways. It is possible for one to believe that numbers form the base layer of all things, so that mathematics is and will always be the primary abstract field. However, where we can find mathematical expressions for the structure and relationships, we can also find structural and relationship expressions for the mathematics.

The structure and relationships of information is similar, but in many ways my expectation is that it supersedes mathematics. In a way, mathematics is the concrete 'relationships' between the abstract things in an abstract place, so the relative structure and abstract relationships should be a higher level of abstraction.

Irregardless of where it fits in, the two key properties for this new science of information are that it is abstract and that it is rigorous. Both are necessary to position it for maximum usability. The foundations need to be self-evident and universally true. It should not matter were you are, the basis is never changing. Even though we are dealing with 'real-world' information, the essence of what we are working with is totally abstract.

Some things will be tied to concrete entities, even thought the science itself is abstract. Some things will not. The parallel to mathematics can be seen in simple things like 'complex' numbers. They do not exist in our world, yet they can be the roots of equations that do have physical bindings. In that same way, there will be meta-data that comes to play, but has no real world equivalent. That arbitrariness may seem confusing, but it is not uncommon to have to leave the base dimensions of a problem to find or quickly compute a solution, some linear programming algorithms for example do that.


SOCIAL FACTORS

Some of what I image possible is the ability to better understand our social organizations. There are consequences to owning this knowledge.

The 20th century saw tremendous strides towards making manufacturing consistent. The idea that we can monitor, and control for quality the physical output of a lot of people, in an almost mechanised way made a huge difference in manufacturing. The modernization of factories was significant. It caused upheaval, but ultimately our societies integrated the changes.

The revolution was stunning, but it was really confined to the way we saw physical activities. One of the things we can't do, is apply those rules towards intellectual endeavours. We learned how to consistently build a model-T, but not how to consistently fill out a form or run a meeting.

Although using muscles is different than using the brain, the two share a relationship to each other. The biggest difference being that intellectual effort can't be visibly measured. This seemingly makes it an intangible quality like confidence or morale. Yet we know it is there and we know the output of it. We also know for instance that bad morale reduces physical and mental output, so it in itself is not really intangible. We just don't know how to measure it properly.

One of the things we should be able to do with our advanced understandings, is see these things based on their imprints from the world around us. In large organizations like governments, we should be able to determine how effective is the organization, and how many problems it has. We should be able to see the effects of 'intangible' qualities, and see how changing simple things improves or degenerates the circumstances. Intangible things may not be physical, but by definition they have some sort of an effect, or they are imaginary.

Some day we will be able to model these qualities. To get there we need to cross over the point where we understand how to mechanize intellectual effort because we need to relate it back some type of underlying model. This is the fear of all white collar workers, that we can measure their work. But until we can, we can't tell for instance how effect a social policy is. It is a horrible trade-off, but a necessary one.

It is the dream of the executroids, and the night-mare of the workers. But we must remember that factory-workers survived, so shall the white-collar workers and middle management.


SUMMARY

At some point, with each new discovery we shed light on a new and smaller, more refined set of details. Then over time as more details emerge we often coalesce our knowledge back into larger abstract theories. In a real sense we see the trees first, then the forest. We are forever in a cycle of slowly building up our knowledge.

It is amazing the things we know, but it is also amazing how we cannot truly verify them. Everyday we are bombarded by facts and figures based on bad science and dubious mathematics. Underneath all of this, is our use of computers to amplify the amount of data we are collecting and mis-interpreting.

We need help in quickly identifying bad information. You could spend time in a study for instance, to prove that the process was flawed, but it would be far easier just to discount it because it failed to use proper information gathering techniques. Since we don't have those specified, we have to go to the results on a line-by-line basis even thought we can easily suspect that the problem was the initial data gathering process.

Because of this, too much that should be objective, has been rendered subjective. We cannot be sure of its validity, and now often different groups are openly contradicting each other. We've fully fell into the age of mis-information, because we have easy ways of building up large piles of information but no easy ways of validating that those piles actually match reality in any shape or form.

It was computers that created this problem, so it will have to be computers that get us out of it in the end. Once we understand how to understand information, we can then go about recollecting it in ways that will really provide us with true underlying knowledge. With enough of an understanding, much of what is now very murky, will come clear.

We lack an understanding of our information, but now at least we know what it is that is missing. If our next step is to create a real science of information, one that is undeniably and universally true, then we can give ourselves the tools we need to really make sense of the world around us. Realistically it is not that complicated to get a Renaissance, if you can figure out that you are a pre-Renaissance culture. It is accepting where you are that is hard.