Intro  Learning  Patterns  Real Life  Genetics  Classifiers  Biology  Neural Nets  Connectionism  Life  AI

Essays on Complex Systems Part 1 - Learning

Benefits of abstracting the requirements for learning

The next section will be an attempt to state the requirements for learning in a way that can apply to any adaptive system. If that is possible, it should have practical benefits such allowing us to:

1. Identify the different ways in which each aspect of the process is accomplished in different adaptive systems.
2. Discover how the elements of one adaptive system map to those of another.
3. Compare the relative efficiency or other characteristics among different adaptive systems.
4. Create and compare, in artificial adaptive systems, methods of implementing the individual parts of the process.
5. In situations where learning does not occur, identifying what abstract step of the process is not occurring, possibly without (or before) having to understand completely the underlying specific process (brain chemistry, or whatever). (Examples: 1. learning disabilities, 2. dysfunctional families where environment is so chaotic or random that social learning can't happen.)
6. Develop models for intentionally disrupting adaptive systems (usually human organizations, but also consider cancer, etc.).

The Requirements for Learning

[While working on this, I was struck by how closely these relate to ideas in Robert Pirsig's book, Lila, about how societies have an "exploration mode, with a ratcheting lock-in mechanism for preserving gains", a concise description of the requirements for learning.]

1. Information, a data set. Learning is change in the data set.
2. Memory, an information storage medium.
3. Environment in which the entity exists.
4. Feedback, interaction between the entity and its environment, with implicit or explicit definitions of reward and/or punishment.
5. Adjustment, a mechanism that alters the information, based on the feedback.
6. Innovation, a mechanism that randomly alters the information.

1) Information

Information is a data set. The data set is a pattern. It can persist through time by two methods:

1. By continued existence of a particular instance (an individual's survival).
   Examples:
   • Bristlecone pine DNA which survives via long-lived individuals.
   • A rock, which although not alive is a data set, a pattern.
      
2. By creating a copy of itself (propagation). 
   Example:
   • Bacteria DNA which persists by rapid replication to compensate for the short lives of its carriers. It's interesting that bacteria DNA therefore has greater opportunity to mutate and evolve (learn) than bristlecone pine DNA. Generally speaking, rapid replication should produce the fastest learning and most responsive adaptation.

Learning is the result of a change in the data set. Such a change can be beneficial, detrimental, or neutral with regard to the continued existence of the set. Usually only the beneficial changes are described as learning.

The smaller (in information content) the data units, the more adaptable the entity's behavior will be, but the longer the time required for learning will be.

In biological organisms, the organism is actually a creation of the DNA data set. When the program encoded in the data set is run, it creates a "body" for itself to live in, which thereafter protects and propagates the data set.

There is no human value judgment associated with continued existence or with the lack of it. It doesn't "matter" whether a data set does or doesn't continue to exist. In fact, considering that every pattern everywhere is a data set, there are many more "random" ones than "adaptive" ones. It's just that the random ones don't form populations, so we don't notice them. Something that's there one moment and gone the next isn't remarkable -- or even if it is remarkable, it just plain isn't there once it isn't there. Something that sticks around is there to be noticed, and is there to mutate, evolve, form populations, etc.

Consider the universe as nothing but energy that "does stuff" according to rules, procedures, patterns which lie along a continuum of adaptiveness (merely whether a pattern continues to exist). At one end is total randomness, patterns which disintegrate as rapidly as they come into existence (the pattern of molecules in a cloud). Going towards the other end, you encounter patterns that have characteristics cited above as requirements for adaptation, such as stability (memory), which contribute to the pattern not disintegrating: continued existence.

Relevance of information theory

(Disjointed speculations and questions that information theory probably covers well. Don't spend a lot of time on it. This was inspired by the problems of good cell programs getting torn apart in Adapt.cpp.)

In any detailed description, some things are more critical than others. Some things are essential, and you can't change them much at all.

Just as organisms can be hardy or fragile, so can an algorithm (?) or its coding (which also highlights the fact that organisms are organic patterns with some level of "fitness" relative to their environment, but which can contain within them other patterns existing in entirely different environments and that themselves have some level of "fitness" of their own.)

The genetic reproduction method is a harsh environment. Was "mybest" (a program for the runtime behavior of the "imaginary cells" in adapt.cpp that I manually coded to be an "excellent" strategy for ensuring that a cell ate lots and reproduced lots and thus should thrive in the population; in practice, it completely bombed!) an effective but fragile program that required perfect reproduction to survive at all, and therefore was coded unsuitably for the genetic environment?

The information encoding method can contribute to the information's longevity

What features help information to survive well in a genetic environment?

1. Redundancy. Examples: in biology, among cells, if one cell develops a defect, there are still many surrounding cells that are ok. In DNA there are duplicates of critical sequences so that if any are corrupted, others survive intact.
2. Simplicity: A short crude program that barely does the job can be more effective than a long complicated one that does it elegantly but is hard to reproduce without errors.
3. Small encoding units, much smaller than the actual functional units they encode, such that there's lots of genetic material, and mutations are therefore small in relation to the whole.
4. Error correction (?).

2) Memory

Memory is the storage medium for the data set. 

The medium must be stable enough to provide reliable storage, but not so stable that it can't be changed. That is, stable but not too stable, and flexible but not so flexible that it spontaneously loses what it's storing.

Carbon is an excellent storage medium. So are semiconductors. Water vapor and granite are very poor, for opposite reasons. It might be an interesting exercise to expand the list of good ones.

Any other intelligent life in the universe would have to be based on a good one. They would probably be similar to us chemically. But other weirder ideas are possible, and provide some interesting science fiction premises. For example, a solid intelligence seems almost impossible, unless something is allowed to flow, as electrons do in semiconductors, so it's not totally impossible.

Given that carbon compounds are so good at storing information in a "stable, but not too stable" form, and that some DNA processes seem so well understood (specifically, for example, the way enzymes for producing lactase proceed along the chain doing their processing, but can only begin at certain points, and the way that other molecules can block those starting points, thus effectively turning off that gene. The blocking molecules get stripped off by the presence of sugar molecules, which is the condition where you need lactase production.),

I wonder if anyone is working on chemical information storage as the next generation beyond electronic microprocessors. That is, to use DNA-like processes for information processing for its own sake, unrelated to genetic engineering, without any goal of creating or modifying anything that is actually alive. I think I've seen shows that refer to "chemical machines" in development, which might be related. Also somewhat related, people definitely are doing molecular engineering, determining what structure a molecule needs to do a particular job, and creating it to fit (a company working on protease inhibitors for anti-viral applications, recent development of single-molecule transistors).

Remember that the net effect of DNA processes is the production of proteins, and thus the net effect of mutations is the production of altered proteins.

• Point mutations are most common, and the source of variation.
• Frame shifts tend to be disastrous because the proteins coded for after the shift are probably completely different from those intended. (Similar to the step insertion/deletion problem encountered in Adapt. But in Adapt, it was acceptable because each step is at least able to stand alone, unlike the codon triplets which have no meaning without their neighbors.)

Much of human memory has a multi-media sense basis: you remember how something felt.

A previous incarnation of this section said, "Some storage media are objectively better than others." That is not correct. It is environment-dependent. In fact, in biological evolution, although at the molecular level DNA is the storage medium, at the organism level the storage medium is the organisms themselves, and biological evolution is the history of the various storage media. A species going extinct means it wasn't a good storage medium, container, for the DNA within it. At a different level, DNA and RNA compete with each other over which is the better medium.

3) Environment

The environment in which the entity exists.

4) Feedback

Feedback is provided by the interaction between the entity and its environment, with implicit or explicit definitions of reward and/or punishment.

There is an implicit value judgment in the one feedback loop that is inherent in nature:

Since the data set can only exist (and optionally propagate) as long as it continues to exist, the universal hard currency IS continued existence. Interactions between a data set and its environment will either prove 1) beneficial, 2) detrimental, or 3) irrelevant to the continued existence of the data set. (An individual piece of information in the data set will likewise prove beneficial, detrimental, or irrelevant to the interactions between the data set and its environment.)

The "implicit" value judgment (continued existence) is a harsh binary one, though: on/off. When a data set dies, it's dead. A data set can develop its own internal value system (an explicit set of value judgments). If these are accurate, and especially if they're more graded, they can avoid death by steering in the direction of more positive outcomes and away from more negative ones. That is, they can "score" situations or intermediate outcomes before they actually result in death or not.

Interesting problem is determining WHICH behavior the feedback relates to. Usually the most recent - immediate feedback most effective. But we often make incorrect connections, such as developing an aversion to something that was only coincidentally connected with some bad experience.

It doesn't really matter, though; it's ok to apply the reinforcement to any recent behavior, because if a behavior gets a "bad grade" unfairly, it will eventually be diluted by other different experiences. If a behavior gets its bad grade because of frequent proximity to the "real" culprit, that's appropriate and useful.

In an artificial system, could assign credit or blame to the entire set of recent behaviors, with an accelerated decay rate; e.g. 100 "blame" units apportioned among 4 sequential behaviors might be 70, 15, 10, 5. (Also look at "bucket brigade" method, but the problem there might be not knowing in advance whether reinforcement is + or -.)

Applicable to any learning system: success or failure can depend on the scale viewed. If you make a bad move in chess, you still successfully extended your arm, grasped and moved the piece. The blame for the bad move doesn't get assigned to those actions, even though they were the most recent and available for blame.

A TV show noted that although we generally consider that physiology generates behavior, the behavior produces results that in turn affect the physiology. (Either directly -- muscles growing in response to use, or indirectly -- the more usual examples of environmental feedback.) This is kind of obvious, but emphasizes the importance of the feedback loop.

(See Complexity: Waiting for Carnot. I think the talk about a new "law" describing a synthetic or "building-up" force comparable to or in opposition to the 2nd law of thermodynamics misses the point. The new law is natural selection, and already exists, very well described: some things persist, and some things don't. Things that resist destruction survive. Things that help each other resist destruction survive together as a unit or building block. As noted elsewhere, over time, larger building blocks are possible. And that's all you need for the development of increased organization, without the necessity for any constructive or synthetic "force".)

5) Adjustment

Adjustment is a mechanism that alters the information, based on the feedback.

6) Innovation

Innovation is a mechanism that randomly alters the information stored in memory, not based on feedback. Without such a mechanism, there is no way for the data set to be produced in the first place, nor for anything new to be produced upon which feedback can be received.

Implications

I think it's intuitively clear, and demonstrated by Adapt.cpp, that in any environment that contains the above requirements for learning, learning will inevitably occur. That is, if there are environmental rules that can produce differences in how likely data sets are to persist, and if the data sets can vary or be modified in ways that are relevant to, or affected by, the environmental rules, then the data sets will inevitably evolve. The data sets present at any given time will be those with the characteristics that produced the best chances of survival up to that time. If the environment changes, there's a time lag before the data sets can change, so they are always slightly out of date.

You can think of language as that which caused human culture to become a higher level adaptive system. An individual human has a memory, a method of storing information in a stable form where it can be accessed and changed. Spoken language gave groups of humans a common memory, somewhat lacking in stability. Written language provides a method of storing information in a very stable form where it can persist through generations, and where it can be accessed and changed. It isn't the memory of any individual. It's the memory of a new entity, the culture.

It is necessary to avoid mixing levels of description

The independent agents in many adaptive systems are adaptive systems themselves, but when analyzing the learning characteristics of one level of the overall system, the characteristics of the levels above or below should not be mixed in.

For example, a corporation is an adaptive system whose component agents are people who, in turn, are also adaptive systems.

An analysis of the dynamics within the corporation might be temped to say that the humans are "adaptive agents" in competition with each other within the corporation who "try" to evolve in accordance with the behavioral rules at work in the company.

That would be wrong. At the level of analysis of the corporate system, the humans do not adapt or evolve. As agents, they are component parts of the system. Each person has a "genetic behavioral makeup", a range of behaviors they are capable of. The fact that each person may try different behaviors in an attempt to do better within the corporation is irrelevant when analyzing the corporate system.

Each person's range of capabilities, whatever may be the many different strategies they try, constitutes their complete data set.

The factor that determines whether they are effective or not is whether they survive within the corporation: avoid getting fired. At this level of description, it is not important that an individual person may have a wide variety of strategies in his or her arsenal of behaviors. 

Thus, the corporation operates just like the natural environment, with survival consisting of continued existence within the environment. 

That the people are complex adaptive systems in their own right and that they are conscious and that their behavior is intentional are indeed interesting, but mixing that level of description into an analysis of their place as agents in the corporation is irrelevant and runs the risk of confusing the analysis.

First, it makes it look more complicated than it really is, and second, it can inspire a human analyst toward fuzzy thinking. For example, it encourages concepts like "adaptive agents", a phrase that in itself inherently mixes levels and should not be used.

As soon as you attribute (or find) intentionality, it's a signal that you've jumped to a level outside the one under consideration. Surprisingly, intentionality actually isn't very important. If you remove intentionality and replace it with random behavioral changes based on dice throws, it doesn't change the analysis. 

At any given level of analysis, the agents do not adapt. The collective behavior of the population of agents adapts. 

Notes and misc

• Make list of various learning techniques:

1. imitation of an example (see Tictac.cpp. Simply requires a stored, scored, description of the example, and a mechanism that imitates stored examples if the score warrants it.)
2. imitation of a similar example (learning by analogy),
3. memorization,
4. trial and error, etc.

• Remember how children learn: simplest, broadest categories first (even learning the simplest, most general letter shapes, animal pictures, etc. first), then refinements are constantly made thereafter.
• A general purpose learning program (or module) must be able to learn how to learn. It would solve the problem of a human having to decide how input is to be coded, how it's stored, etc., which is a big task even for trivial problems because they must be determined for each and every problem you expect the program to encounter.
• Inferring cause-effect relationships from statistical coincidence is probably important. "I did this, then that happened." And if that sequence happened 5 times, then you can be more sure.

Exercises

• Design an adaptive system. Examine each of the Requirements for Learning and choose a method for each aspect that is based on the examination above, not on what I already know how to do or what is traditional. Then tie them together into a learning entity. Note that each step probably has an entire field of inquiry associated with it. The information and storage aspects pull in the whole field of information coding, storage, and retrieval.

Some definitions

What is an Adaptive Agent?

Better if not used at all: an adaptive system (brain, species, ant colony, neural net, classifier system) is composed of agents (neuron, individual, ant, neural node, classifier rule). Each agent does not adapt. It is the system that adapts, usually as the result of some of the agents in its population surviving, and some not.

What is Organization?

Where the activities of multiple components interact, affect each other, or combine to produce coordinated activity that affects all of the components of the system and that thereby creates an actual distinction between the organized system (self, involved in the coordinated behavior) and its environment (other, not involved). (Since no system is truly isolated, the test must be one of degree or relevance.)

A system is organized if it exhibits emergent properties.

Question: What is the measure of the information content of a system? Randomness is hardest to fully describe, and thus has maximum complexity? Does organization of a system reduce its complexity by making it easier to describe via summarization using new levels of description, or does it increase it because the new levels exhibit new behavior that requires description, or does it have no consistent effect? (!)

What is Symbiosis?

A relationship between organisms. Watch throughout file for misuses. There are 3 kinds:

1. Mutualism: Both benefit
2. Commensalism: One benefits, the other is neither benefited nor harmed
3. Parasitism: One benefits, harming the other.

Self-Organizing Behavior

Self-Organized Criticality

Where a system naturally evolves to the brink of an unstable state.

Requirements

1. Input source (of something: energy, matter), source of change.
2. Constraints that define the critical state.

Seems like the relationship between input and constraints should be able to produce either steady-state, chaos, or complexity.

Steady-state and chaos are end states. Complexity isn't an end state or equilibrium state; it's a description of the processes that a system goes through on the way to equilibrium. Since transients of any length are possible, the time required to reach an equilibrium can be infinite.

Examples

1. A sandpile "organizing" to its critical angle.

Criticism

The concept of self-organization is animism or anthropomorphism. It tends to attribute intentionality or motive where there is none. A sandpile doesn't "organize itself" into a critical state any more than a dropped pencil "organizes itself" onto the floor beneath it or a satellite "organizes itself" into orbit. 

Nonetheless, tentatively adopting a new way of looking at something can inspire interesting or useful insights even if the premise itself is questionable. 

Positive Feedback (increasing returns)

Reinforcing, "them that has, gets", possession of some thing or characteristic makes it easier to acquire more of it or of something else. Amplification. Sensitivity to initial conditions can apply here. A trivial incident that sets the initial course can be magnified through positive feedback until there is complete "lock-in" to that course.

1. VHS, QWERTY: Any technology that is widespread, and has multiple dependent markets or technologies, such that the cost of changing to a different standard is greater than the benefits, even if the new standard would have been a better one to start with.
   
   Thus, if you are the innovator of a new product, it is to your benefit to create for the product a large web of dependencies (such as PC specific software for the IBM PC in the 1980's).
   
   For an old product, it is important for it to fit into any existing web that exists for that type of product (PC clone must run PC specific software).
   
   An old established web probably won't get displaced until the benefits of switching to a new standard clearly outweigh the costs.

2. Celebrity popularity produces sensational news articles, which increases celebrity status, etc.
3. TV violence attracts viewers, encourages more.
4. "Media feeding frenzy": media pick up on anything, draw people's attention to it for a while, which makes it more newsworthy, which generates more stories. The pinnacle of success for such a story is when it spawns meta-stories, not about the story itself, but about the coverage of it.
5. Actual shark feeding frenzy: one kill generates blood that stimulates other sharks to feed.
6. White Earth scenario: Heavy snowcover reflects sunlight, produces cold and more snow.
7. Sand dune growth: a small irregularity in terrain slows down and traps sand particles, which produces a larger irregularity that traps yet more particles, increasing the rate of growth of the dune.
8. Gravitational accretion: added mass increases gravitational field, which attracts more mass.
9. Alienation/ostracism cycle. It doesn't take a lot to make someone feel like an outsider, and once that's started, it can cause them to develop attitudes or take actions that amplify the polarization, which in turn makes the original ostracizers feel even more justified in their original opinion.

Negative Feedback (decreasing returns)

Damping. Small variations die out. Pressures exist to attenuate excursions from an equilibrium state. Possession of some thing or characteristic makes it harder to acquire more of it or of something else.

1. Pendulum. Spring.
2. Acceleration against friction.
3. Large financial trader in an illiquid market: purchases drive price up, making further purchases more expensive; sales drive price down, making further sales less profitable.

Interactions of positive and negative feedback

Multiple interacting feedback loops will create complexity. The nonlinearity is important.

In economics, increasing returns and decreasing returns would seem to be different stages of the same process. Economics examples often begin, "since sales tend to increase exponentially...", but they don't do that forever. Diminishing return isn't a factor until the market becomes saturated.

What is an emergent property or emergent phenomenon?

A property exhibited by a combination of things that none of the components has independently -- synergy.

Is degree of synergy measurable for a complex system? The degree to which it has properties that its individual components could not possibly have. The degree to which it is really a system rather than just a collection. Useful?

Low-level rules of interaction produce higher level patterns.

Must distinguish between emergent behavior and merely collective behavior. Presumably, behavior that is intentional on the part of the agents isn't emergent (e.g. the vote of a club to take some action). Not clear how you know when a phenomenon is emergent. A general rule is "if you couldn't have predicted it from the basic rules", but that may be more of a comment on your own predictive powers than on the phenomenon itself. I.e. anything we don't understand is mysterious.

-----

A liver is a number of cells that happen to be close together. There aren't any rules that define what a liver should do, but the cells function together in such a way as to produce overall liver behavior.

This is self-organizing behavior of symbiotic agents whose behaviors interact to allow them to survive even though the individual behavior of each agent would not be sufficient to allow its survival.

Further, the liver cells acting collectively isn't by itself sufficient. There must be heart cells producing "heart" behavior somewhere else, etc.

So the entire process is piles upon piles of cellular rules interacting to produce an enormously complex system, without there being any master rule set for the system as a whole. It's just that when each part is functioning, the whole system works, and that's what gives it the ability to persist. All you have to do is code for the properly functioning parts. In proximity with each other, they'll work.

Autocatalytic sets

Chemical and otherwise = Symbiosis (mutualism type).

Biological evolution is only an example of the more general process

Extinction is what happens when a change is fatal to the data set.

The only alternative to extinction in a changing data set is evolution, which describes the cumulative changes in a data set during the time when it doesn't go extinct.

These phenomena of change are properties of any data set, of any structure or composition. They simply do happen, and the examples are clear and are everywhere in people's daily lives.

Anyone is familiar with the progressive, cumulative (evolutionary) changes that occur over time in clubs, corporations, governments, countries, societies , cultures, economies, languages, families, cities, landscapes, rivers, mountain ranges, (what else).

Even computer programs evolve: The portions of a program that do work carry forward into a new version, while the portions that don't are killed off and replaced. Which once again raises the idea that every pattern everywhere is involved in an evolution where the stakes are simply continued existence.

As a property of data sets, or of organized information of any type, learning is not something that people do, or even something that only living things do. It is an inherent natural phenomenon that occurs anywhere and everywhere that there is information, and there is information everywhere.

Learning/adaptation/evolution is a natural law.

What makes it a natural law is the fact that organizations or structures, of anything, either exist or they don't (the "universal hard currency", above), and that changes in those organizations or structures can determine whether they continue to exist or not. 

Biological evolution is just one instance of this phenomenon, an example of it, not an analogous, "similar", process.

Biological evolution is the phenomenon of the progressive changes in living forms that result from the progressive cumulative changes in the structure of DNA molecules.

The key concept is that everything changes. Some changes cause the thing to fall apart, so it doesn't exist anymore. Other changes are merely changes, so the thing continues to exist, and therefore it can change again, and thus it evolves.