The one constant in life is change.  Turn around for but a moment and that tiny little infant you once knew becomes a young adult with his own wants and needs and behaviors.  He becomes you and yet not you.  He becomes who he is and yet who he still was.  But it is not enough to appreciate this wisdom and wistfully, blissfully to enjoy this.  Rather like drinking fine wine where the experience is heightened by the knowing, perhaps we would like to know exactly what is going on in the brain.

We shall not dive into the microscopic details of neuron spiking and neurotransmitters at a particular point in time.  We shall not dive into the Brodmann’s Areas of differentiating brain regions or re-connecting them in the integration problem.  We shall instead look at the growing brain at a high level through the competing lens of optimization, equilibrium, and fitting models over time.  Here was a behavior in the past; here is a behavior now.  What changed? 

Imagine a simple regression as a starting point.  Here is an early experience, infant learner.

There are a few experiences (blue squares) – heavily abstracted and leaving aside for the moment what constitutes an experience – and a best-fit linear regression line (black line).  A regression line is a fancy word for a one-liner summary of the set of experiences.  It is an average.  It can curve and twist at higher complexity to better fit the experiences, but the point is the same. Here is a later experience, seasoned learner. 

There are some additional experiences (red squares) to add to the prior experiences.  The best-fit regression curve has shifted to better summarize the total set of experiences. 

That’s it.  It is that simple.  We have regression lines to provide a summary description of a small set of early experiences and a summary description of a larger superset of later experiences.  We can compare the two and describe changes and create a story to explain how and why the summary/average shifted.  But the truth is that it is we who are doing the explaining and not the regression.  We are taking massive liberties to explain something from images that claim to explain nothing more than the average at a particular point in time.  This is the great issue with statistics.  They are greatly helpful in purely describing the situation.  They do nothing in explaining how and why and what to do. 

Question 1: Should we invest effort in exploring better fitting curves? 

Question 2: Should we invest effort in exploring different data subset selections?

There is limited time and limited effort to invest.  But investing here is data reporting; it depends on what the manager wants out of the report.  With no manager, it is playing blind with the data.  The manager provides the theory.  The theory is the manager.  Without a theory, we may spend so much time exquisitely stirring the pot that we forget to spend time figuring out what dish we are making. 

So we must first ask a hypothetical question to pull us back on track: where did the original starting point early experience, infant learner go?  Is he gone forever, overwritten in the annals of read/write computer-like memory?  What really changed? 

The second question to ask is what is the point?  The point is that artificial neural networks, including the ever-popular deep learning models currently in vogue, are derivatives of complex regression curves.  One of the most common models of the working brain is based on regression.  One of the most common models of the working brain provides only a summary average of the set of experiences that we choose to include.  One of the most common models of the working brain tells us what we told it.  This again tells us nothing at all about what really changed.  How should we approach the problem, the mystery of what goes on in the brain?

As a (re)starting point, the Rescorla-Wagner model, crossed with the work of Schultz, et al. (1997) point out how to test learning behavior through pairing events.  Here is an obscure factoid from perusing their works: consider that a bell ringing precedes the arrival of a favorite food.  Eventually, the bell becomes associated with the food such that the bell signals its arrival.  It takes, say 4 repetitions before this happens and the bell becomes learned / associated with the food. 

Now the researchers become a bit cruel and ring the bell falsely, and have it signal no food.  It takes, say 4 more repetitions before this happens and the bell becomes disassociated / extinction trained away from the food.

Finally, the truly mean researchers change their minds again and ring the bell to precede the favorite food.  How many repetitions does it take to re-train / re-associate the bell with the arrival of the food? 

Answer: one.

Congratulations, we just violated regression. 

Implication: there is not one single, unitary experiential summary regression to describe our memory and our ensuing behavior.  It is not a tug of war on a single line between food and no food.  It must be something more akin to multiple, competing paths.  There is no average.  The earlier experience is still present, unchanged but only suppressed.

It is “as if” there were multiple brains and behaviors stored in the brain.  This does not mean there is rampant schizophrenia or a holographic brain or a mirrored brain on another plane of existence or anything quite so avant-garde.  It is merely an abstraction, an “as-if” to start us off in the theory of what we are even looking for.

It is as if the experiences from different times and perhaps places are kept separate, autonomously and automatically.  It is as if the experiences of infancy, toddler-hood, and adolescence are still intact even if not particularly salient or applicable most of the time as adults.  It is as if memory and learning are compartmentalized.  It is as if saying, “I am ____ years old,” is the inappropriate regression summary.  It is as if we should instead be saying, “I have ____ years of experience,” on which to draw.  The adult becomes who he is and yet who he still was. 

Do I contradict myself?
Very well then I contradict myself,
(I am large, I contain multitudes.)

-Walt Whitman

As for how to explain this on a cellular and molecular level with neuron spiking and neurotransmitter release and re-uptake?  That is a story that will have to wait for another time.