There is an equally concerning dual problem to convolution, and that is fragmentation. While convolution is about multiple ideas hiding in the same terminology, fragmentation is about a single idea hiding between multiple terminologies. This causes problems when we understand different aspects of an idea
For example, I know that eggplants are delicious, and know many recipes for them, but then I’m placed in a bizarre British cooking show, and told that today’s ingredient is aubergine. What is an aubergine??? I’ve heard it’s a nightshade, that sounds poisonous…
Notice there are actually two places for fragmentation to hide, in the hierarchy, Thingspace – Thoughtspace – Wordspace, it can hide at either junction. We could have multiple words referring to the same idea. This is analogous to having multiple textual aliases for some function in code. It’s annoying, but not that hard to solve: textual substitution is easy, we can just look up the definition and unify them. In practice, humans are very good at on-the-fly substitution/translation, so these sorts of fragmentation survive but don’t cause us too much trouble. The more difficult sort of fragmentation is at the Thingspace -Thougthspace link, analogous to two different algorithms that do the same thing. Unlike in Wordspace, where most translations are just direct substitution, and equivalent ideas are strictly equivalent, computing equality of generalized ‘things’ is Hard (think function equality, but harder). Part of this is because ‘things’ want to be weak; the way in which things are “equivalent” matters.
There’s a more general problem. Maybe two ideas don’t represent exactly the same thing, but they sorta seem to have related subcomponents. The “sameness” of thoughts/things is related to the degree to which their subcomponents match.
It seems to me that the optimal way to think (and the optimal way to design an AI.. hint hint), is to break all problems down into a complete set of primitive ideas, and then memoize all the ones that get used frequently. An immediate question comes up: How do we know when we’ve found a minimal concept? This business of seeing internal structure, and factoring concepts into their primary components is exactly the sort of thing category theory is good at, and so this problem has a solution. The notion of “universal property” exactly encodes what we mean by “minimal concept relative to some property”. Roughly, a concept has a “universal property” if every other concept with that property can be described in terms of it. For example, a “product thing” for things A and B has exactly the information to give you an A or a B via projecting. Any other thing that could give you an A or a B can be described as a special case. I encourage you to actually read about universal properties, because I can’t give satisfactory coverage here.
The whole point of this is that Fragmentation is a big problem whenever you’re trying to coordinate research between diverse fields of study, because it leads to the encryption problem. Fragmented ideas “want to” be united, in that they have higher entropy than their unification. When two ideas merge into a lower entropy state, the information difference (“counterfactual pressure”) is released as a sort of free energy and can be captured via “counterfactual arbitrage“