The Counterintuitive Physics of Turning a Bike

>Do you remember the insights that were crucial in learning to ride a bike or drive? The author mentioned bicycles and inadvertently missed an opportunity to show another "unknown reality" that beautifully illustrates his point. Consider typical adults (not physicists) with 20+ years of bicycle riding. If you ask them, "how does a bicycle turn?" , they'd respond that "you just lean into the direction of the turn and it turns." But that's not the complete picture of what actually happens.[1] If you were to capture the bicyclists with a high-speed camera, you'd notice that there's always a micro movement of countersteering to make the bike momentarily lose its balance and "fall into" the turn. Then there's an immediate correction of the steering to match the turn. This all happens in the span of milliseconds. Even though the human body "discovered" the countersteering by way of kinesthetic feedback, the bicyclists' brains don't explicitly communicate this intermediate step to others. (Tacit knowledge.[2]) It's not always the case that we "simplify" reality for innocent purposes of pedagogy. (E.g. we tell "lies" to children about "numbers", "functions", etc and as they get older, we successively remove the layers of lies as they get into high school and college math.) Instead, we often don't even know the reality (e.g. bicycle countersteering) to consciously omit it. That extra reality may not even be important. If one human is teaching another human, you can leave the "counters

Here's what it means to understand how something works. If you can accurately predict what _change_ you need to make to its workings in order to achieve a desired _change_ in the resulting behaviour, then you know how it works. The more reliably and precisely you can predict the impact of various changes, the better your understanding. I know how a bicycle works. I can prove that I have a very basic level of understanding, for example, by showing you that if I want the bicycle to go faster, I can pedal harder; if I want the bicycle to slow down, I can squeeze the brakes. I can prove a greater level of understanding by showing you that I can increase efficiency by increasing the tire pressure, or showing you that it's impossible to make a left turn ( [this video] ) unless I first start by turning right. That's what it means to explain how a bicycle works. (Notice that saying the bicycle is made of atoms does not help you do any of this.) I don't think you can show me that kind of understanding of large language models. Which is to say, I don't think you can accurately predict what changes you need to make to the internal structure of an LLM to cause specific changes to the interesting high-level ("emergent") behaviours that people are seeing. That's what is meant by "nobody knows."