Abstraction is a term thrown around a lot in the context of computer science. It refers to a complexity that can be wrapped up in a way so that you can treat the whole thing as a simpler thing. In programming, this could be writing a function to sort a list and then being able to call that function on a list without having to think about how it works. I'm increasingly thinking about abstraction as the the main thing that enables complex reasoning.
In other words: the ability to take a big complex jumble of information and represent it mentally as one cohesive chunk is what allows complex reasoning. I don't think this is a particularly hot take, but I want to develop this a bit because I think it's a pretty useful mental model.
Why is Abstraction Important?
The reason abstraction is important is because humans really can't hold many things in their head at once; the typical number given for working memory capacity is 6-8 pieces of information. It seems to me that this is one of the fundamental limitations of the processing that humans can perform.
When the number of things that need to be kept in mind is too high, it is mentally straining and easy to unknowingly make mistakes.
Take a moment to think of the last time you were reasoning about a problem that you found mentally straining. Were there a lot of things you had to keep in mind?
Here are the experiences that came to mind first for me:
- designing a deterministic finite automata that correctly identifies a given language
- writing a function that lexes a string into a series of tokens according to some rules
- multi-threaded synchronization in an operating system
In all of these, there are many cases to reason about, and they're often all related in some garbled way. I think this is the reason why these things are hard. Here are some of the next steps I took that made these problems easier to deal with:
- giving a name to different parts of the automata that describe what properties of a string they're responsible for checking
- refactoring the lexing function into a class with smaller, semantically meaningful methods
- thinking of high-level invariants while doing multithreaded programming
All of these made it easier to reason about the problem of interest with fewer distinct pieces of information at once. Notably, all of them did this by partitioning the problem into sub-problems that could be solved independently and then composed together.
Beyond my very anecdotal examples, two studies from the psychology literature tell a similar story.
In the two famous "Chunking" studies, participants with varying levels of chess ability were briefly shown boards with chess pieces. In the first study, it was found that masters were able to reproduce the shown board with a much higher degree of accuracy than novices, but only when the pieces were in a reasonable place for a chess game. When the pieces were placed randomly, the masters performed no better than the novices, able to recall the position of only around 7 pieces.
In a follow up study, participants were subjected to a similar setup, except with the ability to take more glances at the target board. The time between the placing of pieces during reproduction was recorded. All players would place a few pieces at a time, then pause for a second before placing a few more. It's hypothesized that these groups of pieces are "chunked" together into a concept. More experienced players were able to reproduce the board in fewer glances by recalling more and larger chunks, suggesting that they have a larger collection of chunks with which to represent the chess boards seen.
Great Thinking Tools enable abstraction
I think that many tools are useful precisely because they enable us to reason at a higher level.
One way that tools do this is by performing some reasoning for us, freeing up mental capacity. An obvious example is calculators. Another example is the ownership model in Rust, which guarantees memory safety in safe Rust. This closes off a whole class of errors that programmers need to check their program for. Static type systems are a similar example.
We can think of mathematical proofs as doing this too. Once we're convinced that a certain mathematical object has a property, we can reason about the object and the property without holding all the details of the proof in mind. In this way, the complexity of math becomes tenable, as each individual theorem can be proven with a toolset of increasingly high-level abstractions.
Many of these tools also help us structure our reasoning in terms of higher level concepts. Good notation (ex. $\Sigma$ for summation) as well as specialized words or phrases ("lexer" instead of "thing that turns a text stream into a token stream") give us concise handles that are useful during reasoning and communication.
Of course, we can't talk about tools for thought without mentioning writing and language. I constantly talk to myself, and I don't think I'd be very productive without this habit to be honest. My guess is that this is useful because it's easier to produce cohesive ideas when producing full sentences, and cohesive ideas are easier to reason with than many half-baked thoughts.
I also love to jot down my thoughts into a hierarchical bulleted list. I think one of the reasons this is useful is because it allows me to think about something new without spending mental space remembering the old thought. This allows me to jot down the essence of many thoughts, and then devote my full capacity to developing each of them.
Consequences of this?
I think the main utility in having this mental model is to be able to notice when you're struggling to reason about something and think about how to approach it more productively. There are many problems that initially seemed quite difficult to me that later seemed simple once I figured out the right approach using abstraction.
Is there a way to group your reasoning into more cohesive chunks? Can you give a name to things? What are the sub-problems that you can solve individually?
And while learning: what are the pre-requisite concepts that are taking up too much mental space (and that you should study further first)?
This also suggests that there's high upside to using and building tools that make important and hard-to-reason-about problems "simpler" by making them more amenable to abstraction. This is one of the reasons I think compilers are so cool and why I'm excited to work more on the tensor shape inference project.