Never Delegate Understanding
The famous American designer, architect, and filmmaker Charles Eames once said that we should "Never delegate understanding". I think these words contain immense wisdom for how we approach learning, building tools, and progressing knowledge. Applying the principle of 'Not delegating understanding' to web programming, I think we need more easily understandable tools, along with better documentation for how those tools fundamentally work and the design assumptions and constraints inherent in them.
If you're not familiar with the work of Charles and his equally brilliant wife Ray, the documentary Eames: Painter and Architect does a beautiful job of capturing their incredible story. The film is a showcase of how they mainfested their life philosophy, brought talented people together, and created a magical environment where people could express themselves and realize their talents.
One of the important aspects the film touches on, is that Charles and Ray were able to innovate within their craft only because they actually meticulously worked with the materials and tools available to them, regullarly, first-hand, not delegating. They deeply understood the design assumptions and constraints of their era's tools and materials. This is the notion embodied in Charles' aphorism.
I think there is something essential to this idea of never delegating understanding. It's almost a way of approaching things authentically, that captures the immense value inherent in having a deep understanding of a subject: its assumptions, and principles. When trying to innovate, more often than not, new ideas don't materialize in a vacuum. Instead, worthwhile ideas accumulate gradually, from the process of genuinely committing yourself to a nuanced understanding and exploration of the systems, tools, and activities of your craft. Trying to use tools which build on certain basic principles that you don’t yet understand, while glossing over the fundamentals of a subject, is likely to limit your potential for truly meaningful innovation.
Not having a deep understanding is also likely to make your life tougher in the long run: eventually there will come a point where you will have to use those tools for purposes that they were not originally intended for, and when that happens you won’t have sufficient understanding to adapt those tools to your novel needs.
This difficulty of adapting things when you don’t really understand them, and the frustration of trying to do so, is something I believe many people will sympathize with from past experience.
In general, I believe, that there are two ways to give people power to create and innovate:
- Abstraction - making tools that hide the workings of their more complex internals (i.e. building more powerful tools)
- Understanding - teaching people the fundamental principles of how systems and tools work, so they can leverage that knowledge to build new tools, and adapt existing ones.
The two should co-exist. But, too much abstraction, and not enough knowledge, makes people feel like tools are magical, and in many ways - too magical to truly understand. Magic can be frightening and disempowering.
It seems to me that right now, in the world of web programming, we're doing a lot better on the abstraction front than we are doing on the teaching front. Even though our tools are constantly improving, and we're coming up with better and better abstractions, not enough people really understand how those tools work. I think that lack of understanding is actually slowing them down, and also preventing them from being able to truly innovate. The tools themselves also perhaps do not offer enough transparency into their inner workings.
Frameworks like Rails, and other abstractions, are often making things too mystical, or 'assumed to be fool-proof'. Meanwhile, the knowledge of how these tools work, and the assumptions their construction is based upon, is not being adequately communicated.
This makes them seem more magical and ultimately intimidating to people than needs to be the case. People look at some framework code, and see something like: "show website", and have no idea how that one line of code "show website" is able to generate their entire website. Of course this is a ludicrous example, but maybe not so far away from the sensation I had in early experiences with web programming. People are afraid to ask about the basics, because they may be afraid that their questions sound stupid. In fact the questions they have are likely the right questions to be asking.
When people don’t understand the basic assumptions the tools they use encapsulate, they don’t have the ability to question those basic assumptions - and to really ask: “Is this the best way that things could be done?” They accept the wisdom of the crowd, and assume that the current tool is the best tool available, because otherwise a better tool would have already been made. This type of belief is what makes people feel complacent, and unempowered.
I encountered this not too long ago when I started working on my first startup: The Shared Web. The idea there was to make a site where you could see at any given time where on the internet all your friends are browsing. From a technological perspective, this posed the problem of many thousands of concurrent connections. I began to think about why the Internet was so focused on pull and polling mechanisms rather than push. I also started looking at Apache a bit more closely, this was in late 2009, and realized that it might have some big limitations. I had this feeling that Apache may not be the best server solution possible for particular types of web programming - but at the same time I had a feeling that... well... if that was the case, surely someone else would have built a better server by now - there’s a reason everyone uses Apache!
It’s true that there’s usually a reason that people use a particular tool: history, convention, availability - but it's often not because that tool is the best way to do that thing. I fell victim to what we might call the “Pangloss” innovation syndrome - where I just assumed things were as great as they could be. Eventually, I found this article about the C10K problem, and realized that other people were thinking about this same thing. I then found out about NodeJS, and saw that people like Ryan Dahl were already questioning some of the fundamental assumptions web servers like Apache were built upon. My assumption that the way things are done is the best way they can be done was unfounded, and doing this deeper research was an important step in boosting my confidence and encouraging me to always trust the merit of my questions.
In my philosophy courses at university, we would try to translate arguments we read in terms of their assumptions and conclusions. Premise A, Premise B, and Premise C, therefore Conclusion D. We would then try to see if there were problems with any of the assumptions that would render them false (e.g. A is not a valid assumption because of Y). In most fields, this refutation of basic assumptions is often how innovation happens, and that’s why it’s important to dig deep and understand and question those assumptions. You want to question the assumptions, while still being able to use things at a higher level, and appreciating the value of employing existing abstractions. It also helps to unearth what are the sometimes ‘hidden’ assumptions that are being made - and those ‘hidden’ assumptions are often the best opportunities for innovation.
But, getting back to abstraction and understanding. In the case of web programming, I think what’s missing on the understanding front is high-quality, freely available documentation about the fundamentals of how web stacks work, what frameworks are, etc. These can be written even in the form of questions and answers, so that we highlight and answer the basic questions some people are afraid to ask (e.g. how does the Internet work at a deeper level in terms of the TCP/IP stack). I'm also a big believer in teaching the history of a subject, because I think that seeing the history of how a subject/approach evolved makes things seem much simpler, and removes the feeling that the people building the tools and abstractions we use simply have greater mental capacities than us. More often than not, people who have built tools that move us forward were reacting to immediate questions and needs that emerged.
Now, it may be that I haven't googled enough, but I think that there's really not a lot of great documentation on some of the most basic concepts in web programming. At least they’re not compiled in a nicely presented format. I've taken time to sit with a few developers and designers recently to talk about this feeling, and also in some cases to also explain some of these core concepts to them. From those I talked to a bit, I can see that many shared this experience of having to sort of figure things out on your own. I can also already see that having some of those basic questions answered has had a big impact in terms of their confidence when programming, and their general sense that things are no longer mystical incantations (e.g. Rails Magic). They understand the tools they are using, and have a sense of how they were made. This makes them feel empowered. It makes them feel like they can build their own tools too. The deeper you go, the more fearless, and confident you become.
I think a good learning curve for web developers may be something like:
- Explain the basics of web-programming at a high level (e.g. how servers, browsers, and frameworks work)
- Let people start using a framework like Rails, and explain to them how the pieces of Rails correspond to the overall framework concepts
- As an exercise, then have them try to roll their own framework, server, etc. - just for educational purposes - to learn about the tradeoffs that are made, and to think deeply about what's really going on.
This is the path I went down, and I feel I benefited from doing this … well except that we ended up using the framework I built in an actual production environment for The Shared Web. That maybe wasn't the best idea, even though the site didn't go down too often.
Having talked to a few people who have participated in Hacker School, it sounds like that's the general approach they take too: immersing people in the code. You can see the difference it makes when talking to people who come out of the program. So it looks like some organizations are taking this approach. People become less afraid of going deep, and that's a great thing.
One of the things I'd like to do in the coming months is to spend some time putting together some lessons on the basics of web stacks, and web frameworks. Here's a few topics I think are important, and I'd love to hear if you guys have other suggestions, or things you wish people told you:
- Figure out what a Web Framework is.
- Understand the principles of Model-View-Controller. Alternatives to MVC
- Understand what a Web Server is.
- Understand the different ways that routing can be done: e.g. At the server level, at the language level with a front-controller approach (etc.)
- Understanding the idea of the client-server model in principle (e.g. many databases run as services).
- Explaining the concepts of the HTTP protocol, APIs, REST,
- How Source Control Systems Work and why we use them.
- Shells, and Shell Scripting
(As a side note, I was talking to our Systems Director recently, and he was saying that the basic principles of how things work, and how to do things when it comes to data centers, and scaling sites is even less well documented. Some documentation there would be great too.)
Looking forward to your feedback. Never delegate understanding, dive in fearlessly.