Time for the ol’ annual blog post! (Well… more on that below.)
I added this section to my annual reading list blog post last year, and I feel like it’s provided some much-needed structure to my self-directed learning. Let’s see how I did!
I set out to cover two broad, somewhat meandering topics in mathematics in 2019:
- The foundations of mathematics (logic, proofs, set theory)
- Select topics in abstract algebra (primarily group and category theory)
I think this was the goal I made the most progress toward: I deepened my understanding of logic and now have a firm grasp of set theory and reading/writing proofs (at least for someone who isn’t a practicing mathematician). The books I found most helpful for exploring these topics were Set Theory and Metric Spaces, A Programmer’s Introduction to Mathematics, How to Prove It, and, weirdly, The Elements of Real Analysis. I hadn’t really planned to do real analysis this year, but this book offered an overview of set theory (covering the algebra of sets, functions, and finite and infinite sets), the real numbers (including the Cantor set), and topology of Cartesian spaces. I skimmed the sections on convergence, continuous functions, and infinite series, and left the bulk of the book (infinite series, differentiation, and integration) for whenever I review calculus.
Most of my group theory self-study was pretty ad-hoc and cobbled together from online sources (such as Wikipedia and Math Stack Exchange); I’d like to find a more structured resource in 2020. For category theory, I started digging into this book and these lectures by Bartosz Milewski, and I’m loving it so far (I got about halfway through in 2019 and am nearly done now). I wasn’t quite ready for the Mac Lane book, but I’m excited to dig into it this year. The mathematical topics I’d like to cover in 2020 include the Curry-Howard-Lambek correspondence (the “Lambek” bit is new to me), mathematical logic, boolean algebra, and discrete math.
I’m disappointed I didn’t get around to creating any math ‘zines in 2019, but it turned out I didn’t need to create them to get the level of understanding I was looking for. (I tend to do this by explaining topics to myself on my walk to and from work, filling in holes in my understanding when I get back to a book/computer; I later found out this is a variation on the Feynman technique.) I may still create ‘zines in 2020 if there’s interest, but it would be more for teaching/sharing than cementing my own understanding (though arguably I’d still derive value from structuring and ordering the topics for someone else).
My audacious (and fuzzy) goal in 2019 was to somehow make progress on all the languages I want to speak (some are languages I’ve learned and want to brush up on; others are pretty much new to me). As expected, an overly ambitious, blurry goal resulted in not much measurable progress, though I did get a ton of value out of Justin Searls’ KameSame and will be using it in earnest this year.
Learning from last time ‘round, this year I have two specific foreign language goals: to get to a Spanish B1/B2 and a Japanese JLPT N4/N5 by the end of the year. Also ambitious, but much less so than “master five languages lol” and much easier to measure. In the meantime, I’ll continue to read French and German and ideally participate in conversation (e.g. our informal French group at work) in order to keep my existing abilities up.
Okay! On to the new goals.
I’ve tried to make note of the times in my career I’ve gotten dramatically better as a programmer (e.g. attending the Recurse Center, moving from a customer-facing application development team to an SRE team, designing my first major cross-team service), and I’d like to try to deliberately recreate that on my own in 2020. I’m still sort of putting together a self-study curriculum, but here are the broad topics (divided into 12-week deep-dive “batches,” similar in length/scope to a Recurse Center batch):
01/01 - 03/25: Programming theory: Haskell, category theory, and the lambda calculus (the Destroy All Software series on the lambda calculus is excellent). 04/08 - 07/01: Programming environment: Go, Docker/containers, Kubernetes/container orchestration, and the Linux kernel. 07/08 - 08/30: Databases: PostgreSQL, the relational calculus, and query optimization. 09/07 - 12/30: Optimization problems: the Bellman equations (reinforcement learning, MDPs, dynamic programming), game theory, and statistics.
I may change the content/order of these deep dives as I go along, but I tried to find topics where I have basic to advanced intermediate knowledge and really want to dig deep to learn more.
Finally, as I hinted earlier, I think I could be more disciplined in my approach to tracking my learning progress and synthesizing new ideas, so in 2020, my goal is to blog on technical topics at least monthly.
Whew! Okay. On to the reading list.
I mostly read what I set out to read in 2019 (see below for the ones I’m rolling into next year). Here are a few extras I read:
The First 90 Days
This was a re-read, but I always learn something new each time I come back to it. I joined ZipRecruiter as director of engineering earlier this year, and this time around I found the sections on matching strategy to situation and achieving alignment especially valuable.
I’ve been interested in classical cryptography and historical oddities like the Voynich manuscript for awhile now, so I really enjoyed this as a kind of survey course of unsolved crypto-mysteries (ranging from undeciphered ancient writing systems to the cryptograms of the Zodiac Killer!).
Patterns of Enterprise Application Architecture
While I read this more or less cover-to-cover, I think its real utility will be as a reference guide (especially the chapters on concurrency and distribution strategies).
Never Split the Difference
I honestly didn’t expect to like this book (I was worried it would have too strong a “bro” vibe), but I actually really enjoyed it (less so for the war stories and more for the negotiation tactics and strategies). There was still a bit of swagger, but overall, I found it down-to-earth and self-aware.
Getting to Yes
Roger Fisher, William L. Ury, and Bruce Patton
I picked this one up specifically because Chris Voss references it in Never Split the Difference (generally in contrast to his own approach, which is derived from his years of training and experience as a hostage negotiator). I did find Getting to Yes to be very academic and process-oriented, though I did glean some interesting insights (particularly around separating people from problems).
The Leadership Pipeline
Ram Charan, Stephen Drotter, and James Noel
I read this on the recommendation of a friend and colleague of mine, and I particularly liked the idea that the pipeline zig-zags, illustrating that often during advancement, “what got you here” (the skills that made you stand out in your prior role) “won’t get you there” (are not necessarily the ones you need to excel in your new role). Speaking of…
What Got You Here Won’t Get You There
…I picked this one up to dig further into the new skills I might need to acquire in order to succeed at increasingly higher levels (both personal growth level and company growth level, as covered in The First 90 Days and The Leadership Pipeline), but instead, this was more about identifying the weaknesses and bad habits one might have succeeded in spite of thus far and addressing them. Still useful, but not quite what I was looking for (luckily I borrowed it from the library).
The Elements of Real Analysis
Robert G. Bartle
This was just a general overview of real analysis; I found the early, foundational chapters on the algebra of sets, finite and infinite sets, and the topology of Cartesian spaces the most applicable.
Solving Mathematical Problems
This was a quick read (absolutely doable in a single sitting), and I particularly enjoyed the general strategies for problem solving and the examples from number theory. Hard to believe he wrote this when he was just fifteen years old!
Software Design Decoded
Marian Petre, André van der Hoek, and Yen Quach
I was hoping to glean some real insights from the experiences of seasoned developers, but was disappointed to find a Page-A-Day calendar of statements ranging from obvious (“Experts prefer solutions that they know work,” “Experts go as deep as needed,” “Experts know how things work”) to vaguely patronizing (“Experts focus on the essence,” “Experts use design methods (selectively)”).
Nicole Forsgren, PhD, Jez Humble, and Gene Kim
This book was the result of several years of studies conducted on the effect of lean, agile, and DevOps strategies/mindsets as applied to software engineering teams, with the aim of identifying what distinguishes high-performing teams from low-performing teams. These studies mostly affirmed my experiences and instincts around testing (such as who should write tests, their correlation with success, &c), architecture, management, and the role of DevOps/SRE in an organization, but it was good to see these beliefs backed up by concrete data. I particularly liked the section on servant leadership vs. transformational leadership.
Strategize to Win
Carla A. Harris
This was a selection for our tech management book club at work. The “Starting Out” section was less relevant for me (choosing a career, acquiring necessary skills, and so on), but I found the “Stepping Up” section very useful—particularly the ideas of performance currency and relationship currency, which put into words an idea I’d formed years ago: high performance gets you your initial recognition, but relationship-building is what opens the door to advancement (whether networking for your next job, raising money for your next venture, or rising through the ranks of executive leadership).
A Philosophy of Software Design
I technically finished this one in the first week of 2020, but started and read most of in 2019. I thought this was a thoughtful, well-structured approach to software design, and I agreed with most of it (particularly around the role and handling of errors and the preference for “deep” modules whose simple abstractions/API provide high-leverage access to powerful internal logic), but I wasn’t sold on the author’s opinion of TDD (that it’s “tactical-only” and tends to incur debt). I view tests not only as indications of correctness, but as living documentation and powerful design tools, and I’d like to see that viewpoint expressed more widely. (Maybe this is an instance where I need to be the change I want to see in the world.)
These poems are astounding. You should read them.
Here are the books I want to read in 2020, starting with the four books I’ve rolled over from 2019:
Categories for the Working Mathematician
Saunders Mac Lane
As I mentioned last year, I want to more deeply understand the relationships between category theory and programming, and I think this book will help me better grasp the theoretical foundations I’ve started building with Bartosz’s lectures.
I. H. Herstein
Groups, rings, and fields have come up in my category theory reading, so I’d like to take something of a side-quest into various topics in abstract algebra to better understand how these objects interact.
Michael Anderson and Susan Leigh Anderson, Editors
I’ve read through a few of the papers in this anthology (and even gave a talk on machine ethics a couple of years ago), and I think this text will be really generative (with regard to my interest in the intersection of human and machine problems).
The History of Western Philosophy
I’ve read bits and pieces starting in college (I think sections of it were required reading for one of my philosophy courses), but in 2020 I’d like to finally read the whole thing.
And here are the new books I want to read in 2020:
The Oxford Handbook of Philosophy of Mathematics and Logic
Stewart Shapiro, Editor
I’m particularly interested in the chapters on Wittgenstein, intuitionism in mathematics, and higher-order logic.
An Elegant Puzzle: Systems of Engineering Management
I’ve read probably around a dozen books on engineering management, but this is the first one I’ve encountered that meshes the challenges of people management with systems thinking. I may end up writing a full blog post about this book later this year!
Elements of Clojure
I cheated a bit and read the first several pages, and I’m already hooked. I get the feeling I’m going to learn a lot more about software engineering than just Clojure programming.
Naming and Necessity
This has been on my philosophy “to-read” pile for awhile, and given that it’s mentioned in the first chapter of Elements of Clojure, I think it’s time I read it.
The New Peoplemaking
This is the first of several books (ending with The Mismeasure of Man, below) recommended to me by Rein Henrichs, with whom I worked when I was consulting with Test Double. The New Peoplemaking is about communication and interpersonal dynamics in the context of a family, but I get the sense it has a lot to teach about other domains as well (work, friend groups, other local communities).
Computer Power and Human Reason
This is very much in the machine ethics/intersection of computation and philosophy area of my reading, and I’m particularly interested to see how prescient the author was in his arguments about the nature and future of AI (Computer Power and Human Reason came out in 1976).
Having Thought: Essays in the Metaphysics of Mind
More philosophy! Having browsed the table of contents, I’m particularly interested in the chapters on Hume, supervenience, Dennett, and Searle.
Sources of Power: How People Make Decisions
Gary A. Klein
In particular, this book is on how experts make decisions (rather than how someone makes decisions in a domain that’s brand-new to them).
The Mismeasure of Man
Stephen Jay Gould
One of the Amazon reviews on this one describes it as “an important reminder of what IQ measurement really is and the ugly racist history that has long plagued the field.” 💯 💯 💯
The Anatomy of Peace: Resolving the Heart of Conflict
The Arbinger Institute
This one was recommended to my by my friend and former colleague Michael Schoonmaker. Like The New Peoplemaking, its focus is on family but I think it’ll be applicable in a wide variety of situations.
This will be our tech management book club pick for February, and I’m interested in reading it not just as an employee with a boss, but as someone who serves as someone else’s boss (and in that capacity, have my own habits and predispositions that others might need to manage).
The Making of a Manager
I’m particularly interested in this book because the author emphasizes her ramp-up into management, whereas most management books I read are more retrospective (written by leaders after they’ve worked as managers for many years, if not decades).
The Art of Readable Code
Dustin Boswell and Trevor Foucher
Dustin did a lot of foundational work on our search team at ZipRecruiter, so I’m interested to get his general take on writing maintainable software.
I’m especially interested in this one as the informal textbook for my third Recurse Center-style deep-dive (see Programming, above).
Designing Data-Intensive Applications
I’ve heard nothing but great things about this book, which covers all sorts of tradeoffs, considerations, and challenges encountered while scaling distributed systems, and I am probably unreasonably excited to read it.
The Left Hand of Darkness
Ursula K. LeGuin
I really enjoyed Cixin Liu’s The Three-Body Problem, and I realized I’ve been neglecting my fiction reading for way too long. While poetry books aren’t super long, I tend to bucket my fiction and poetry reading together, so this year, I’m going to intentionally not commit to reading any particular poetry collections in order to focus more on fiction. Further, none of this fiction is actually new—these are books I’ve been meaning to read forever and haven’t gotten to (and LeGuin is at the top of that list).
Modern dystopias call for dystopian science fiction!
I’m actually pretty sure I read Ringworld in high school, but I remember pretty much none of it, so I’ll give it (another?) go.
This is one I’ve been meaning to read since I first heard about it (which I think was something like 15 years ago).
I did indeed read fewer papers in 2019 than when I was in grad school, though I managed to read all the ones I set out to. Here are the additional papers I read beyond those:
The Part-Time Parliament
This paper was actually pretty entertaining, though I have to admit I don’t understand the protocol well enough to implement it based on this paper. (Having the Raft paper, below, and talked with some of my coworkers, it sounds like I’m not alone here.)
Paxos Made Simple
This paper makes good on the promise of the title (at least, made simpler).
In Search of an Understandable Consensus Algorithm (Extended Version)
Diego Ongaro and John Ousterhout
AKA the Raft paper. This consensus algorithm was much more approachable than Paxos, and I thought the writing was particularly clear for an academic paper (one of the co-authors is the same Ousterhout as the author of A Philosophy of Software Design, above!).
Dynamo: Amazon’s Highly Available Key-Value Store
Giuseppe DeCandia et al.
The title pretty much says it all. (I’ll be pretty brief with my descriptions of these last several papers, since they’re more or less just descriptions of the underlying data models/algorithms, but I’ll note any surprising insights or interesting tradeoffs.)
Cassandra - A Decentralized Structured Storage System
Avinash Lakshman and Prashant Malik
Cassandra’s data model is derived from Google’s Bigtable (see next paper).
Bigtable: A Distributed Storage System for Structured Data
Fay Chang et al.
This powers a bunch of core Google services, including search and Gmail!
Kafka: a Distributed Messaging System for Log Processing
Jay Kreps, Neha Narkhede, and Jun Rao
We’re starting to adopt Kafka for more and more use cases at work, so I thought it’d be a good idea to refer back to the paper (especially regarding delivery guarantees).
MapReduce: Simplified Data Processing on Large Clusters
Jeffrey Dean and Sanjay Ghemawat
I think I expected this paper to be more groundbreaking/illuminating, but that’s likely because the general pattern for parallelizing computations over huge datasets isn’t really “new” anymore (this paper is over 15 years old!).
Spanner: Google’s Globally-Distributed Database
James C. Corbett et al.
I thought it was pretty cool that Spanner instances are Paxos state machines, and I particularly appreciated the details around the specifics of Google’s Paxos implementation.
The Google File System
Sanjay Ghemawat, Howard Gobioff, and Shun-Tak Leung
First, component failures are the norm rather than the exception.
This paper really drove home for me the inevitability of failure in large, distributed systems, and part of the reason why our move to Kubernetes at work hasn’t been as conceptually difficult for me as it otherwise probably would be. (This is probably true of many of the distributed systems papers I’ve read, but few call out failure as the norm quite so explicitly.)
Amazon Aurora: Design Considerations for High Throughput Cloud-Native Relational Databases
Alexandre Verbitski et al.
The paper is a good overview of Aurora, though there are a couple of areas that I would have preferred be more explicit. For instance, while the paper notes that:
The database engine is a fork of “community” MySQL/InnoDB and diverges primarily in how InnoDB reads and writes data to disk
…it doesn’t explicitly mention that it disables InnoDB’s change buffer (though a careful reader can infer this). As a result, it’s very possible to read the paper and not realize the performance penalty for tables with lots of secondary indices (normally, MySQL uses the change buffer to lazily merge updates to these indices, but without the change buffer enabled, these updates have to be write-through).
Finally, here the papers I want to read in 2020 (hopefully I’ll pick up a few more along the way):
Making Reliable Distributed Systems in the Presence of Software Errors
This was Joe Armstrong’s doctoral thesis, and at almost 300 pages it’s probably more of a book than a paper, but I’ll include it here so I have an excuse when I only read one paper this year!
Similarity of Neural Network Representations Revisited
Geoffrey Hinton et al.
These last three papers are all machine learning papers by Geoff Hinton’s team; this one’s on about comparing different neural network representations.
When Does Label Smoothing Help?
Geoffrey Hinton et al.
Label smoothing is a common tool for improving generalization in multi-class neural networks, but it’s not really well understood why it works. I’m hoping this paper sheds some more light on the topic.
Lookahead Optimizer: k steps forward, 1 step back
Geoffrey Hinton et al.
Finally, this paper is about a new optimization algorithm for improving stochastic gradient descent in deep neural networks.
Off we go!