While I’d also like to speak of personal failures today, I’ve been thinking a lot more about system failure.
Alan Perlis famously wrote that you shouldn’t bother to learn a programming language unless it changes the way you think about programming, and Erlang delivers in spades. Almost a year in* to ScrabbleCheat, only now do I feel like I’m understanding the value (and love) of Erlang. Namely, what it means to write dependable software.
Another blog post said it best: “Erlang is about building reliable systems.” Singling it out to concurrency, or distributed systems, or hot-code loading, or simply for it’s functional nature is missing the point.
It’s not immediately obvious: the most used and referred-to book for newcomers trying Erlang is Joe Armstrong’s Programming Erlang. This was my first book learning Erlang, and while it has a great tone and expresses many of the introductory concepts clearly, it sadly seems a bit out of date, and has it’s priorities in the wrong order. If you only read the first 200-300 pages, you come out not knowing exactly how big a deal the OTP is.
As an example, an entire chapter on using Makefiles to build your code is probably unnecessary: any developer worth their salt knows enough about Makefiles to compose shell commands, and that’s essentially all he demonstrates.
I don’t mean to be too hard on it, but here are some tips/resources for those interested in exploring Erlang:
Learn You Some Erlang is an excellent, excellent introduction. It starts slowly with the language, but always gives you context of how Erlang fits with other programming models, simple but direct writing, common pitfalls, and even “Don’t drink the Kool-Aid” sections where he brings Erlang away from the hype and back down to Earth. And while the language sections are fine enough, the OTP chapters near the end have you understanding exactly why gen_server and gen_event are the way they are.
Erlang and the OTP in Action is another good choice, though un-free. They walk you through generating an Erlang release by hand with the default tools, which is good to do at least once or twice. They also have an excellent focus on the OTP, though with more emphasis on it’s practical use in production applications rather than the important ideas behind it. If you feel like you want to buy and work through something to understand Erlang, I would recommend this.
Rebar as a build tool. It’s lean, it usually does just what you want it to, and the source code is embarassingly clear and easy to read if it doesn’t. The mailing list is friendly, and it’s easily extensible. Much better than raw Makefiles.
Get to learn the tools: Appmon, Dialyzer, Common Test, and others. It’ll boost your productivity, and give you a sense of confidence you won’t get often.
You’ll never really love the language until you design a proper application with OTP. While every book/tutorial wants you to write a chat server, I rather enjoy optimization problems, like ScrabbleCheat.
Back to failure: after programming in Erlang, I feel naked in other languages without supervisors and trap_exits. Lack of proper Actors means distribution (and therefore fault-tolerance) is a chore, never mind the concurrency benefit. Erlang has been a dream to play with, and I would recommend you do a hobby project or two in it ^_^
*= Madly Brilliant and I joke about the fact that I’ve been working on this for so long. Really, it’s been about a year since the initial commit, where I implemented trie using gb_trees, way before I had even heard of a GADDAG. While I like to say “a year in” to make it sound like a major commitment, I’ve sadly only gotten to work on this very rarely. Were I someone who hadn’t just moved into a city and started my first job while maintaining an overseas-then-cohabitating relationship, I would call this 2-3 months of real work.
Often when coding and confronted by a problem you don’t know how to solve, it’s nice to have music that’s fun/good/comfortable enough to tune the world out, but not one that will overtake you. Here’s a bunch of YouTube links that I bounce between.
My awesome cousin Maddy pointed me to this video:
I’m not familiar with stop-motion used in music, but the tune is excellent (especially after playing hundreds of hours of Smash in the Zelda Palace level). This guy makes excellent videos:
My favorite Zelda song on Guatemala’s national instrument!
“null is the billion-dollar design mistake. null breaks abstractions, composability. null eats live babies. null hammered my toe. null sucks.”
And it had a lot to do with my frustrations in working with other people’s code. This blog sometimes tries to explain technical concepts to non-technical people, so let me take a moment to explain to you what the hell null is if you’re not into programming, and why it’s such a buzzkill.
We’ll start with a metaphor: writing programs is very much like writing recipes, and the computer is your team of chefs. Recipes and code have a similar structure with two major parts: you have a section for the stuff you’re going to use (in a program we call them declarations, in recipes it’s the list of ingredients), and you have a set of instructions telling you what to do with said stuff (in both cooking and programming, these are your instructions). The basic gist is that ingredients + instructions = delicious, delicious, product.
This isn’t a perfect metaphor, but it serves. Now, like recipes, oftentimes for complicated products, you’ll rely on other recipes; e.g. you’ll need to make guacamole for your burrito, or a complicated cake batter to eventually make a wedding cake. As long as you follow ingredients + instructions, you will get delicious products, and you can build recipes on top of recipes to make grander, more delicious recipes without a problem. This allows your recipes to be composable: you can build them up from smaller ones. This is critical in making anything of scale.
There’s a hidden step between ingredients and recipe, one that rarely gets written down, because it’s the one that sucks the most: preparing the ingredients. When your cooking time is advertised as 40 minutes but you have to mince garlic, have potatoes cooked and peeled and in cubes, that alone will add about an hour. Cooking shows look effortless, but when you try the same things at home you find it’s not so easy when the ingredients don’t come expertly prepared in beautiful glass bowls.
So suppose you write a brilliant recipe, and you hand if off to the cooks to make. They see “4 cups shallots in a blue bowl” in the ingredients. Later, they have a bunch of whole shallots, and they see the instruction “place the shallots from the bule bowl into the pot…” What do they do?
In programming, we often “return null” when something doesn’t go as expected, which is the equivalent of the chef reading the directions deciding to put dogshit into the pot.
Here’s the rationale: if you bite into dogshit, you know the recipe went wrong and will immediately stop eating. Another reason: you can’t cover every case, so if we put in some dogshit, we can still keep cooking! Why shut down the entire kitchen because one stupid chef was careless with one of their directions?
Do you see the problem though? There’s dogshit in the goddamned food.
Just like the weird shallot example, we have it (very, very frequently) in programming: you can declare that you’ll use some data, and you can write instructions using that data, but you might miss the step of saying what that data is, or by the time the data gets to that instruction it’s no longer what the coder thought it would be, and the current instruction is impossible.
Like many of Computer Science’s worst blights (e.g. goto), this was originally a feature! You can imagine the inventors thinking “we’ll let programmers be responsible people, no need to baby them!”
BAD. THIS IS BAD THINKING. While you shouldn’t tie the hands of programmers, you also shouldn’t give them something that never, ever provides a useful value, like keeping dog shit in the kitchen.
“But Paul!” says the imperative programmer, “I frequently use null checks when I want to say that something went wrong, such as fopen not getting a valid file handle! To which Paul says, “couldn’t you use an error code? Proper exceptions? This is 2011, kids!”
Sorry. Back to cooking.
Given that most chefs of the 90’s were making food in kitchens with dogshit flowing to and from them, what did we end up doing? We ended up writing recipes that looked like this:
Make guacamole.
*Check if it's dogshit*
If it isn't, make refried beans.
*Check if it's dogshit*
After that, make some rice.
*Check if their dogshit.*
The punchline is, if you didn’t check, you’d only know you had dogshit when you took a bite of food. And if you did find dogshit, what would you do? Normally, return more dogshit for someone else to cook with or figure out.
And the frequent checks for dogshit make for uninteresting and sad recipe writing.
All that is minor annoyance, the real issue is this: you can’t compose your recipes or programs if there is random dogshit flowing from all sides. You need a certain level of trust in other code that it will either work as promised, or if it fails, you can prepare adequately without your first warning having to be biting into dogshit.
The functional languages (unsurprisingly) are on top of this: by eschewing state and favoring evaluation of expressions rather than execution of statements, you rarely end up with uninialized data. In the rare cases you want to, even a solution as simple as the Maybe monad provide you null-like device that’s statically typed, and any function that calls it knows that it needs to handle the Nothing case.
Blah. Rant over. Just code responsibly, comrades, as people break down and cry because of code. I mean, I haven’t. But people have, I’m sure.
Here’s an old, but good article, ostensibly about writing. The author is pretty funny, and I think most of what he writes applies just as well to any endeavor you choose to pursue with any interest.
Having tried to find my theatre identity in college, only to become a computer scientist, the question of “How do I get good at …” is something I thought about (and continue to think about) all the time. Most of what I think is already present in the article in one form or another, but with my words:
It starts with the Lorrie Moore quote at the top: don’t even do it unless you feel like you have to. Otherwise, you won’t want to when you will have to. The Lorrie Moore quote he opens his article with is almost verbatim to the most prescient acting advice I’ve ever received from a professional actor, when I asked if he had any advice for me going into the Real World: “If you can be talked out of it, do so.”
The idea was crazy at the time, but it turns out I found a way to do it. And while it’s hard to imagine for anybody who’s only met me in the last 2-3 years, leaving theatre was the hardest thing to do in my life. Science intimidated the hell out of me. It was something I didn’t do, it was for those people, and I’d felt my whole life that I should only do what I was good at, and that was Art.
So first things first: if you don’t yearn to be amazing in your field, or do amazing things in it, I suggest taking a good look at what you’re doing, and asking if you actually want to.*
Stop talking/thinking about it, do it. It’s really easy to have big dreams and ideas, and we often talk about them with friends. The hardest part is doing it.
I follow this advice, in that any time I’m thinking of doing something but don’t want to, I just say “Just Fucking Do It Now.” Works best of anything else I’ve tried, which is to say, I still leave many tasks unfinished :-p.
In almost the same vein: have a healthy mix of study with that practice. But you probably need the practice more. Which is to say, don’t just play 4000 games of Chess or Starcraft or hours at piano without some reflection and critical analysis of how those you look up to do it. You can practice blind, or practice with a guide.
My only reluctance in this guideline is that I tend to put too many hours on the analysis/thinking/observing part, and not enough on the practice part. I’ll watch a lot more Starcraft than I play, read more chess books than I play, read about programming languages more than I use them, etc. But I imagine many have the opposite problem.
Well, bloviation complete.
*= Two little notes: a) not everybody wants to be the best/do amazing things at X/Y/Z, they just want to do it, happily, at their own pace. That’s totally fine, in which case, carry on ^_^. And b) the bullet point makes it seem like you will know immediately when you love/care about something. It may take some time: I didn’t know I was into computer science until I took the theory classes, about a year in.
