Backstory

There are times when you need to middle truncate a string. In many cases it’s for UX/human purposes, though in some situations it’s the best way to generate unique string for a length-limited field. This is the case I ran in to recently in trying to automate submission of IAP to both Google Play and the App Store which require short unique names for each SKU.

Which of the following lists is sorted in the most “natural” fashion?

A:
    Elementary Season 1 Episode 1
    Elementary Season 1 Episode 10
    Elementary Season 1 Episode 11
    Elementary Season 1 Episode 12
    Elementary Season 1 Episode 13
    Elementary Season 1 Episode 2
    Elementary Season 1 Episode 3
    Elementary Season 1 Episode 4
    Elementary Season 1 Episode 5
    Elementary Season 1 Episode 6
    Elementary Season 1 Episode 7
    Elementary Season 1 Episode 8
    Elementary Season 1 Episode 9

Async-What

The secret to building high performance sites which depend on external web services is asynchronous HTTP. The trick to asynchronous HTTP (or anything with the exception of UI) is to avoid callback hell. Enter futures/promise objects. When used correctly they make doing substantial asynchronous IO, relatively straightforward.

In the past couple days there’s been several articles about how Google’s interview process has failed. The headlines are overblown, inaccurate really, but to be expected giving the goal of getting clicks. In reality Peter Norvig was talking about counterintuitive findings. They were surprised to find that some of their best employees were not unanimously voted in. That makes a lot of sense given my personal experience with the Google interview process and my opinions of what matters in making software engineering hires.

Background

There are two main sources of latency in the backend of web applications: rendering (HTML templating or data serialization) and IO (database or external service calls.) Today we’ll look at the latter and more specifically focus getting rid of extraneous database round trips. The fastest query possible is one that doesn’t happen.

It’s a fairly common use case to have a latitude and longitude and want to find the closest objects to a given point. While there are heavyweight solutions: MySQL Spatial Extensions, PostGIS, they can be more trouble than they’re worth especially if you’re making use of an ORM.

I recently gave a presentation on graphite and statsd to the East Bay Django meetup It’s more about the importance of stats in general when running services, web or otherwise. The slides are available and embedded here. There’s also a related demo project that turned out to be much more interesting than the slides themselves.

A recent Django security release added a new configuration option, ALLOWED_HOSTS. It’s optional in 1.3 and 1.4, but required in 1.5 setups. The relevant detail is that when enabled in production Django will throw 500 errors if the Host header doesn’t match one of the values in ALLOWED_HOSTS.

LinkedIn does a pretty good job of email communication. I probably read 50% of the emails they send me and I’ve remained subscribed to them for quite a while now.

This is part two of a two-part post on the general categories of software engineers. To pick things up at the beginning, check out the first.

You can’t hire solely rock stars and hiring the wrong person is worse than leaving the position vacant, but what makes someone right? In the first of a series of posts on building and manging teams, I’d like to talk about the types of software engineering positions. Who you should hire depends on your exact needs.

In what is probably my biggest WTF with Django to date, it doesn’t validate your models before saving them to the database. All of the necessary code exists and when a dev sets up her models she usually adds the relevant validations using EmailField, URLField, blank, null, unique, …, but unless you explicitly add code the constraints won’t be enforced (adequately.) Some things will be caught with IntegrityErrors, but not everything and not consistently.

A new year, a new theme. Since switching XOR Media from it’s hand-coded, mostly placeholder, last May I’d been planning to build a custom theme for it and over the holiday break here in the US I had a chance to tackle it. I started by looking around for inspiration. I was looking for something clean and simple with a horizontal top-nav, but that was all I started with. From there I skimmed over the wordpress’s theme directory and visited a lot of the sites that I read, but normally don’t visit thanks to Google Reader.

If you’re in the startup world you’ll run across the startup curve, rather frequently. The canonical post on the subject was done by Fred Wilson, but the original concept came from a talk given by Paul Graham

About the only thing a rock star developer and their namesakes have in common are they’re both exceedingly rare.

I’ve run across several articles recently that discount the value of rock star developers and after reading them I’m left with the impression that the authors wouldn’t know one if their life depended upon it. The article “6 home truths about rock star developers” by Andrew Oliver begins by claiming that you can’t afford to hire all senior developers. I thought this was about rock stars, what’s seniority got to do with it. The fact that you’ve been doing something a long time has absolutely nothing to do with being exceptional at it or else we’d have a lot more 65 year old musicians debuting on the top 40 charts.

We hate ads. You and I do, but apparently some people don’t or else we couldn’t make any money with their use. When I released Muni Alerts with ad support about 5 months after originally publishing the App I decided to give users the option of turning off ads, as in a check-box in the App’s settings no payment required. It’s worked out just fine and I think more people should do it.

It’s been a busy few weeks for me at work, we launched our app, were featured in the iPad app store home page, and have been called out in other app store sections since. It’s been a wild ride, capped off by a trip to Comic-Con where I spent 5-6 hours a day on my feet giving my spiel about our app and company. Our booth was pretty sweet, definitely the best in the area we were in, and it was capped off with a really cool iPad video wall we built. The wall was a great conversation piece and drew a lot of attention.

If you’ve spent some time with Django chances are you’ve run across its signals system. It is most commonly used to provide hooks in to the model setup and CRUD processes, but there are many other uses of the facility throughout the code including auth, db, and the request life-cycle. What it doesn’t offer out of the box is a signal that’s called during startup after your models have been loaded.

As part of a project I’m currently working on we wanted to have a brief splash video that plays the first time our app starts up for branding purposes. The video also kills a little time as some initial processing is happening.

I ran across this gist on Hacker News which has a quick rundown of the latencies of various operations. It’s nothing earth shattering, but if you compare the intra-data center round-trip (0.5ms) to reading 1MB of data from disk (20ms) you can pretty quickly see why systems like memcached work so well even after adding the overhead of reading from memory (0.25ms) and some time for the sending and receiving of the request. Summed up the memcached response is often back in a few ms, nearly an order of magnitude faster than reading from local disk.

Python decorators are extremely useful when used with care, and using them is really straightforward. Coding them up on the other hand can be complex and requires, reading along with trial-and-error. There are some helpful modules out there worth taking a look at, but for this use-case we won’t be making use of them.

If you’re storing or transmitting time information in any way shape or form (web-services, databases, queues, NoSQL, caches, log files, or just plain txt files) the safest and thus best thing to do is to use UTC (Coordinated Universal Time) everywhere.

What we’re after

jQuery’s extend function is really useful and if you’ve ever written a plug-in for the library chances are you’ve made use of it. I’ve run across a use for this functionality in python and it also makes an interesting interview question (regardless of language.)

If you want a quick and easy way to extract a clip from a larger video it doesn’t get much simplier than using avconv (the new ffmpeg.) All you need to know is the start time and duration of the section you’d like to extract. Once you have those you can plug them in to the following command.

If you’re looking to serve both HTTP and HTTPS out of EC2 from python, using gunicorn and ELB you’ll need to add a bit of configuration to get gunicorn to correctly detect which scheme the original request was made with and thus allow Django (or whatever framework you happen to be using) to correctly generate urls with the matching scheme.

If you run in to a situation where you’d like to determine if a commit has been pushed to a remote git branch there’s a quick and easy command-line way to do so.

I’ve run in to a couple situations where I’ve wanted to iterate over a list in pairs lately and decided to create a simple iterator to clean up the code a bit.