In a recent interview with Entrepreneur magazine, Paul Graham said something that reminded me of an interesting story:

When Google started, there were eight to 10 successful established search engines already, and search was so uncool that they were trying to get people to call them “portals”.

I landed my first real programming job at a long defunct dot-com startup when I was a senior in high school in 1999. We were building an EBay for home improvement. Homeowners would post home improvement projects, and contractors would bid on them. We were going to take a cut out of each job.

Alex Popescu wrote a blog post asking some questions about RethinkDB and SSD performance. There is a related Twitter conversation happening here. There are two fundamental questions:

• In which cases does SSD performance begin to degrade over time?
• How does the TRIM command affect performance degradation?

The questions are very deep and I cannot do them justice in a single blog post, but I decided to post a quick write-up as a start.

A few days ago Salvatore Sanfilippo wrote about his work on a new Redis diskstore – a reworking of the old VM layer based on a different set of assumptions and tradeoffs. I was very interested in the new approach because it is very similar to the RethinkDB caching architecture in principle. When we designed our caching layer we had a number of constraints we needed to satisfy:

On my computer, the callstack of a new process is around 10MB [1]. Modern operating system automatically reserve some amount of virtual memory and install protections on the page below the stack to create a segmentation fault on stack overflow. This ensures that a stack overflow won’t go corrupting random parts of memory.

We want to have a lot of coroutines, so they should have smaller stacks, maybe 16 or 64KB. This makes stack overflow an even greater possibility, but at the same time, coroutines implemented in user space don’t get this checking for free–we have to build it ourselves. In the process, we can even do better: we can give some information about the coroutine which crashed.

Previously, I wrote about using coroutines in RethinkDB. Coroutines are a nice alternative to callbacks because they are easier to program in, and they are a nice alternative to threads because of their greater performance. But how fast are they?

Just using an off-the-shelf library like libcoroutine isn’t as fast as you might think. The graph below shows the huge performance degradation of a naive implementation of coroutines (the short red bar) compared to the old callback- based code (the blue bar on the left). But with the right set of optimizations, we can recover a level of performance about equal to the non- coroutine version. With these optimizations, throughput is recorded as the pink bar on the right, which is within the margin of error of the original version.