docopt gets CLI argument parsing right

Brilliant ideas can be painfully obvious in retrospect. They’ll leave you thinking, “Why didn’t we I think of that before?!” Docopt is that for parsing CLI arguments.

fast_blank: String#blank? in C #

Sam Saffron is part of the team doing Discourse, an open-source Rails-based discussion platform. He’s been kicking all kinds of ass all over Ruby-land lately, but I wanted to share a specific bit of that with you. It’s a new gem, fast_blank.

Basically, due to these awesome-looking flame graphs that Sam added to his MiniProfiler gem. Here’s what it does: it samples stack traces while your app runs, and then lays them all next to each other so you can see where they’re especially deep. He has a demo here, which you can click around and check out what was going on on Discourse’s home page before Sam started working on making it faster.

Anyway, so what’s this has to do with fast_blank? Well, Sam noticed that a ton of time in Discourse’s code was being spent in String#blank?, a method which was added to Rails by _why the lucky stiff way back in the day. Sam attempted to fix it, but making it 100% compatible, yet fast, was difficult.

This lead him to notice that Ruby and Rails have different implementations of #blank?, he decided to fix the issue in Discourse by writing a #blank? in C so that it was super fast.

To use fast_blank, just add it to your Gemfile:

gem 'fast_blank'

That’s it! Your String#blank? will be faster.

Actually, that’s not quite it. Over on GitHub, Sam told me that it’s not up on Rubygems yet, but will be in a few days. For now, you need to clone it down and use Bundler’s :github option:

gem 'fast_blank', github: "SamSaffron/fast_blank"

That’s it.

Now, you may be wondering why I’m making such a big deal out of all of this. Well, this is pretty much an absolutely model open source interaction between a bunch of different projects.

  1. A performance issue was found. Rather than wonder or speculate, measurements were taken.
  2. A new tool was developed to help make sense of the measurements.
  3. Attempting to fix performance in the main project itself, with a pull request.
  4. Recognizing that there was a lot going on here, so fixing it from himself via a patch.
  5. Sharing the fix with everyone until the details in the main projects could be sorted out.

Obviously, this isn’t the end of this story. What will happen next? Will Rails and MRI end up with the same implementation? Can we make it work in Ruby as fast as the C version? Will fast_blank just replace the implementation of #blank? in MRI?

I guess you’ll have to stay tuned to find out. ;)

Explore a piece of Unix history: Dennis Ritchie’s earliest C compilers #

Roger Roach, a.k.a. Mortdeus, was researching the history of Unix and Bell Labs when he discovered an article by the late Dennis Richie explaining two of his earliest C compilers which had been recovered from some old DECTapes.

These artifacts were of too much historic importance for Mortdeus to leave just collecting dust in some obscure corner of the web, so he aggregated the source files and posted them to GitHub as legacy-cc.

The README states:

These are not capable of being compiled today with modern c compilers like gcc. I am only posting these here for the enjoyment and reminiscing of the spark that ignited a soon to be trillion dollar industry. Enjoy. :)

Mortdeus gave 3 reasons for posting the source to GitHub:

  1. Exposure
  2. Convenience (the articles links to each file separately)
  3. Collaborative learning

His plan appears to have worked. The repo on GitHub is growing in popularity very quickly!

Searching, improved with The Silver Searcher #

The Silver Searcher by Geoff Greer is a tool which aims to take searching to the next level. Installation is simple:

brew install the_silver_searcher

If you aren’t on a Mac, or using homebrew, there are other installation instructions on the README. Using it is very straightforward, especially if you have used ack (or grep) in the past:

ag stringtofind ~/code

I know what you’re thinking, what is ag? Well, Ag is the chemical symbol for silver, and it’s a lot easier to type than the_silver_server, so Geoff went with it! There are a number of features that make The Silver Searcher worth giving a try:

  • Speed: 3-5x faster than ack
  • It honors .gitignore, .hgignore, and .agignore
  • ag is 33% shorter than ack (admittedly this is silly, but you can’t argue the facts!)

Discuss this on HackerNews or view the repo on GitHub.

watch – simple periodical execution utility in C #

Guard is a big part of my development workflow for preprocessing styles and coffeescript as well as running tests. TJ prefers the power of C and make:

$ watch make test &
[1] 3794
✔ bifs.components
✔ bifs.dark
✔ bifs.darken
✔ bifs.image-size
...

It should be noted that Guard uses filesystem events instead of polling and has a robust DSL and a long list of plugins, but watch + make make a lean yet useful combo. As TJ says in the README:

use a Makefile. With the large quantity of copy-cats (Rake,Jake,Sake,Cake…) people seem to be forgetting that Make is awesome, if you take a little bit of time to learn it you’ll love it (or at least most of it). Make will utilize mtime and only build what’s necessary, this is great.

Check the project source for more info and usage.

RestSharp: Simple REST and HTTP API Client for .NET #

I’m a big fan of making idiomatic language bindings for APIs. Making common tasks such as URI generation, transport, authentication, and payload parsing configurable options and providing a higher-level API lets application developers focus on the business domain instead of low-level networking.

.NET developers looking to write API wrappers without starting from
scratch should check out RestSharp from John Sheehan. Similar to Ruby’s HTTParty and others, RestSharp provides a number of features to make writing wrappers for REST web services a snap, including:

  • Automatic XML and JSON parsing including Fuzzy element name matching (“product_id” in XML/JSON will match C# property named ‘ProductId’)
  • Support for GET, POST, PUT, HEAD, OPTIONS, DELETE
  • oAuth 1, oAuth 2, Basic, NTLM and Parameter-based Authentication
  • Multi-part form/file uploads
  • Custom serialization and deserialization via ISerializer and
    IDeserializer
  • Both sync and async requests

Example

Here’s a quick example of making a simple request for XML from an API using Basic Auth:

var client = new RestClient();
client.BaseUrl = "http://twitter.com";
client.Authenticator = new HttpBasicAuthenticator("username", "password");

var request = new RestRequest();
request.Resource = "statuses/friends_timeline.xml";

RestResponse response = client.Execute(request);

In addition to using the built-in Authenticators,
you’re free to create your own:

var client = new RestClient("http://example.com");
client.Authenticator = new SimpleAuthenticator("username", "foo", "password", "bar");

var request = new RestRequest("resource", Method.GET);
client.Execute(request);

Perhaps most impressive is RestSharp’s support for a number of environments including .NET 3.5+, Silverlight 4, Windows Phone 7, Mono, and MonoTouch. Be sure and check out the project readme for advanced usage and tips on contributing.

[Source on GitHub]

Node.js on your (jailbroken) iPhone #

Nathan “Too Tall” Rajlich has gotten
Node.js to run his jailbroken iPhone 4. If you’ve got SSH access on a
jailbroken phone, simply extract the .deb package:

dpkg -i node-v0.4.5-ios-arm-1.deb

Now you can see if Node is running:

$ node
> require('os').cpus()
[ { model: 'N90AP',
    speed: 0,
    times: 
     { user: 9209240,
       nice: 0,
       sys: 6997410,
       idle: 255377220,
       irq: 0 } } ]

Nate has created node-iOS to
play with iOS-specific functionality with Node bindings:

var iOS = require('iOS');

iOS.vibrate();
// Quick vibrate, like when you receive text message

if (iOS.compass.available) { // true if the iDevice has a digital
compass
  iOS.compass.on('heading', function(heading) {
    console.log(heading.magneticHeading); // Degrees relative to
magnetic north
  });
}

Of course if you want to play with Node on mobile without
jailbreaking your phone, Node.js powers the JavaScript services in
WebOS
.

[Source on GitHub] [Blog post]

Raphters: A web framework for C #

For those that thought C had been delegated to the internals of your
mobile devices or favorite database engine, Daniel
Waterworth
wants to string you up by
the Raphters.

Raphters is a web framework written in C. Yes you heard that right, a
shiny new framework for the web written in everybody’s favorite
close-to-the-metal programming language. The project gets its name from
RAPHT, a pattern that extends MVC that aims for greater security and
flexibility:

  • Resources include things served up to clients like a database or API.
  • Actions provide ways to interact with a Resource.
  • Processors transform data.
  • Handlers provide the entry point for a request.
  • Templates render data.

A simple Hello World example to demostrate the patter might look
something like:

#include "raphters.h"

START_HANDLER (simple, GET, "simple", res, 0, matches) {
    response_add_header(res, "content-type", "text/html");
    response_write(res, "hello world");
} END_HANDLER

START_HANDLER (default_handler, GET, "", res, 0, matches) {
    response_add_header(res, "content-type", "text/html");
    response_write(res, "default page");
} END_HANDLER

int main() {
    add_handler(simple);
    add_handler(default_handler);
    serve_forever();
    return 0;
}

If you’re a C developer looking for speed (and security) you might give Raphters a
look for your next web project.

[Source on GitHub]

Episode 0.5.1 – MongoDB, NoSQL, and Web Scale with Eliot Horowitz

Steve and Wynn sat down with Eliot Horowitz from 10gen to talk about MongoDB, the NoSQL landscape, and the fun of building at Web Scale. Items mentioned in the show: Eliot Horowitz CTO and Co-Founder of 10gen Dwight Merriman CEO & Co-Founder at 10gen NoSQL is a loose term for Key Value Stores, Graph Databases, […]

inception – The movie, explained through C code #

I’ll be honest: this is one of the cooler things I’ve seen recently:

/*
 * Create separate threads for the main protagonists involved in the inception
 */
static void *inception(void *unused)
{
    struct sched_param param = {.sched_priority = 99 };
    int policy = SCHED_OTHER;
    if(!getuid() 
       ||
       !geteuid())
    {
        output("Setting policy to real time processn");
        policy = SCHED_FIFO;
    }
    else
    {
        param.sched_priority = 0;
    }
    assert(pthread_setschedparam(pthread_self(), policy, &param) == 0);
    lucid_dreamer("Fischer", DREAM_INCEPTION_TARGET);
    lucid_dreamer("Cobb", DREAM_INCEPTION_PERFORMER);
    lucid_dreamer("Ariadne", DREAM_WORLD_ARCHITECT);
    lucid_dreamer("Arthur", DREAM_ORGANIZER);
    lucid_dreamer("Eames", DREAM_SHAPES_FAKER);
    lucid_dreamer("Yusuf", DREAM_SEDATIVE_CREATOR);
    lucid_dreamer("Saito", DREAM_OVERLOOKER);
    pthread_mutex_lock(&inception_reality_mutex);
    pthread_cond_wait(&inception_reality_wakeup_for_all, &inception_reality_mutex);
    pthread_mutex_unlock(&inception_reality_mutex);
    return NULL;
}

That’s right: karthick18’s ‘inception’ repository contains a 1900 line C program that simulates the plot of the movie Inception. How, you ask?

We need to go deeper.

Have you heard of the Dining Philosopher’s problem? Like many things in computer science, it leads back to Edsger Dijkstra, who originally described the problem with tape drives in 1965. Here’s a summary:

Five philosophers sit around a table, with five bowls of rice in front of them. In between each bowl of rice is a chopstick. A philosopher can do two things: think and eat. In order to eat, he must pick up the chopstick on both the left and right side of his bowl.

dining philosophers

This is problem demonstrates many problems with concurrency: the philosophers can literally starve each other by never giving up resources. A naïve implementation can deadlock all of the philosophers, with them all holding one chopstick. Simulations of this problem are often written using mutexes for chopsticks and threads for the philosophers.

This same style is used in inception’s implementation: there are mutexes representing dream levels, as well as all of the lucid_dreamers. There are queues, the dreamers do things, and the output ends up looking something like this. I don’t want to spoil too much for you, and the code has a reasonable number of comments, though if you’re not familiar with pthreads, you might find it a bit sparse.

Totally cool.

The code on GitHub.

webdis: HTTP + JSON API for Redis #

HTTP is the dial tone of the web. Apps that speak HTTP tend to grow in popularity, such as CouchDB whose built-in HTTP-based RESTful API makes it super easy to store and retrieve JSON data.

Nicolas Favre-Felix gives some web love to Redis fans with Webdis, an HTTP interface for Redis that serves up JSON. Built in C, Webdis aims to be fast like Redis itself. The URL structure takes the pattern of:

http://127.0.0.1:7379/COMMAND/KEY/[VALUE]

Let’s look at some concrete examples:

curl http://127.0.0.1:7379/SET/hello/world
→ {"SET":[true,"OK"]}

curl http://127.0.0.1:7379/GET/hello
→ {"GET":"world"}

curl -d "GET/hello" http://127.0.0.1:7379/
→ {"GET":"world"}

While still early in development, the feature set is impressive. Webdis currently offers:

  • Support for GET and POST
  • JSON and JSONP parameter (?jsonp=myFunction).
  • Raw Redis 2.0 protocol output with .raw suffix
  • HTTP 1.1 pipelining
  • TCP or UNIX socket Redis connections
  • CIDR or HTTP Basic Auth security
  • Pub/Sub using Transfer-Encoding: chunked. Coupled with JSONP, Webdis can be used as a Comet server
  • Built-in support for json, txt, ‘html, xml, xhtml, png, and jpg
  • Custom Content-Type types based on file extension, or ?type=some/thing
  • Cross-origin XHR, if compiled with libevent2 (for OPTIONS support)
  • File upload with PUT, if compiled with libevent2 (for PUT support)

Check the README for the complete list. If you’d like to contribute, Nicolas is thinking about how to add other HTTP verb support like PUT and DELETE, Websockets, and Etags.

Also, if you missed it, be sure and catch Episode 0.4.5 with @antirez, the creator of Redis.

[Source on GitHub] [Comment on Hacker News]

#45: Redis with Salvatore Sanfilippo

Wynn caught up with Salvatore Sanfilippo aka @antirez to talk about Redis, the super hot key value store. Items mentioned in the show: VMWare signs the paychecks for Salvatore and Pieter Noordhuis Redis is an open source, advanced key-value store and data structure server wherein keys can contain strings, hashes, lists, sets and sorted sets […]

Selene: Asynchronous SSL/TLS #

A common issue when working with SSL/TLS libraries is that they are synchronous, you ask the library to do something, then have to wait for it to finish before you can do anything else, Selene by Paul Querna aims to change this.

Selene is an C library that sits on top of your SSL/TLS library — currently only OpenSSL is supported — providing an asynchronous API for interacting with it. Currently selene is in an early stage of development, but aims towards the following goals:

  • Completely Asynchronous: it does no IO itself, but provides notifications that IO should be done.
  • Test Driven Development: A test framework to test all code paths.
  • Pluggable Backends for Cryptography: Currently only focusing on OpenSSL, but others are possible.

Selene is definitely something that you are probably going to want to checkout if you’re working with SSL/TLS and want to get the best performance and throughput possible.

[Source on Github] [Mailing List]

CSpec – BDD with C #

Very nice, readable syntax from CSpec that alllows you to do behavior driven development with C.

#include "cspec.h"

static int nbefore_each;

describe "CSpec before_each"
  before_each
    ++nbefore_each;
  end

  it "should be called before each spec"
    nbefore_each should equal 1
  end

  it "should be called before each spec"
   nbefore_each should equal 2
  end
end

[Source on GitHub]