The Changelog

Open Source moves fast. Keep up.

Utterly simple data capture with Dataset #

Dataset is databases for lazy people from Friedrich Lindenberg.

Why do we see an awful lot of data stored in static files in CSV or JSON format, even though they are hard to query and update incrementally? The answer is that programmers are lazy … This is what dataset is going to change!

We’ve all been there. You have an idea, but before you can start working on the fun part you need to setup a database. Now you don’t have to do that. Use dataset to take care of the grunt work and let your self concentrate on the juicy bits.

Facebook released RocksDB #

Even Facebook Open Source stands upon the shoulders of giants. The Facebook Database Engineering Team just released RocksDB, a persistent key-value store for flash storage, based on LevelDB by Sanjay Ghemawat and Jeff Dean at Google.

RocksDB builds on LevelDB to be scalable to run on servers with many CPU cores, to efficiently use fast storage, to support IO-bound, in-memory and write-once workloads, and to be flexible to allow for innovation.

Reminder: Upgrade your Postgres today! #

Last week, I told you all about an incoming security patch for Postgres. Well, today, it’s here. Please check out this page and upgrade your Postgres. As the Postgres team says, ‘This is the first security issue of this magnitude since 2006.’

What’s the issue?

As always, you can find the latest information about security patches via the CVE system. Here’s the one for this vulnerability, CVE-2013-1899.

There are three things that can happen with this vulnerability:

  • Denial of Service. Error messages can be appended to files in Postgres’ data directory. This can fill up disks, or cause Postgres to crash.
  • Configuration Setting Privilege Escalation. If they have a legitimate login, and the username and database name are identical, then that user can set a config variable as the superuser.
  • Arbitrary Code Execution. The ‘boss level’ of vulnerabilities. If they can do both of the above things, and can save files outside of the data directory, then they can execute arbitrary C code.


What versions are affected?

Versions 9.0, 9.1 and 9.2.

Where can I find more?

The Postgres team has a FAQ for this release, and here are the release announcements.

You can also see the commit that fixed the issue, with all the gory details.

Or, discuss on Hacker News.

mini_record brings schema declarations back to your model #

From Davide D’Agostino, mini_record brings schema declarations back to your ActiveRecord models in the style of DataMapper, MongoMapper, and others:

class Post < ActiveRecord::Base
  key :title
  key :permalink, :index => true, :limit => 50
  key :comments_count, :as => :integer
  key :category, :as => :references, :index => true

when you execute MyModel.auto_upgrade!, missing columns, indexes and tables will be created on the fly. Indexes and columns present in the db but not in your model schema will be deleted* also in your db.

Don’t go installing in your production apps just yet. Check out the extensive README for advanced usage.

#40: Riak revisited with Andy Gross, Mark Phillips, and John Nunemaker

Wynn sat down with Andy Gross and Mark Phillips of Basho and John Nunemaker of Ordered List to talk about Riak, Riak Search, and moving an open source community to GitHub. Items mentioned in the show: NoSQL smackdown, live from SXSW 2010. Are you web scale? Drop us a and let us know who […]

sequelize: ORM for MySQL and Node.js #

Although NoSQL seems to get a lot of attention in Node.js circles, probably because so many document-oriented databases speak JavaScript fluently, wrappers and drivers for relational databases like MySQL are starting to emerge. The latest entry is Sequelize from Sascha Depold. Sequelize provides basic Object-Relational-Mapper (ORM) between your JavaScript objects and MySQL database records.

Install Sequelize via npm:

npm install sequelize

… and require the library:

var Sequelize = require("sequelize").Sequelize

Once the library has been loaded, set up your database connection by creating an instance of Sequelize, with optional host and port.

var sequelize = new Sequelize('database', 'username', 'password', {
  host: "my.server.tld",
  port: 12345

Call the define method to create your models, mapping fields to database types:

var Project = sequelize.define('Project', {
  title: Sequelize.STRING,
  description: Sequelize.TEXT

var Task = sequelize.define('Task', {
  title: Sequelize.STRING,
  description: Sequelize.TEXT,
  deadline: Sequelize.DATE

You can now work with instances of those models with new:

var project = new Project({
  title: 'my awesome project',
  description: 'woot woot. this will make me a rich man'

var task = new Task({
  title: 'specify the project idea',
  description: 'bla',
  deadline: new Date()

Keep in mind that Sequelize was built completely on the Node async architecture, so there are callbacks for most everything, including save():

// way 1
task.title = 'a very different title now'{})

// way 2
  title: 'a very different title now'
}, function(){})

Sequelize also has very slick method chaining to let you batch save objects:

  {save: project}, {save: task}
], function() {
  // woot! saved.

Be sure and check the documentation for more advanced features like Associations and Finder syntax.

[Source on GitHub] [Hompage] [Sascha on Twitter]

High Quality MySQL driver for Node #

Now that Felix Geisendörfer launched his node powered site, he’s tacked the much needed task of writing a high quality MySQL protocol implementation in pure JavaScript.

Using the high performance networking capabilities of node, and the awesome speed of V8, it’s now feasible to implement binary network protocols without resorting to writing C++ extensions and wrapping blocking APIs in thread pools.

Felix is a great programmer, but could use some help in the form of sponsorship and/or patches. We all win when high quality drivers are written and shared with the community.

Design Goals

  • TDD: All code is written using test driven development, code coverage should approach 100%
  • Simplicity: The MySQL protocol is easy, a good parser should reflect that
  • Efficiency: Use fast algorithms, buffers and as little memory as possible.
  • Portability: Should run anywhere node runs
  • Completeness: The goal is to support the full MySQL API.
  • Compatibility: MySql >= 4.1

Check out the code and more info on his git repo.

nStore – easy database written for node.js in node.js #

Ever wanted to fire up a quick node.js app, but didn’t want to mess with finding a database driver that worked with the latest node and then installed and set up that database. Ever wanted the simplicity of an in-process db engine (like sqlite).

Wait no more, nStore is here. It’s written in pure js and is contained in a single js file to include in your node project. It supports full CRUD operations on key based documents.

The data is persisted to disk using a smart append-only format that’s easily recoverable by design. You can update/insert documents by key, read documents by key, remove documents by key… (get the picture, it’s key/value based).

Also you can build a node stream interface from a document collection or query the collection using a filter function and get the results as a single callback.

Removing stale data is easy and efficient too. Just supply a filter function to the automatic data compacter and it will remove the offending documents while compacting the data file. Thus the file is never needlessly bloated with delete lines.

Here is a short example that creates a database, inserts a document, and pulls it back out.

// Load the library
var nStore = require('nstore');
// Create a store
var users = nStore('data/users.db');

// Insert a new document with key "creationix""creationix", {name: "Tim Caswell": age: 28}, function (err) {
    if (err) { throw err; }
    // The save is finished and written to disk safely

// Then later pull it back out
users.get("creationix", function (err, doc, meta) {
  // do something with the data

Try is out as a standalone database, or use it with Connect as a session store.

[Source on GitHub]

#18: NoSQL Smackdown!

While at SXSW Interactive, Adam and Wynn got to attend the Data Cluster Meetup hosted by Rackspace and Infochimps. Things got a bit rowdy when the panel debated features of Cassandra, CouchDB, MongoDB and Amazon SimpleDB and started throwing dirt at everybody else’s favorite NoSQL databases. The participants: Stu Hood from Cassandra Jan Lehnardt from […]