Lark is a REST interface for Redis #

You might have seen our post on webdis a couple years ago. Like webdis, Lark is a REST interface for Redis.

At it’s core it’s just a way of transforming HTTP requests into redis commands, but it comes with a few additions to make this a little more sane.

It comes with a Flask blueprint and a Django app, but it should work with any python web framework.

Disclaimer: Alex (this post’s author) is the creator of Lark.

Utterly simple data capture with Dataset #

Dataset is databases for lazy people from Friedrich Lindenberg.

Why do we see an awful lot of data stored in static files in CSV or JSON format, even though they are hard to query and update incrementally? The answer is that programmers are lazy … This is what dataset is going to change!

We’ve all been there. You have an idea, but before you can start working on the fun part you need to setup a database. Now you don’t have to do that. Use dataset to take care of the grunt work and let your self concentrate on the juicy bits.

Give any SQL database a REST interface with Sandman #

Let’s say you have a SQL database. You might wish it had a REST interface and a nice admin UI. That used to mean writing a bunch of code but now with Sandman it’s a one liner.

Zero boilerplate code is required. In fact, using sandmanctl, no code is required at all. Your existing database structure and schema is introspected and your database tables magically get a RESTful API and admin interface.

Sandman was created by Jeff Knupp and can be used with any database that SQLAlchemy supports.

AlienFeed: Reading Reddit with Python #

Jared Wright is an impressive high school student who has created a command line interface to Reddit.

AlienFeed is a simple, but effective, way of interacting with Reddit and takes advantage of Python’s ability to open items in your web browser for you. It comes with a man page (unlike some Python CLIs) and a custom set of bash completions.

To try it out: pip install alienfeed.

Here is screenshot to whet your appetite:

AlienFeed displaying the top 10 links on /r/python

#108: Exercism.io with Katrina Owen

Adam and Jerod talk with Katrina Owen – Panelist on Ruby Rogues, Instructor at Jumpstart Lab and creator of Exercism.io, an open source platform for crowd-sourced code reviews on daily practice problems.

Explaining Python implementations #

Have you ever wondered why there are so many different kinds of “Python”? CPython, Cython, RPython, PyPy, Jython, and IronPython are all flavors of Python but not everyone knows why each exists.

#107: Balanced Payments with Marshall Jones

Andrew and Adam talk with Marshall Jones from Balanced Payments about all they do in open source, and how they approach being an open company that desires to release as much software as they can as open source. Sponsored by DigitalOcean – Use the code mentioned on the show to save $10! If you’re a […]

Pynt versus Invoke

Just the other day I saw someone mention on Twitter that Pynt was their new favorite pure Python replacement for make. I have some minor experience using Invoke, a very similar tool. This prompted me to experiment with both utilities and report on my findings. Background Both tools take great advantage of decorators in Python […]

Zero: A Wrapper Around the ØMQ Python Library

I don’t use ØMQ but by just looking at some examples of how to use the standard python library and this wrapper, “a wrapper for a wrapper” just makes sense. So let’s look at some code!

Requests 2.0 coming soon with proxy support and bug fixes #

Requests logo

The author and maintainers of the popular Requests library are working on a crucial version 2.0 release. As you can see this new release will include a large number of bug fixes and features which are backwards incompatible with the 1.x branch of Requests. Not to worry though, the transition from 1.x to 2.x will be far less painful than the transition from 0.x to 1.x. Most of the backwards incompatibility arises from how 2.x will handle headers (as will be explained below).

Improving HTTPS proxy support

Perhaps the most exciting part of this upcoming release is the improved proxy support. Version 2 will include support for the CONNECT verb which will make talking to HTTPS services possible from behind a proxy. For example: anyone who wishes to use Requests on PythonAnywhere‘s free tier will be able to once version 2 is released. As noted in the pull request, this would not be possible without the amazing work of the contributors to urllib3 — the library on top of which Requests is implemented.

Fixing a subtle bug with headers

Beyond adding proxies, there was a particularly nasty bug on Python 3 where some headers could not be set using native strings. As this was a backwards incompatible change this is only being fixed for the first time in version 2.0. If you have run into this problem your headaches will be long gone.

Adding new convenience methods

Finally, if you are a Requests user who creates their requests carefully by hand, the new method on the Session object will prepare them for you! You no longer have to jump through extra hoops to include the cookies stored on the Session.

Future Requests updates

As soon as Requests 2.0 is out, we will have a full review here, so be sure to subscribe to our Python tag and The Changelog Weekly for further updates.

Lint against the stdlib and more with PyLint-Brain #

Most of us Pythonistas who have used either NINJA-IDE or PyCharm have benefited from their abilities to lint against the Python standard library. If you then started using tools like PyFlakes or PyLint, you probably longed for that feature. Users of PyLint can now rejoice since Logilab has announced PyLint-Brain. The standard library can be very difficult to lint against and can require some inspection of C extensions, but this is handled gracefully by PyLint-Brain.

Besides adding the ability to lint the standard library, PyLint-Brain will also support some of the most popular third party libraries used by Pythonistas. The list of libraries supported right now is short — just mechanize and qt4 but new libraries are being accepted (like requests) so that list will certainly grow.

As of this moment, you do not need to install Brain separately to use it. It is being provisionally included in PyLint so you will immediately start seeing the benefits.

Check out the source and overview on Bitbucket for more details.

Meet RegExpBuilder: Verbal Expressions’ rich, older cousin #

Following the popularity of Verbal Expressions — a library which generates regular expressions by chaining semantic functions together — Andrew Jones reached out to tell us that he has a library in the same spirit as Verbal Expressions that is a little older and more feature rich. He calls it RegExpBuilder and there are currently versions in JavaScript, Dart, Java, and Python.

I asked Andrew to point out a few differences between Verbal Expressions and his offering. He replied,

RegExpBuilder allows you to specify all kinds of quanitites, whereas VerbalExpressions has focused on just a couple of situations (either it might have something, or it does have something). So in RegExpBuilder you can say “it has between 3 and 100″, or it has “at least 7″ or it has “at most 5″ of something. There are all kinds of ways of specifying quantity. Eg:

var regex = new RegExpBuilder()
  .startOfLine()
  .then("thechangelog is ")
  .max(7).of("really ")
  .then("cool")
  .getRegExp();

regex.test("thechangelog is really really really cool"); // true

and,

VerbalExpressions is mostly limited to working with strings. So for example, you pass in “http” to then(“http”), but unless your regular expression is very linear, you will have to deal with groups of patterns, and work with patterns themselves. RegExpBuilder allows you to pass in and work with patterns themselves, so you can do eitherLike(pattern1).orLike(pattern2), for example. Eg:

var pattern = new RegExpBuilder()
  .either("massively ")
  .or("amazingly ");

var regex = new RegExpBuilder()
  .startOfLine()
  .then("thechangelog is ")
  .max(7).like(pattern)
  .then("cool")
  .getRegExp();

regex.test("thechangelog is massively massively massively cool"); // true

and finally,

RegExpBuilder also has support for character classes (like letters, digits, etc), and advanced features like lookaheads

It’s nice to see some friendly competition in this space to push us all to greater heights.

RegExpBuilder is MIT licensed and hosted on GitHub.

#85: We’re back and we’re LIVE!

Adam Stacoviak, Andrew Thorp, Steve Klabnik, Kenneth Reitz and Jerod Santo take the show live for the first time since August 8th, 2012. Tune in LIVE every Tuesday at 3pm PT / 6pm ET. We’re live every Tuesday! thechangelog.com/live Hack in style with your very own Changelog tee! We are now member supported! Groovy on […]

Detect and highlight your heart-rate using just a webcam and this Python app #

webcam-pulse-detector is a cross-platform Python application that can detect a person’s heart-rate using their computer’s webcam. I could write 1,000 words about it, or just show you this:

webcam-pulse-detector-screenshot

Pretty rad, huh? If you’re wondering how it all works, you’re in luck! There is an entire section of the README dedicated to the topic.

The app depends on Python 2.7+, OpenCV 2.4+, and OpenMDAO 0.5.5+, so it might take some work to get up and running. From the looks of it, it’d be totally worth the effort.

View the project on GitHub.

Bring your server side debug logging into the browser with Chrome Logger #

If you find yourself jumping back and forth between Chrome’s Dev Tools and a terminal displaying your server side request logs, Craig Campbell’s Chrome Logger might be just the thing you need!

It’s a Chrome extension which lets you see your server side logs right in the browser. There are currently libraries for:

Is your server side language/environment of choice not on that list? Don’t worry, Chrome Logger uses an open and published protocol so you can easily write client libraries of your own!

See the project’s home page for more info or check under the hood if you’re curious about how it all works.

Mackup keeps your dotfiles and app settings in sync using Dropbox #

If all you want to do is keep your dotfiles and app settings in sync between machines (OS X machines, of course), then Laurent Raufaste’s Mackup might just be the perfect thing for you.

Basically it symlinks the supported applications to copied versions of your dotfiles and app settings to ~/Dropbox/Mackup.

For example, once you’ve ran mackup backup your .gitconfig file will live at ~/Dropbox/Mackup/.gitconfig and ~/.gitconfig will be symlinked to it. That way all your precious dotfiles and app settings are synced to Dropbox. When it comes time to get another machine (presumably your next machine) setup, it’s as easy as installing Mackup, and running mackup restore.

Laurent’s notes from the readme:

I change my workstation every X months. Each time I either loose the configuration of all the apps I use, or I just waste a bunch of hours getting setup like I was on my old box. I also spent a lot of time reconfiguring the same stuff again on all my workstations (home, work)

Boring…

I agree.

There are a number of ways to backup your dotfiles, but Mackup is super simple, doesn’t require sharing them on GitHub (if that’s not your thing), and leverages Dropbox — which you likely have on your machine anyways.

Check out the install instructions to get started. You might also consider reading this as well to learn exactly how Mackup does what it does. If you haven’t yet, checkout the unofficial guide to dotfiles on GitHub while you’re at it. There’s more than one way to skin this cat.

Simmetrica is a simple-metric-aggregator in Python #

Simmetrica, by Osman Üngür is described in the README as:

A lightweight framework for collecting and aggregating event metrics as timeseries data.

It has two requirements, Python >= 2.6, and Redis. Installation is simple, with git and pip being the preferred method:

git clone git://github.com/import/simmetrica.git
cd simmetrica
pip install -r requirements.txt

After installing Simmetrica, and making sure you have redis running, you can push events with the library, from the command-line, or by using REST:

curl 127.0.0.1:5000/push/add-cart-action

After pushing some data, you can run the HTTP application:

python app.py

Then fire up your browser and visit http://localhost:5000 to see the data in a nicely designed dashboard. Simmetrica has a lot of other features: configuring the dashbard, command-line querying, REST querying, etc. You can view them all at the README or view the source on GitHub.

nginxparser: A Python module that loads and dumps Nginx configs #

Even tiny, single-use open source tools are worthy of our attention. If you have the need to programmatically configure your Nginx servers, look no further than nginxparser by Fatih Erikli.

Weighing in at less than 100 lines of code, nginxparser provides two features.

loading:

from nginxparser import load
load(open("/etc/nginx/sites-enabled/foo.conf"))

    [['server'], [
        ['listen', '80'],
        ['server_name', 'foo.com'],
        ['root', '/home/ubuntu/sites/foo/']]]]

and dumping:

from nginxparser import dumps
dumps([['server'], [
    ['listen', '80'],
    ['server_name', 'foo.com'],
    ['root', '/home/ubuntu/sites/foo/']]])

    'server {
        listen   80;
        server_name foo.com;
        root /home/ubuntu/sites/foo/;
    }'    

I was impressed with how simple it is to define a parser using Pyparsing, which nginxparser does to great effect.

It’s definitely worth checking out!

Export Esri feature classes to open data formats like CSV, JSON, and GeoJSON #

We’ve been covering the topics of open government and open data since nearly the start of The Changelog. More recently, we covered the city of Chicago being on Github and how they open sourced various datasets for civic and commercial usage. Needless to say, the topics open government and open data is something we hackers get excited about.

Wednesday — March 13th (2013), Michael Byrne, Geographic Information Officer for the FCC, announced a new project on Twitter, and welcomes your help to improve it.

From the readme:

Much of the data in government coffers is contained in spatial databases. A large percentage of government spatial data is ESRI software. While the common interchange format, the ESRI Shapefile, is easily exported and imported by many other softwares, this data file format (the Shapefile) is not intrinsically part of the www ecology.

Basically, many government agencies use proprietary software, such as ESRI software, to do their day to day data storage and analysis, but getting that data out and into an open data format like CSV, JSON, or GeoJSON takes some extra effort — esri2open helps to solve this problem.

Check out the project on GitHub to learn more and help improve it!

Flawless catches exceptions and uses git-blame to email the developer who wrote the code #

From John Egan, Growth Hacker at Shopkick, Flawless is a Python framework that catches exceptions and uses git-blame to notify only the developer that wrote the buggy code.

How Flawless works:

The Flawless client wraps your code with a try/except block. When an exception is caught it then sends the entire traceback to the Flawless server. The Flawless server then aggregates exception reports from clients and figures out which line of code caused the exception. Once the line that caused the exception is identified, Flawless runs “git blame” to determine the email address of the developer that last touched that line of code. Flawless then sends the developer an email with the traceback.

Check out the GitHub page for Flawless for more details and setup instructions.

Topaz, a new Ruby written in Python on top of RPython #

From the announcement blog post:

Topaz is written in Python on top of the RPython translation toolchain (the same one that powers PyPy). Its primary goals are simplicity and performance.

Because Topaz builds on RPython, and thus much of the fantastic work of the PyPy developers, it comes out of the box with a high performance garbage collector, and a state of the art JIT (just-in-time) compiler. What does this mean? Out of the box Topaz is extremely fast.

Topaz is far from complete and is missing many builtin methods and classes. However, it does have nearly every element of Ruby, including classes, blocks, many builtin types, all sorts of method calls, and much much more. We don’t yet consider it stable, but it’s getting closer every day.

Check out the source and readme on GitHub to get started.

Give a shout out to Alex Gaynor for his hard work on this.