Using find and xargs

In regular linux or unix you can recursively find the files in a subdirectory and apply some utility with arguments on them using xargs as,

find <dir> -type f | xargs <utilitiy> <args>

If the filenames in <dir> have spaces, quotation marks, or other characters in their filenames that make xargs barf, then you can use the following.

find <dir> -type f print0 | xargs -0 <utility> <args>

That should totally work–unless you’re on SunOS, then you have to do,

find <dir> -type f -exec <utility> <args> {} +

It looks crazy, I know, but that’s the honest truth.

Using Optional Values in Swift

Optional Values

Swift has a nil keyword that is sort of like the None keyword in Python; however, regular variables and constants cannot be nil. In order for a variable to be nil, it must be initialized as an optional. (Constants cannot be nil or optional.) An optional is decalred by appending a question mark the variable type. For example, the two statements below are equivalent.

Continue reading Using Optional Values in Swift

Installing Canvas for Node on OSX

I’d like to work with Processing.js, but I had trouble installing the canvas dependency. This should be as easy as,

$ npm install canvas

But there’s a thingy that doesn’t work in Homebrew. I found the answer in this thread on their GitHub. On OSX you have to call this command before you can install canvas:

$ export PKG_CONFIG_PATH=/opt/X11/lib/pkgconfig

Then we can install canvas and processing using npm like we’d expect.

Return Distinct Elements in a Field of a MongoDB Collection

I’ve been working with MongoDB, and I noticed that querying the unique values in a field of a given collection follows a non-idiomatic pattern. That is to say, it does not look like,

db.collection.find( { "Field": { $unique: 1 } } )

Rather, it looks like this,

db.runCommand( { distinct: <collection>, key: <field> } )

And this returns a document with a "values" and "stats" field, and hopefully an "ok":1 field. This is the documentation.

Using Node and Mongo to Collect Data from Reddit

I am taking a course on MongoDB development with Node.js from Mongo University. In the second week we covered a thing that I thought was very interesting. They walked you through how to grab the JSON data out of a Reddit page. Reddit apparently offers its data up as a JSON if you pass it a .json path. Here is the coffeescript that produces the code provided in the development course.

Continue reading Using Node and Mongo to Collect Data from Reddit

Blum Mental Hash with Node and Angular

In this post I’ll walk through building a simple Blum Mental Hash application using Node and Angular. I’ll write my Node and Angular code using CoffeeScript, and I’ll build my HTML templates with Jade.

Writing the Application in CoffeeScript

I really enjoy CoffeeScript. I get that real programmers don’t use CoffeeScript, but I don’t care. Javascript makes me crazy. This is my CoffeeScript code–I require a bunch of stuff: express, http, jade. Then I define an express object named app and set some settings on it. Then I tell the root path, '/', to use the './static' subdirectory on the server. Then I tell any GET requests to '/' to return HTML that’s been rendered by jade using my index.jade template.

#!/usr/bin/env node
# app.coffee

express = require "express"
http = require "http"
jade = require "jade"

app = express()

app.set( 'views', './static' )
app.set( 'view engine', 'jade' )
app.engine( 'jade', jade.__express )

app.use '/', express.static('./static')

app.get '/', ( req, res ) ->
    res.render( 'index.jade' )

server = app.listen 8000, ->
    host = server.address().address
    port = server.address().port
    console.log 'Example at http://%s%s', host, port

We can compile this guy at the command line by calling,

 
coffee -c app.coffee

This will create an app.js file in your working directory.

The Controller

The file contoller.coffee sits in the subdirectory ./static/js/. The first items, mapping and perm, should ideally be picked by the user. In this example I’ve hard coded them for convenience. The convert() function brings everything together; it combines mapping and perm in such a way as to hash a string of capital letters. From there myapp.controller can compute the hash for the user.

#!/usr/bin/env node
# ./static/js/controller.coffee

mapping =
    A: 4,   B: 9,   C: 4
    D: 5,   E: 8,   F: 5
    G: 7,   H: 1,   I: 8
    J: 3,   K: 6,   L: 8
    M: 9,   N: 0,   O: 9
    P: 0,   Q: 6,   R: 1
    S: 9,   T: 6,   U: 4
    V: 7,   W: 5,   X: 0
    Y: 9,   Z: 3

f = ( x, mapping ) ->
    ###
    Maps a string `x` to an array of digits
    ###
    a = []
    for i in x
        a.push( mapping[i] )
    return a

perm = [ 8, 2, 5, 4, 9, 0, 3, 7, 1, 6 ]

permute = ( i, perm ) ->
    ###
    Returns the next value in the permutation
    ###
    idx = perm.indexOf( i )
    idx = ( idx + 1 ) % 10
    return perm[ idx ]

g = ( a, perm ) ->
    ###
    Hashes a list of digits into a string of digits
    ###
    b = [ permute( ( a[0] + a[ a.length-1 ] ) % 10, perm ) ]
    for i in [1...a.length]
        b.push( permute( ( b[i-1] + a[i] ) % 10, perm ) )
    return b.join("")

convert = ( x ) ->
    ###
    Convert a string of characters to a hashed string of digits
    ###
    a = f( x, mapping )
    return g( a, perm )

myapp = angular.module( "myapp", [] )

myapp.controller "controller", ($scope) ->
    $scope.plain = ""
    $scope.b = ""
    $scope.update = ( plain ) ->
        $scope.b = convert( plain )

This CoffeeScript file should also be compiled into a Javascript file.

The View

The index.jade file lives in ./static. This prompts a user for a password, and then returns a hash of that password below. That’s it.

doctype html
html
  head
    script(src="http://ajax.googleapis.com/ajax/libs/angularjs/1.2.5/angular.min.js")
    script(src="/js/controller.js")
    link(rel="stylesheet",href="css/style.css")
  body(ng-app="myapp")
    h1 Manuel Blum's Mental Hash (BMH)
    div(ng-controller="controller")
      |Password: 
      input(name="Password: ",type="text",ng-model="plain")
      button(ng-click="update(plain)") Convert
      p Hash: {{ b }}

We can serve the page by calling

node app.js

from the command line, and then navigating to localhost:5000.

The Pearson Chi-Squared Test with Python and R

In this post I’ll discuss how to use Python and R to calculate the Pearson Chi-Squared Test for goodness of fit. The chi-squared test for goodness of fit determines how well categorical variables fit some distribution. We assume that the categories are mutually exclusive, and completely cover the sample space. That means that the everything we can think of fits into exactly one category, with no exceptions. For example, suppose we flip a coin to determine if it is fair. The outcomes of this experiment fit into exactly two categories, head and tails. The same goes for rolling a die to determine its fairness; rolls of the die will result in (exactly) one of (exactly) six outcomes or categories. This test is only meaningful with mutually exclusive categories.

Continue reading The Pearson Chi-Squared Test with Python and R