One of my favorite things about pandas is that you can easily combine temporal data sets using different time scales. Behind the scenes, pandas will fill in the empty gaps with null values, and then quietly ignore those null values when you want to make a scatter plot or do some other computation, like a rolling mean. It takes so much tedious book-keeping out of the data analysis process.
I explained in a previous post how to quickly and easily grab data from Excel files using pandas. I use this approach when I know that there’s a ton of data in the Excel file and I want it in a pandas DataFrame. If I am only extracting a handful of values, I like to use a lower level module,
In this post I will discuss an implementation of sequential Gaussian simulation (SGS) from the field of geostatistics. Geostatistics is simply a statistical consideration of spatially distributed data. Sequential Gaussian simulation is a technique used to “fill in” a grid representing the area of interest using a smattering of observations, and a model of the observed trend. The basic workflow incorporates three steps:
- Modeling the measured variation using a semivariogram
- Using the semivariogram to perform interpolation by kriging
- Running simulations to estimate the spatial distribution of the variable(s) of interest
In this post, I’ll describe how to change the color of an anode RGB LED with a potentiometer. I’ll be using an Arduino UNO, and components from this RadioShack components kit. The motivation for this post was to have an LED change color in response to the reading from a thermistor next to my stove, but when I read about how I’d first need to calibrate the thermistor with some kind of thermometer, my motivation scurried under the sofa like a terrier in a thunderstorm. As a compromise I substituted the thermistor with a trim-pot, reasoning that a variable resistance was a variable resistance.
In this post I’ll describe how to get started using gonum/matrix package for using matrices for math and stats applications. (Documentation here.) I’ll begin with a bit about setting up the Go environment drawn from the How to Write Code page on the Go website. (I highly recommend reading this if you’re unfamiliar with Go.) Next I’ll provide a commented usage case.
In this post I’ll present the z-score forward and backward transforms used in Sequential Gaussian Simulation, to be discussed at a later date. Some geostatistical algorithms assume that data is distributed normally, but interesting data is generally never normally distributed? Solution: force normality, or quasi-normality. All of this is loosely based on Clayton V. Deutsche’s work on the GSLIB library, and his books.
In this post I’ll discuss the basics of walking through a directory tree in Python and Go. If you are dealing with a smaller directory, it may be more convenient to use Python. If you are dealing with a larger directory containing hundreds of subdirectories and thousands of files, you may want to look into using Go, or another compiled language. I enjoy using Go because it compiles quickly, and it doesn’t use pointer arithmetic.
In this post I’ll demonstrate an iterative closest point (ICP) algorithm that works reasonably well. An ICP algorithm seeks to find a transformation between two sets of points that minimizes the error between them, i.e., you are trying to find a transformation that will lay one set of points exactly on top of another.
In this post I’ll present a solution to a puzzle using Python. I think the primary value of this post is that it provides an example of how to translate an objective and a set of constraints into data structures and functions that can be interpreted by a computer. This problem breaks down into two interrelated parts:
- Translate the problem into data structures and functions
- Choose a strategy for finding the solution
If you create a bootable USB flash drive for installing linux on a machines, you may have trouble reformatting it later in Windows. It is possible to create a bootable USB in such a way that the Windows reformatting utility, obtained by right-clicking on the drive and selecting Format…, does not see the partition containing the bootloader. In such a situation, you may have a 8GB drive, but the reformatting utility only sees 6GB, and reformatting will not recover the original 8GB of space.