Tag Archives: R

The Pearson Chi-Squared Test with Python and R

In this post I’ll discuss how to use Python and R to calculate the Pearson Chi-Squared Test for goodness of fit. The chi-squared test for goodness of fit determines how well categorical variables fit some distribution. We assume that the categories are mutually exclusive, and completely cover the sample space. That means that the everything we can think of fits into exactly one category, with no exceptions. For example, suppose we flip a coin to determine if it is fair. The outcomes of this experiment fit into exactly two categories, head and tails. The same goes for rolling a die to determine its fairness; rolls of the die will result in (exactly) one of (exactly) six outcomes or categories. This test is only meaningful with mutually exclusive categories.

Continue reading

Time Series Forecasting in Python and R

A friend recently made a prediction about the price of oil for the next three months. I thought I would perform some time series forecasting on the West Texas Intermediate prices and see if his numbers were reasonable from a dumb-numbers canned-forecasting perspective. I’m not making the claim that one can reasonably and accurately forecast oil prices with traditional time series techniques. (That’s bogus.) I’m simply doing this to learn more about forecasting.

Monthly petroleum prices can be found at the Energy Information Administration. Ever relevant, Wikipedia has a great write-up on recent trends in oil prices. Also, there is this Times article on the spike and drop in 2008 which had this apt summary,

[Oil prices are] the product of an extremely volatile mixture of speculation, oil production, weather, government policies, the global economy, the number of miles the average American is driving in any given week and so on. But the daily price is actually set — or discovered, in economic parlance — on the futures exchange.

Continue reading

Creating a Presentation with R

In this post I’ll look at creating a presentation using the R ecosystem. I’ve used beamer before, and I love it, but I haven’t used the knitr R package yet. Incidentally, the creator of knitr, Yihui Xie, does not like beamer. This is fine, I have been wrong about technology before–I recall thinking in college that facebook was for losers and that it would never catch on. Anyway, Yihui’s work is really impressive and I strongly suggest checking it out.

Continue reading

Updating R from the Command Line

This is a tiny post, but if I lumped it as an aside into a longer post I might never find it again. If you’re trying to keep up with Hadley Wickham you might need to update R from time to time. The installr package is there to help you keep up with the Wickhams. To update R, just follow the following steps:

install.packages("installr");
library(installr);
updateR();

For further infromation, check out this r-statistics post on the topic.

Hypothesis Testing in R

In this post I’ll look at different statistical hypothesis tests in R. Statistical tests can be tricky because they all have different assumptions that must be met before you can use them. Some tests require samples to be normally distributed, others require two samples to have the same variance, while others are not as restrictive.

We’ll begin with testing for normality. Then we’ll look at testing for equality of variance, with and without an assumption of normality. Finally we’ll look at testing for equality of mean, under different assumptions regarding normality and equal variance.

Continue reading