I've started three thematic microblogs you may be interested in.
Sunday, May 05, 2013
Thursday, October 18, 2012
Thursday, May 10, 2012
Saturday, December 03, 2011
Thursday, September 15, 2011
My new implementation of random mate for mapreduce, using the package rmr from Revolution Analytics open source project RHadoop.
Posted by Antonio Piccolboni on Thursday, September 15, 2011
Wednesday, April 27, 2011
At long last, a complete implementation of the algorithm I described some time ago.
Posted by Antonio Piccolboni on Wednesday, April 27, 2011
Friday, April 15, 2011
Monday, April 11, 2011
Thursday, April 07, 2011
Monday, November 29, 2010
From a job interview challenge, an interesting probability exercise in two parts. One of the themes here is pretty standard fare. You are given a clearly defined random procedure whose outcome is a mixture of two distributions. The problem is, given a certain set of outcomes, find which of the two distributions it is coming from. For instance, imagine you have to assign one of two classes to an item based on repeated noisy measurements and you know the relative size of the two classes (a priori probability of belonging to one of the two). The second part of the challenge is a bit more interesting but also eccentric. It is asking for a best case outcome that would make it easiest (smallest sample) to detect the class of the item with a certain error probability. I am not aware of any practical statistical question where such a best case problem arises, even if we consider the converse, the worst case outcome. But being used to worst case analysis from my CS training, I came up with an optimality proof based on induction and manipulation of binomial coefficients, which confirms the intuition that a very unlikely, extreme outcome is the best one. The main idea is that when lower bounding an expression including binomial coefficients, it is somehow easier to prove a tight lower bound because the binomial coefficients on the two sides of the inequality are very similar in that case and one can simplify a lot and then use simple algebra. It won't set the world of Mathematics abuzz, but it seemed interesting enough to share.
Posted by Antonio Piccolboni on Monday, November 29, 2010