Google returns more AdSense rich results than Live: conflict of interest?

2008-08-17T12:15:00.000-07:00

Several search pundits have commented on conflicts of interest Google is potentially exposed to. Google itself admitted to not wanting to hold onto Performics, the search engine marketing division of their recent purchase Doubleclick, and in fact they quickly sold it. But that's only one of many conflicts of interest Google has to deal with. John Battelle observes in his searchblog that, in a comparison between Google Ad Planner and Comscore data, sites that use AdSense, Google's advertising platform, get better traffic numbers. If you use the free Ad Planner to plan an ad campaign your money is more likely to flow to Google's coffers than if you were using Comscore. Google provided a statement denying any deliberate bias, but it's hard to find alternative explanations. A similar conflict of interest arises for Quantcast and their Media Planner and rating services w.r.t. being fair to publishers that don't use their tag (they are not quantified in their jargon). They provide ratings and demographic analyses for both, but directly get data only from quantified publishers. Since I am a former employee I will abstain from a more detailed analysis in this case.

Another conflict of interest is between Google the search company and Google the media company: will Google search give prominence to Google properties Picasa, Knol, Blogger and YouTube over the rest of the net? Timo Paloheimo is afraid that could be the case and so he created a custom search engine that excludes such properties, a draconian solution in my opinion but a clear expression of the malaise generated by Google's dominant position in search and the multiple conflicts of interest with other Google products.

But this is only the tip of the iceberg IMHO: the main conflict of interest is between search and advertisment. When companies like Answers.com report that their traffic is down 28% because of a change in their Google ranking, Om Malik rightfully comments that this very fact "raises questions about Google’s power".
Google has an interest in returning the most relevant and useful results in each search; and also has an interest in directing traffic to sites that use adSense because they provide a big part of Google's revenue. How do they balance the two? Google's answer is that the two businesses are run as independently as possible (I forgot the source here, but I am almost sure it was a Google spokesperson). But is that a sufficient answer? Or should we trust Microsoft not to tweak their live search? I think I have strong evidence that Google search results are more likely to carry adSense ads than Microsoft's live.com.

The methodology is as follows:

Sample random search keywords
Use each of them as a query on google.com and live.com
Check if any of the results in the first page uses adSense

I considered the ratio between the number of search matches using adSense and the total number of accessible search results (normally 10, but sometimes some could not be accessed within a reasonable time) for each keyword. Out of 632 searches, Google-provided matches were more adSense rich than live.com ones 316 times, the opposite was true 126 times and the ties were 190. Pretty remarkable, isn't it? If you are not convinced by your intuition or are statistically inclined, applying a Wilcoxon signed rank test rejects the hypothesis of the two search engines results being equally adSense rich with a p-value smaller than 2.2e-16.

Correlation is not causation. It could be that either company is tweaking their engine to favor or penalize adSense; or it could be some other mechanism I haven't figured out yet. But consider this hypothetical scenario: Google doesn't feel it needs to compete so hard in search anymore, wants to boost revenue, decides to give a small but competitively significant rank boost to adSense-running sites, those in turn outcompete their non-adSense-running competitors. We really don't want this to happen. We just recovered from another stranglehold on the computing world.

reCAPTCHA, a bogus motivation

2008-07-30T15:34:00.000-07:00

The authors claim they use spare cycles for a good cause, but it isn't true.

CAPTCHAs are little reading tests that allow to distinguish humans from machines when accessing a computer interface. A word is displayed in a slightly noisy form and the user is required to type it in, easy for humans but not so for machines. The authors of reCAPTCHA wanted to associate a useful task to this test. They say

... in aggregate these little puzzles consume more than 150,000 hours of work each day. What if we could make positive use of this human effort?

So instead of picking random words for the test, they pick word-sized fragment from large public digitization efforts (currently the one led by the Internet Archive) and submit them to the users. That is, reCAPTCHA words are words that are not in digital format, appear in a paper document of interest and machines could not understand. So by solving a reCAPTCHA you are helping a useful project. So where is the catch? Well it took me a moment of thought but it's kind of obvious: if these are fragments of real digitization projects, by definition we don't know the solution. If we don't know it, we can't use them as CAPTCHAs! So I went back to the reCAPTHCA web site, and what they do is to propose two CAPTCHAs, one with a known but useless solution and the other with an unknown but useful one, in a random order. If you want to use a website, you have to complete both. Therefore there is no use of spare cycles whatsoever: the effort for the user has been doubled. If reCAPTCHA were the standard, there would be another 150,000 hours a day spent solving the second word. Fine by me in exchange for a free service, but not as efficient as advertised, not even close.

Map of Science

2008-06-08T23:47:00.000-07:00

Fascinating map of the relation between scientific disciplines. Not all of the methodology is clear, but it is fun to look at. Notice the lack of ties between CSE and Biology.

Map of Science

Piccolblog

Google returns more AdSense rich results than Live: conflict of interest?

reCAPTCHA, a bogus motivation

Map of Science