Thoughts on optimizing sites, traffic, and revenues

March 6, 2004

AOL Leads Search

Well, obviously not. But in the 'Future of Search' session as SES I thought the most interesting ideas came from Gerry Campbell of AOL.

Specifically, he talked about search results that weren't just pointers to related data, but also included data extracted from those pages. So if you did a search for Elvis Costello, the results page would include a photo taken from one web site, a bio taken from another, some CDs and DVDs, recent news etc. The search companies have already shown they can identify images pretty well, and with product feeds flowing and lots of RSS news and info, they could do a lot of this without even scraping HTML.

While these kinds of results are clearly beneficial to users, they'll raise interesting questions for site operators - what is the financial impact of having unique content or valuable data is displayed on composite pages? If it's a taste that gets people to your site, that's great. But if they are satisfied with the composite, you might lose some valuable traffic. And if the engines slowly but surely get better and better at extracting just the valuable bits, and simultaneously improve the targetting, selling, and profit they make off the context they create from the work of others, when do the sites really start opting out (via robots.txt) in droves?

This problem seems like a long way off, but it probably is only a year or so. Already we're seeing weather, news, shopping, and other actual content displayed in standard search results. Add their existing image search results and a few more items and fewer and fewer search results will lead to pages off site, or at least to pages off site that haven't paid for the priviledge.

Gerry made another interesting point, but I can't recall it now. Anyone who can please add it to the comments.

Posted by Craig Danuloff at March 6, 2004 10:52 AM