SEO Blog - Internet marketing news and views  

Google Insights for Search updates

Written by David Harry   
Tuesday, 18 August 2009 08:19

Hey there…. Just a few tidbits in passing that are worth mentioning...Yesterday there was some interesting news in the form of an update to Google Insights for Search, you can get some coverage here;

New features and languages for Google Insights for Search - Official Google Blog
Google Adds Keyword Forecasting To Insights For Search Tool – Search Engine Land


Essentially they got some toys recently (embed and gadget even) as well as a new feature for evaluating query space predictions via Keyword Forecasting, (for selected categories). That’s also clever and comes with a short video;

OK… now that we’re up to speed…Let’s git outta’ dodge an’ get geeky.


Some interesting research on I4S

As it turns out the gang over at the Google Research Blog, were also getting in on the action. Not only did they get more into the details, but released a formidable paper into the wild as well titled; On the Predictability of Search Trends (PDF)

This document is 32 pages of fun for search geeks and tin foil hatters alike! It is essentially the report on the research that went into the decision making processes and adaptations made to the system..


They mention;

In conjunction with this study, a basic forecasting capability was introduced into Google
Insights for Search, which provides forecasting for trends that are identified as predictable.
Researchers, marketers, journalists, and others, can use I4S to get a wide picture on search
trends which now also includes predictability of single queries and aggregated categories in any area of interest.

Which sums up the thoughts behind working with predictability measures inside of I4S. Much of the paper deals with systemic elements, error checking and methodologies, but there are some interesting insights to be had hear as well.


In the blog post they summarized the observations as;

  • Over half of the most popular Google search queries are predictable in a 12 month ahead forecast, with a mean absolute prediction error of about 12%.
  • Nearly half of the most popular queries are not predictable (with respect to the model we have used).
  • Some categories have particularly high fraction of predictable queries; for instance, Health (74%), Food & Drink (67%) and Travel (65%).
  • Some categories have particularly low fraction of predictable queries; for instance, Entertainment (35%) and Social Networks & Online Communities (27%).
  • The trends of aggregated queries per categories are much more predictable: 88% of the aggregated category search trends of over 600 categories in Insights for Search are predictable, with a mean absolute prediction error of of less than 6%.
  • There is a clear association between the existence of seasonality patterns and higher predictability, as well as an association between high levels of outliers and lower predictability. For the Entertainment category that has typically less seasonal search behavior as well as relatively higher number of singular spikes of interest, we have seen a predictability of 35%, where as the category of Travel with a very seasonal behavior and lower tendency for short spikes of interest had a predictability of 65%.
  • One should expect the actual search trends to deviate from forecast for many predictable queries, due to possible events and dynamic circumstances.
  • We show the forecasting v actual for trends of a few categories, including some that were used recently for predicting the present of various economic indicators. This demonstrates how forecasting can serve as a good baseline for identifying interesting deviations in actual search traffic.


They go on to discuss in the paper about various implementations (including watching recession related query spaces and the effects) and the prediction layer. It should be noted that the research was done on a monthly basis over the period of 2004-09. Essentially, at this point, more granular results would require a more complex solution and have been left as something to work on… Still, I think the data can be interesting and hopefully a sign of things to come with greater expansion (in categories/terms and granularity).


If you have some time… be sure to give it a read


So how’s it work?

Well, kinda’ hit and miss at the moment. It seems that trends over the last 18 months or more seem to have some problems.. I ran a bunch that had this problem, but this one highlights the problem as 'social media' doesn't have a predictive measure;

(you have to click through, the 'predictions' don't carry through with the embed code...duh)

It would seem that one needs more long term categories at this point. So, for those ready to start screaking ‘real time search’ – whoa back, grab some wood! As they mentioned in the research paper, they hope to expand the system and we’ll have to wait and see on the ‘freshness’ of the predictive elements.

OK, no probs… so why not have some fun and search a category and leave the search term box empty;

Also of interest are the rising searches and top searches below the main graph. And not only can you embed data as we did above (which kinda sucks), but you can also send over to your iGoogle homepage should you be so inclined (I have to wait and see to useful the system/data is, I put a few up).


A viable KW research tool still

All things considered it is an interesting twist, just not entirely as kick ass as I’d like to see. I look forward to more granular temporal additions which will make I4S a more reactive predictor. From what we can see in the research paper there are certainly some great ways this tool will continue to evolve (that’s MY prediction ...hehe). If you haven’t already been playing around with it for keyword research and supplementary market data, there are a few more reasons to do so.

I would also, as always, caution that it is still an imperfect system and they admit to certain levels of errors sneaking in. But we all know to get a variety of data points right? Sure.. silly me. Anyway, if you’re into some god geeky reading, give the paper a go…


Here are some of the related resources listed;

[Askitas and Zimmerman 2009] Nikos Askitas and Kalus F. Zimmerman.
Google econometrics and unemployment forecasting. Applied Economics Quarterly,

[Choi and Varian 2009] Hyunyoung Choi and Hal Varian.
Predicting the present with google trends. Technical report, Google, 2009.

[Ginsberg etal. 2009] Jeremy Ginsberg, Matthew H. Mohebbi, Rajan S. Patel,
Lynnette Brammer, Mark S. Smolinski & Larry Brilliant.
Detecting influenza epidemics using search engine query data. Nature 457,

[Suhoy 2009] Tanya Suhoy.
Query indices and a 2008 downturn: Israeli data. Tech. Report, Bank of Israel, 2009


Search the Site

SEO Training

Tools of the Trade


On Twitter

Follow me on Twitter

Site Designed by Verve Developments.