Google re-ranking and personalized search study

Browsers;

1. FireFox (FF) 88.46%
2. Chrome (GCRM) 7.69%
3. Internet Explorer (MS) 3.85%
   

Most used Google app;

1. Gmail (GML) 53.85%
2. GoogleTalk (GT) 11.54%
3. Analytics (GA) 11.54%
4. YouTube (YT) 11.54%
5. Webmaster Tools (WMT) 7.69%
6. Google Reader (GR) 3.85%
7. iGoogle HomePage (iG) 0
   

Cleared Cache lately?

1. Yes (Y) 42.31%
2. No (N) 57.69%
   

Related searches in last 60 days?

1. Yes (Y) 30.77%
2. No (N) 69.23%

Is personalized search on?

1. Yes (Y) 30.77%
2. No (N) 69.23%

Are the top 3 stable?

In the initial task the top 3 placements were rock solid. It was on the secondary queries that extended the task that they would become somewhat instable, (see ‘concurrent tasks’). Now, it wasn’t huge, but there was some small movement that became more prominent with each related query performed.

Ultimately not much has changed as far as aiming for the sweet spot (above the fold). These are easily the most stable listings and it is unlikely these would be affected more than 95% of the time and even then, rare to drop too far down when they do.

Are top 10 stable?

Yes. While the movement always would be found from #4 on down, they did for the most part remain fairly consistent in content if not ranking. As was noted with the top 3, the more tasks in the chain (queries) the more instability would creep in.

Where is the biggest re-rankings?

As noted so far, the biggest area of flux was on the expanded task elements. The first query ran, in most cases, was fairly stable. It was on further related queries in the session that higher levels of instability were noted.

We’re considering running the test again while reversing the order to see if indeed the order of the session queries created the increased movement the further we went into the process. It does seem like this is an important element to watch for in future testing.

What causes SERP flux?

I really have no conclusions at this point. We’ve likely not hit a trigger often enough to see any deeper personalization in play. I believe the next round, using more familiar queries and of the informational nature, there could be stronger Psearch elements present (as there were in the first round, late 08).

For the moment, in case you hadn’t guessed, the actual ongoing task development seems to be an important element. Much of the more gradual movement could easily be attributed to various servers the locales were hitting at the time. Many of the other elements we looked at (related searches, Psearch on/off) didn’t really seem to have a strong bearing in the SERP flux.

My guess? Query analysis will play as important a role as the Psearch does. There does seem to be a task related SERP adaptation from what we’re seeing in this round. The jury is out though until we know more.

What do strong listings have in common?

Certainly the money listing is #1-3 as these seem to be the most resistant to movement. What was also interesting is that the odd 8th place or 5th even would remain more solid than some above and below them.

Indented listings would often be vulnerable but other than that, the higher the ranking, the more security one would have. Beyond that we’d need to actually dig into each of the sites for further clues.

Is related searches past 60 days making a difference?

Not any more than the avg. general flux. There doesn’t seem to be a huge difference or much of one at all from those with recent related searches to those without. I thought this was surprising as there was some expectation of this playing into SERP flux… It didn’t seem to make much of a difference ultimately.

Are concurrent tasks creating a difference?

This is the most active and interesting area that we’re seeing. All three queries the respondents ran for us were related to the same task. We were essentially building a task towards a goal. It seems that with each concurrent search, we’d see more and more flux.

Are browsers making a difference?

We’re seeing some flux in IE, FF and Chrome. Of interest is that the Chrome SERP was a model SERP (consistency wise). I really can’t see this as being a major element, but we would probably need more data for a conclusive decision.

So far, I’d say that this doesn’t play into it….or at least isn’t much of a signal of note.

Rankings are dead? – Yes and no. The old days of ranking reports telling us one ranks in X-position, seem to not be a valid approach. Should we be tracking/watching rankings? Most certainly we want to be getting top 10 and then above the fold to ensure the best performance.

I’d also suggest that you might occasionally get people in a targeted region to check SERPs for you now and again.

Everyone sees a different SERP? – Yes and no. The rankings are reasonably consistent as far as the make up of the top 10. Even the movement isn’t massive when there is some.

But are they ALL the same… most certainly not.

• Informational – next round we’re returning to the original informational query which is in a more familiar query space which is likely to produce some different effects.

• Navigational – we may also consider running a round on a navigational query space to see how it reacts. I am more interested in the informational/transactional spaces, so time permitting on this one.

• Reverse order – if the trend continues with concurrent searches (task development) we will want to reverse the order on one to see if that remains the case

• Click data – obviously once we get past all of that, we are likely to start introducing click data into the mix to see what effect that has on things.

• Temporal watching – another point brought to me by a information retriever was that any changes to Google’s processing would likely be incremental. This means we’ll re-test in 6 mo to see if things have changed.