Link Header Tag to Fix Internal Website Duplication in Google, Yahoo, and MSN

Submitted by Brian Gilley on February 13, 2009 - 18:25.

Search Engines

Internal website duplication has been one of the more common issues with client SEO audit reports. Blogs, ecommerce websites, and any website where identical, or close to identical, content is found on one or multiple pages can benefit from the new <link> tag to help solve duplication problems. Here's a quick example of how this new tag works.

Let's say you have an ecommerce website with sorting capabilities for product sizes, colors, etc. The same content might appear on these three different URLs:

123example.com/products?itemid=bluewidget
123example.com/products?itemid=bluewidget&sort=123
123example.com/products/bluewidget

Now, you can simply add the <link> tag to specify your preferred version such as:

<link rel="canonical" href="hxxp://www.123example.com/products/bluewidget" />. You would place this inside the <head> section of all of the duplicate content URLs, such as:

123example.com/products?itemid=bluewidget; and
123example.com/products?itemid=bluewidget&sort=123

Google will understand that the duplicates all refer to only the canonical URL: 123example.com/products/bluewidget. Further, any additional URL properties, such as PageRank and related values, are transferred as well.

Prior to this tag the only real way to ensure that internal duplication problems are not occurring within your website was to add a META noindex, nofollow or noindex, follow to possible duplicated pages.

You can read more about implementing the <link> tag at Yahoo, Google, and MSN. If you're using Drupal, WordPress, or Magento as your CMS, you can also check out Joost de Valk's page on canonical links. He has included a plugin or module to add the <links> tag within your Drupal, WP, or Magento website.

Comments

very intresting article some

very intresting article some thing to take for our SEO team

Submitted by Joomla Development (not verified) on February 27, 2009 - 11:09.

I had recently been

I had recently been wondering about this, would this work for pages that are masked using .htaccess too?

Submitted by BenSky (not verified) on March 8, 2009 - 09:46.

Great Tag

This is a great tag for online store. For my blogs, I often use the features of the SEO All In One Plugin which also allow you to prevent duplicate content. Great info. Thanks, Richard

Submitted by Richard Cummings (not verified) on March 17, 2009 - 12:12.

Excellent Article

Hey Brian,

Thanks for the wonderful information. This would surely save me some sleepless nights in trying to figure out what to with the problem you have pointed out here :)

But what are your thoughts on URL Mode Re-writing / 301 Redirection. Cause, I faced a similar problem with one of my projects and we used the above two techniques to over come it.

The solution provided by you is surely much more simple to understand and implement and addresses the reasons for which we used both the above techniques. But still wanted to know your thoughts on the above.

Submitted by Rahul (not verified) on March 21, 2009 - 07:07.

Just twitted about this post

Just twitted about this post :) ..

Submitted by Rahul (not verified) on March 21, 2009 - 07:12.

solution

permanent solution can be achieved by changing the secure pages linking strategy to “Absolute Referencing” and/or implement a meta robots tag to nofollow and noindex the secure pages in question.

Submitted by Richard (not verified) on April 16, 2009 - 16:21.

Great Solution

Hi.. I had experience real problems in ecommerce sites having almost similar content in more than one pages now I feel with this tag one can control and tell Google that they are duplicate and only index one of them.. thanks for the interesting post

Submitted by Iphone battery (not verified) on May 26, 2009 - 11:06.

Awesome

I have posted a lot of landing pages for our site for each city and state. I have a template they are all pulled from in an xslt format. I am going to have to play with this one. Thanks for the awesome post!

Submitted by Randy (not verified) on June 9, 2009 - 05:38.

Area Pages

This is great as when I have a site that needs optimising for many areas I build area pages which is your main key words mixed with the different arrears, you kind of need to write the same script over and over with of slight variations in text to avoid duplicate content.

Submitted by Anonymous (not verified) on July 6, 2009 - 16:46.