Smart Web Center - WordPress Resources
Home >>  WordPress Resources >>  Duplicate Content in WordPress

Duplicate Content in WordPress

WordPress is an excellent blog application, but like most blogs it creates duplicate content that some search engines will find and penalize. This is because WordPress has many features for blogging and linking, such as RSS feeds, trackback URLs, and monthly archive which display the same content on different pages but for different purposes.

Duplicate Content in the WordPress Sidebar

Categories in WordPress are actually duplicates of your individual posts. In the sidebar, the content is sorted by date of post, again in monthly articles, and again in your daily pages. The headers are different, but that's all.

Duplicate Content in the WordPress Home Page

In most default installations of WordPress, the home page is also a duplicate of your individual posts.

Duplicate Content in Word Press Feeds

Search Engines seem to have a mind of their own and all too often will choose your RSS feed to display in search results. This is a problem, because RSS feeds are not human readable.

Duplicate Content in Word Press Trackback URLS

Many WordPress templates add trackback links after posts to make it easy for you track who is linking to your posts.

Typically, if your post URL looks like this: ‘www.yoursite.com/2007-05-23/yourpost/’ its trackback URL will be ‘www.yoursite.com/2007-05-23/yourpost/trackback/’ and may be treated as duplicate content.

Duplicate Content in WordPress Meta Tags

The default WordPress installation doesn't have a tool to add unique meta tags to your individual posts. Your posts will either have no meta tags or site wide metatags that are the same for every page on your blog. Google is likely filter out your pages as being similar based on the metatags or place your results in supplemental search results.

One of the users on the Webmaster World Forums tested this: Read the Test Results

Duplicate Content Solutions for WordPress

You may want to implement some of these solutions to improve your blogs search engine results.

Add NoIndex and Follow Meta Tags

You can tell the search engines what URLs to use by using the ‘noindex, follow’ meta tag.

To do this you have to add the following code to the header section of your header.php page:

if((is_home() && ($paged < 2 )) || is_single() || is_page() || is_category()){
echo '<meta name="robots" content="index,follow" />';
} else {
echo '<meta name="robots" content="noindex,follow" />';}
This code tells the search engine crawlers to index the home page, individual posts, solo pages like "About Me" and category pages and not index any other pages. If you don't want the category pages indexed, delete || is_category(). The search engine crawler will still crawl the page lookng for outbound links, but it won't add the pages themselves to the search engine results.

Add Unique Meta Tag Descriptions

You will need a plugin to add meta tag descriptions. Here is one that has good reviews from people who have used it:

Add-Meta-Tags WordPress Plugin

Add Redirects to Your Blog

Computers are extremely literal and will see www.yoursite.com as a completely different address from yoursite.com whereas people see these as the same. You have to tell the computer that they are the same using what is called a 301 Redirect.

To use the redirects, you edit your .htaccess file and add the following:

RewriteEngine On
RewriteCond %{HTTP_HOST} !^www\.yoursite\.com$ [NC]
RewriteRule ^(.*)$ http://www.yoursite.com/$1 [R,L]
RewriteBase /
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule . /index.php [L]


For more technical details, visit: URL Rewriting Guide

Edit Your Robot.txt File:

You can prevent search engine crawlers and spiders from visiting your feeds and supplemental pages by inserting the following in your robots.txt file:

User-agent: *
Disallow: /wp-
Disallow: /search
Disallow: /feed
Disallow: /comments/feed
Disallow: /feed/$
Disallow: /*/feed/$
Disallow: /*/feed/rss/$
Disallow: /*/trackback/$
Disallow: /*/*/feed/$
Disallow: /*/*/feed/rss/$
Disallow: /*/*/trackback/$
Disallow: /*/*/*/feed/$
Disallow: /*/*/*/feed/rss/$
Disallow: /*/*/*/trackback/$


You can also try using this plugin which does this work for you: Permalink Redirect

Restrict the Number of Posts Showing on Your Home Page

If you reduce the number of posts displayed on your home page to no more than four or five, you have less chance of content being treated as duplicate content in WordPress.

You can learn how to do this by reading this article: Customizing the More Tag in Word Press

More Information and References: