Similar Posts

Caveat

Unfortunately, due to ill-health, this plugin has not been developed or supported properly for some years. It works with the latest versions of WordPress (including on this website) but could possibly conflict with any WordPress features added after 2008 — e.g. custom post types — if you use them.

Purpose

This plugin displays a list of posts which are related or similar to the current post.

This is version 2.6.2.0 download latest version. It is compatible with WordPress 1.5–2.6.2.

  • 2.6.2.0 fixes a problem with stemming and stop words and offers a new fuzzy matching capability; supplies a .pot file making internationalisation possible; introduces the {imagealt} output tag and allows {excerpt} to output whole sentences; the content filter and the widget can now take a parameter string; output can be automatically placed after post content without editing theme files.
  • 2.6.1.2 fixes the German-language stemmer which should have been encoded as utf8.
  • 2.6.1.1 fixes the Italian-language stemmer which was crashing under PHP4.
  • 2.6.1.0 allows the current post to be marked manually where the automatic mechanism fails; when used as a widget the plugin now honours the setting to show nothing when there is no output; {commenterlink} now applies the appropriate WordPress filter; and fixed a problem with some installations not finding the right language files.
  • 2.6.0.1 fixes the option to include attachments and adds a parameter to the {imagesrc} output tag to append a suffix to the image name.
  • 2.5.0.11 has a new option to include posts which are attachments; a new output template tag {authorurl} which points to the archive of the author’s posts; new behaviour for the {php} output tag which can now accept other output tags in the code; and includes a fix for MySQL problems in some locales’.
  • 2.5.0.10 provides the ability to select from two algorithms for term extraction; allows you to specify post relationships by hand; and fixes a problem indexing tags in some languages.
  • 2.5.0.9 adds an option to match the current post’s author and extends the options for snippet and excerpt output tags to make the ‘more’ text into a link.
  • 2.5.0.8 adds an option to show posts by status, i.e., published/private/draft/future, changes the {categorynames} and {categorylinks} output tags by applying the ‘single_cat_name’ filter, and fixes a bug in WordPress pre-2.2 that stopped installation code running on Windows servers.
  • 2.5.0 improves the CJK matching algorithm by using bigrams. Also introduces a new output tag {imagesrc}, and adds more parameters to {image}. Fixes bugs with empty categories, excluded posts, and the option to omit current posts.
  • 2.5b28 improves the matching algorithm and adds an experimental mode for blogs in Chinese, Korean, or Japanese.
  • 2.5b27 fixes a bug with the bulk indexing of tags.
  • 2.5b25 makes some important changes: the {image} output tag now serves real thumbnails (couple of bug fixes too); output can now be sorted as you choose with sub-headings included; the {date:raw} tag modifier has been added to help the sorting; the ‘trim_before’ option has been replaced with the more logical ‘divider’.
  • 2.5b24 fixes to stop recursive replacement by content filter, {gravatar} to allow for ‘identicon’ etc., to {commenter} to allow trimming, and to remove a warning in safe mode
  • 2.5b23 brings a new option to filter on custom fields and adds proper nesting of braces in {if}.
  • 2.5b22 moves the manage menu under settings as a subpage, restores automatic indexing on activation, fixes conflicts with the legacy Similar Posts Feed plugin, fixes bugs in several output tags, and introduces the option to show only pages.
  • 2.5b20 doubles the speed of indexing and reduces the memory footprint considerably.
  • 2.5b19 fixes a bug when snippets are stripped of extra tags.
  • 2.5b18 fixes a problem with filtering the output and introduces the conditional tag {if:condition:yes:no}.
  • 2.5b16 fixes a problem with {php}.
  • 2.5b15 fixes for some more installation problems, one or two bugs, and adds the ‘included posts’ setting.
  • 2.5b14 fixes for some of the kinds of installation problems.
  • 2.5b11 fixes some widget problems.
  • 2.5b10 fixes (some?) of the problems folks have been having with no posts found. Most of such errors seem to arise when the proper table is not created and this version addresses that.
  • 2.5b9 has new features and improvements.
  • 2.3.6 restores the widgetiness I managed to remove in 2.3.5!
  • 2.3.5 has been rebuilt to save memory and can match the current post’s tags. It also fixes a bug with categories in WordPress < 2.3.
  • 2.3.4 now works as a widget.
  • 2.3.3 beta adds the ability to include as well as exclude categories and authors and is able to find posts by tag.
  • 2.3.2 beta fixes a conflict between tags and categories.
  • 2.3.1 beta fixes a stupid bug in category exclusion.
  • 2.3.0 beta is compatible with WP 2.3, fixes the {author} bug, and a number of problems related to versions of MySQL.
  • 2.1.1 beta fixes a badly chosen fallback value for the number of terms used to match similar posts.

Ideally, similarity or relatedness would be based on a post’s meaning. Tagging systems try to add meaning after the fact but suffer from two deficiencies, one practical and the other theoretical. When a blog already has many posts it can be impractical to retrofit a tagging system by tagging every post by hand. ‘Automatic’ services, like Yahoo’s, tend to produce too many suggestions which need to be culled, again by hand.

The theoretical problem with tagging is that it tries to pin down a meaning for a post by categorising it under a small number of types, whether those types belong to a predetermined hierarchy or arise by ‘folk’ classification. In fact, a post has a variety of meanings, a multitude of ways it can be related to other posts. Meaning doesn’t just lie in the intention of the author or in the classification of the reader; meaning also inhabits the text itself. Meaning is in the words.

The Similar Posts plugin compares posts by comparing their words. MySQL has a sophisticated full-text searching facility with a carefully tuned algorithm for judging the similarity between texts. Similar Posts extracts representative words from a post’s content, title, and tags and uses the full-text index to find the best matches between posts. This simple approach gives surprisingly good results.

The results can be tweaked in several ways to tailor them for you blog. By default the plugin chooses the 20 most frequent words to make its matches but the number is adjustable. It is worth experimenting to see how many words gives the best results for your blog — it has hardly any impact on speed, even if you set the value high enough to include the whole post. The relative importance given to words in your title may be adjusted so that well-chosen titles can be used to advantage or titles with little relevance downplayed. Similarly, tags can be used to improve matching or not according to your blog and it’s needs.

It is also possible to override the automatic similarity ranking by using a custom field. In the post edit screen create a custom field called ‘sp_similar’ with the ID value of the post to which you wish to ‘link’. You can link to multiple posts by entering a comma-delimited list of IDs.

The plugin has a settings page which lets you change how the output is generated and displayed. There is also a management page where you can change settings which affect the index.

Note: Similar Posts needs to know the ID of the post for which it is generating related posts. WordPress keeps track of that information in a global variable but unfortunately some other plugins can corrupt the data before Similar Posts gets a chance to use it. Similar Posts tries various tricks to get round this but sometimes it fails. The usual symptom is a list of similar posts that stays the same from page to page. You can help Similar Posts out by marking the current post manually by adding a line to your theme files. Find the place where the_content(); is used to display the current post and right after it put similar_posts_mark_current();.

Installation Instructions

  1. If upgrading from a previous version, first deactivate the plugin via the Plugins page and delete the plugin folder from your server.
  2. If you have been using the Similar Posts Feed plugin you should deactivate it as it is now obsolete.
  3. Upload the plugin folder to your /wp-content/plugins/ directory. You will also need to install the Post-Plugin Library.
  4. Go to your admin Plugins page and activate Similar Posts. This will automatically add a new table to enable fast, flexible full-text matching. If the plugin reports that there was a problem creating the table first try deactivating and reactivating the plugin.
  5. Put<!--?php similar_posts(); ?--> at the place in your theme files where you want the list of similar posts to appear. Lorelle on WordPress has a good guide to modifying themes for plugins.If you are averse to editing template files you can also place the post listing automatically either as a widget in the sidebar of your widget-aware theme or after each post (from the plugin’s Placement submenu).
  6. Use the admin Settings|Similar Posts pages to set all the available options. Alternatively, the options can be overridden by passing a parameter to the similar_posts template tag.

Usage and Options

The configuration page will help you to set up the plugin to your satisfaction.

The Index Management Page

Using this settings subpage you can re-index your blog. There are two main settings which affect the indexing.

PHP is, by default, not very good at handling text that isn’t in English and you might find Similar Posts mangles extended characters. If so, you can get the plugin to use PHP multi-byte string library if it is available.

The second setting attempts to handle words with related meanings. For example, ‘animal’ and ‘animals’ should probably not count as two distinct words, nor ‘follow’, ‘follows’, ‘following’, etc. You can choose to build the index using a stemming algorithm that groups such words as one (if there is one available for your language) or you can try the fuzzy matching algorithm. Whether it is better to be strict or to be relaxed will depend on your website.

A third setting is for blogs written mainly in Chinese, Korean, or Japanese. The MySQL fulltext index used by Similar Posts has problems with these languages but this setting tries several ways to work around the issues. The setting currently only works when posts are encoded as UTF-8. I would be very glad to get opinions from users familiar with these languages.

To avoid excessive memory use the indexing routine processes posts in batches of 100. This figure can be reduced to shrink the memory consumption even further.

Language Issues

The underlying MySQL full-text indexing is obviously very locale-dependent — how words are divided or punctuation handled, what words are treated as noise, etc. all vary from language to language. For the Similar Posts plugin to work well the version of MySQL on your server must be properly setup in the appropriate language.

Similar Posts generates the terms it matches on by analysing the word frequency of a post while ignoring the most common ‘noise’ words — in English, words like ‘of’, ‘and’, ‘across’, ‘someone’, etc. It uses a so-called ‘stop list’ of common English words to ignore. In fact it uses the stop list a standard English installation of MySQL uses. Obviously this list will be useless for other languages so Similar Posts makes the stop list pluggable.

The Similar_Posts folder contains a subfolder, ‘languages’, with stop lists and stemmers for a German, English, French, Spanish, and Italian. The plugin checks the WPLANG constant (defined in wp-config.php) to see which language WordPress is using and looks for a file on that basis. If WPLANG is undefined or the appropriate file cannot be found the default English list is used.

If you are looking for help setting up a stop list in a language other than English a good resource can be found at http://www.ranks.nl/stopwords. Stemmers in PHP are harder to come by. You can work out how to adapt any you find by inspecting the provided stemming files

414 replies on “Similar Posts”

  1. Thanks Rob,
    I downloaded the plugin again using the link at the top of this page. I removed the old plugin through the settings screen and deleted both plugins (including library).

    I reuploaded library plugin, activated it, reuploaded similar posts plugin, activated it and got the same error.

    I then reindexed the pages and still the same error.

    i LOVE the plugin if it works, this will rock… hmmm…

  8. Hi
    I have installed the latest version 2.5.0 bu tstiil get this error where erever I call similar_posts():

    You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ‘) as score FROM `wp_similar_posts` LEFT JOIN `wp_posts` ON `pID` = `ID` WHERE ()’ at line 1]
    SELECT *, ( ) as score FROM `wp_similar_posts` LEFT JOIN `wp_posts` ON `pID` = `ID` WHERE () AND post_date <= ‘2008-06-01 22:06:51’ AND post_status = ‘publish’ AND ID != -1 AND post_password =” AND (post_date <= DATE_SUB(CURDATE(), INTERVAL 0 day) OR post_status=’static’) ORDER BY score DESC LIMIT 0, 6

    I have checked the database and wp_similar_posts table is not been created. Can I created manually and what are the attribute of that table. Or can this problem be solved by other ways?

    Thanks in advance

    BTW, great job.

  9. DarkRaisin: First try deactivating the plugin and then reactivating it. Then rebuild the index from the plugin’s settings screen.

    If that doesn’t work try the same process but instead of deactivating use the settings option to remove the plugin’s settings.

    If you still have no luck please submit a bug report from the settings screen as it provides me with some debugging information.

    Good luck!

  10. Hi
    did all the magic you tought me but no luck, I have reported the error as you instructed.

    Thanks a lot so far.

    BTW. can I create the table my self? just tell me how many rows and culonms and waht attribute please.

  11. is this the query to create the table:

    CREATE TABLE `wp_similar_posts` (`pID` bigint( 20 ) unsigned NOT NULL , `content` longtext NOT NULL ,`title` text NOT NULL ,`tags` text NOT NULL ,FULLTEXT KEY `title` ( `title` ) ,FULLTEXT KEY `content` ( `content` ) ,FULLTEXT KEY `tags` ( `tags` )) ENGINE = MyISAM CHARSET = utf8;

    I will give it a try.

  12. yes, table created now, reindexing done. but no similar post is being shown and I am sure I have some simlar posts. 🙁

  13. DarkRaisin: Sorry to not get back to you earlier. So you have managed to create the database and index it but still no matches are being found…

    There was something rather strange about the bug report you sent me. It should contain a list of the current settings of the plugin but yours were all blank. I do not know why this is.

    Could you send me — via another bug report — the url of your site so I can take a look at it?

  14. DarkRaisin: I think I have found the problem. Are you on a windows server? There’s a bug in your version of WordPress that stopped the installation code from running. I’ll package up a fixed version and post it later today. Please let me know if it works for your case.

  15. You are right. Server is Windows based. I am going to install version 2.5.0.7 and will let you know. And I have installed Post-Plugin Library.

    Thanks again fo ryour great effort.

    DarkRaisin

  16. Ok.
    I have removed the older version, deleted the table in database. upload and install the latest version. tried to activate; got this error:

    Fatal error: Call to undefined function ppl_plugin_basename() in D:\Domains\mydomain.com\wwwroot\wp-content\plugins\similar-posts\similar-posts-admin.php on line 674

    :((

  17. DarkRaisin: Are you sure you got the latest Post-Plugin LIbrary (2.5.0.7) ? My freshly downloaded copy has a common_functions.php file and I can see, at the bottom, ppl_plugin_basename defined.

  18. Fatal error: Call to undefined function: ppl_plugin_basename() in /home.10.13/penseeli/www/blog/wp-content/plugins/similar-posts/similar-posts-admin.php on line 674

  19. Dear Rob

    no 🙂 mine was v.2.5.0.3. I will upgarede it tomorrow and will let you know.
    Thanks again.

    DarkRaisin

  20. well, well. this one has created the table and indexing went also OK, but no similar post is being shown. I will try to make a bug report.

    PS: I couldn’t wait until tomorrow 🙂

  22. If you had the misfortune to download version 2.5.0.7 you may run into problems when you try to update to a newer version. I managed to introduce a circular dependence between this plugin and the Post-Plugin Library which means trying to update either from the admin plugins page will cause an error. Instead you need to delete the 2.5.0.7 version from the server (Post-Plugin LIbrary too) and upload the newer one and the problem should be solved. I apologise for any worry or inconvenience.

  23. Dear Rob
    I have deleted the table, older plugins, uploaded and activated v.2.5.0.8 for both plugins.
    Table has been created, indexing went also OK, but still no similar posts is being shown. I shall only use this right?

  24. Thanks very much for the great plugin, Rob! I’ve tried all the other plugins I could find for this feature, and yours was by far the best after I configured the weightings.

    Now I’m looking to make use of this for the 404 page, and I have yet to find anything satisfactory. I’m currently using the Google solution but it’s too cluttered and not customisable. There’s Related Posts for 404 plugin, but I really don’t want to use the Related Posts plugin, because yours is so much better. I’ll try Tony’s solution (comment #161), but yes, I’d definitely be interested to see this feature available for Similar Posts.

  25. Bee: Thanks for your kind comments! I’ll bear your request in mind. As you know my own feeling is that search engine keywords are too sparse to make for good similar matches… but I am open to persuasion.

  26. Dear Rob
    Did you have time to look at my Bog-report?

    Appriciate your work

    Regards
    DarkRAisin

  27. Feature request! For those of us with multiple authors on our blogs, it would be really really nice to also have the author in the list, like:
    * Article by Author

  28. Angelo: I may be misunderstanding you… but the output tag system allows you to customise your list in many ways including displaying the post author (use the {author} tag).

  29. weird dates on recent posts and similar posts. Just started getting some strange URLs on posts on Similar posts and recent comments. See my url here http://myzerowaste.com/. In the RHS panel you will see ”you may also like to read” (similar posts) checkout the list of links here, they are mixed between permalinks, 1970 dates and default URLs some links are totally wrong. See also latest 5 comments panel, and most popular. Similar faults exist I have made no hacks. All plugins are fully up to date. Please advise. Thanks.

  30. Richard: Very strange indeed! Can you make a bug report … just in case it shows anything out of whack?

    Since it’s both permalinks and dates that are odd but not titles etc. it’s hard to know where the problem is. I’ll delve deeper.

    Thanks for the message.

  32. DarkRaisin: Sorry … I thought I’d replied by email.

    Your bug report shows that your Similar Posts settings are missing for some reason. You can go through the options screen and make sure that all the entries look right, saving at each screen. Alternatively, remove Similar Posts (via the settings screen), then download by hand the latest versions of the Post-Plugin Library and Similar Posts and install them.

    Let me know whether you have any joy.

  33. I’m using the latest version of the Similar Posts plug-in (v2.5.0.9). None of my posts have related posts showing. Not sure if the recent upgrade caused this problem or not, but I thought I’d mention it. I’m digging around…

  36. Father Marsh,
    I have sent you an email through your contact form on your blog. In that email, I have cut-and-pasted slow queries from the similar posts plugin. Perhaps you can take a look and see what you can do? Thanks!

  38. Tony: I’ve spent quite a few hours trying to find ways of speeding up the Similar Posts queries. I tried four alternative approaches and found each to take at least twice as long. Now I am testing on a setup with only 600 posts and it may be that the different approaches scale differently since your setup seems to be much larger. So I have no easy answer for you but I will continue to look for ways to speed things up on large sites.

    If you have any suggestions I’d be very happy to hear them.

  39. hello. i tried to install the similar posts plugin and have been doing that for like 2 hours but was unsuccessful. i also installed your post-plugin library, and that was ok. however, after uploading the similar posts plugin into my plugin folder, it does not even appear on my plugins list. i tried deleting and uploading again several times but nothing seems to work. do you know what i might be doing wrong?

    thank you so much.

  40. gilda: I really don’t know what could be causing the problem. It should at least show on your plugins screen as a plugin to be installed–all it takes for that to happen is for WordPress to read the description at the top of the file. All can suggest is that you download a new copy of the plugin and try again.

    Please let me know if you have any luck.

  42. Soverato News: Thank you for your interest.

    Both of those possibilities are available now. In place of the {link} output tag you can ‘build your own’, e.g.:

    <a href="{url}" alt="{title}" rel="nofollow">{title}</a>

  44. hey,

    i’ve been unable to get the related posts plugin to work, but the recent posts and recent comments do, which is confusing. they all show up in plugins and i can change the options.

    i’ve basically got the same code repeated three times to embed recent posts, related posts, and recent comments into the sidebar, but the related posts part seems to just be skipped over, and it doesn’t create the section in the sidebar at all. any ideas what could be causing that?

  46. okay, this is what i have:

    

<div class="posts">

		<ul id="recentposts">

			<li id="recent-posts">

			<h2>Recent Posts</h2>

				<ul>

					<?php recent_posts(); ?>

				</ul>

			</li>

		</ul>

	</div>

	
	<div class="similar_posts">

		<ul id="similarposts">

			<li id="similar-posts">

			<h2>Related Posts</h2>

				<ul>

					<?php similar_posts(); ?>

				</ul>

			</li>

		</ul>

	</div>


	<div class="recent_comments">

		<ul id="recentcomments">

			<li id="recent-comments">

			<h2>Recent Comments</h2>

				<ul>

					<?php recent_comments(); ?>

				</ul>

			</li>

		</ul>

	</div>

    i’m confused, because the similar posts part doesn’t seem to display anything, but the recent posts and recent comments both work. all 3 of the plugins show up as activated in the wp admin section, and i can change options and whatnot…

  47. well, um…

    it just… started working. and i didn’t change anything relevant that could’ve fixed it. which kinda bothers me, because now i don’t understand what was broken or why it now works, haha.

    now that i’m here, here’s another one: how do you get the little mini “document” icon that appear sto the left of the similar posts? they are there for recent posts and recent comments.

    thanks 🙂

  48. crystalline: I’m glad it came to life for you!

    The icon is part of the theme I use — automatically applied to sidebar list items as a background image I believe. The icon set used is by Kevin Potts.

  50. I post you bug report through form, but I found the problem with upgrade to 2.5.0.11 version.

    Settings change after upgrade and field “Display posts that are:” – “Published” is set to “NO”. That is the problem I couldn’t see any Similar Post.

    My regards, ScoopCelebrity

Comments are closed.