Similar Posts

Caveat

Unfortunately, due to ill-health, this plugin has not been developed or supported properly for some years. It works with the latest versions of WordPress (including on this website) but could possibly conflict with any WordPress features added after 2008 — e.g. custom post types — if you use them.

Purpose

This plugin displays a list of posts which are related or similar to the current post.

This is version 2.6.2.0 download latest version. It is compatible with WordPress 1.5–2.6.2.

  • 2.6.2.0 fixes a problem with stemming and stop words and offers a new fuzzy matching capability; supplies a .pot file making internationalisation possible; introduces the {imagealt} output tag and allows {excerpt} to output whole sentences; the content filter and the widget can now take a parameter string; output can be automatically placed after post content without editing theme files.
  • 2.6.1.2 fixes the German-language stemmer which should have been encoded as utf8.
  • 2.6.1.1 fixes the Italian-language stemmer which was crashing under PHP4.
  • 2.6.1.0 allows the current post to be marked manually where the automatic mechanism fails; when used as a widget the plugin now honours the setting to show nothing when there is no output; {commenterlink} now applies the appropriate WordPress filter; and fixed a problem with some installations not finding the right language files.
  • 2.6.0.1 fixes the option to include attachments and adds a parameter to the {imagesrc} output tag to append a suffix to the image name.
  • 2.5.0.11 has a new option to include posts which are attachments; a new output template tag {authorurl} which points to the archive of the author’s posts; new behaviour for the {php} output tag which can now accept other output tags in the code; and includes a fix for MySQL problems in some locales’.
  • 2.5.0.10 provides the ability to select from two algorithms for term extraction; allows you to specify post relationships by hand; and fixes a problem indexing tags in some languages.
  • 2.5.0.9 adds an option to match the current post’s author and extends the options for snippet and excerpt output tags to make the ‘more’ text into a link.
  • 2.5.0.8 adds an option to show posts by status, i.e., published/private/draft/future, changes the {categorynames} and {categorylinks} output tags by applying the ‘single_cat_name’ filter, and fixes a bug in WordPress pre-2.2 that stopped installation code running on Windows servers.
  • 2.5.0 improves the CJK matching algorithm by using bigrams. Also introduces a new output tag {imagesrc}, and adds more parameters to {image}. Fixes bugs with empty categories, excluded posts, and the option to omit current posts.
  • 2.5b28 improves the matching algorithm and adds an experimental mode for blogs in Chinese, Korean, or Japanese.
  • 2.5b27 fixes a bug with the bulk indexing of tags.
  • 2.5b25 makes some important changes: the {image} output tag now serves real thumbnails (couple of bug fixes too); output can now be sorted as you choose with sub-headings included; the {date:raw} tag modifier has been added to help the sorting; the ‘trim_before’ option has been replaced with the more logical ‘divider’.
  • 2.5b24 fixes to stop recursive replacement by content filter, {gravatar} to allow for ‘identicon’ etc., to {commenter} to allow trimming, and to remove a warning in safe mode
  • 2.5b23 brings a new option to filter on custom fields and adds proper nesting of braces in {if}.
  • 2.5b22 moves the manage menu under settings as a subpage, restores automatic indexing on activation, fixes conflicts with the legacy Similar Posts Feed plugin, fixes bugs in several output tags, and introduces the option to show only pages.
  • 2.5b20 doubles the speed of indexing and reduces the memory footprint considerably.
  • 2.5b19 fixes a bug when snippets are stripped of extra tags.
  • 2.5b18 fixes a problem with filtering the output and introduces the conditional tag {if:condition:yes:no}.
  • 2.5b16 fixes a problem with {php}.
  • 2.5b15 fixes for some more installation problems, one or two bugs, and adds the ‘included posts’ setting.
  • 2.5b14 fixes for some of the kinds of installation problems.
  • 2.5b11 fixes some widget problems.
  • 2.5b10 fixes (some?) of the problems folks have been having with no posts found. Most of such errors seem to arise when the proper table is not created and this version addresses that.
  • 2.5b9 has new features and improvements.
  • 2.3.6 restores the widgetiness I managed to remove in 2.3.5!
  • 2.3.5 has been rebuilt to save memory and can match the current post’s tags. It also fixes a bug with categories in WordPress < 2.3.
  • 2.3.4 now works as a widget.
  • 2.3.3 beta adds the ability to include as well as exclude categories and authors and is able to find posts by tag.
  • 2.3.2 beta fixes a conflict between tags and categories.
  • 2.3.1 beta fixes a stupid bug in category exclusion.
  • 2.3.0 beta is compatible with WP 2.3, fixes the {author} bug, and a number of problems related to versions of MySQL.
  • 2.1.1 beta fixes a badly chosen fallback value for the number of terms used to match similar posts.

Ideally, similarity or relatedness would be based on a post’s meaning. Tagging systems try to add meaning after the fact but suffer from two deficiencies, one practical and the other theoretical. When a blog already has many posts it can be impractical to retrofit a tagging system by tagging every post by hand. ‘Automatic’ services, like Yahoo’s, tend to produce too many suggestions which need to be culled, again by hand.

The theoretical problem with tagging is that it tries to pin down a meaning for a post by categorising it under a small number of types, whether those types belong to a predetermined hierarchy or arise by ‘folk’ classification. In fact, a post has a variety of meanings, a multitude of ways it can be related to other posts. Meaning doesn’t just lie in the intention of the author or in the classification of the reader; meaning also inhabits the text itself. Meaning is in the words.

The Similar Posts plugin compares posts by comparing their words. MySQL has a sophisticated full-text searching facility with a carefully tuned algorithm for judging the similarity between texts. Similar Posts extracts representative words from a post’s content, title, and tags and uses the full-text index to find the best matches between posts. This simple approach gives surprisingly good results.

The results can be tweaked in several ways to tailor them for you blog. By default the plugin chooses the 20 most frequent words to make its matches but the number is adjustable. It is worth experimenting to see how many words gives the best results for your blog — it has hardly any impact on speed, even if you set the value high enough to include the whole post. The relative importance given to words in your title may be adjusted so that well-chosen titles can be used to advantage or titles with little relevance downplayed. Similarly, tags can be used to improve matching or not according to your blog and it’s needs.

It is also possible to override the automatic similarity ranking by using a custom field. In the post edit screen create a custom field called ‘sp_similar’ with the ID value of the post to which you wish to ‘link’. You can link to multiple posts by entering a comma-delimited list of IDs.

The plugin has a settings page which lets you change how the output is generated and displayed. There is also a management page where you can change settings which affect the index.

Note: Similar Posts needs to know the ID of the post for which it is generating related posts. WordPress keeps track of that information in a global variable but unfortunately some other plugins can corrupt the data before Similar Posts gets a chance to use it. Similar Posts tries various tricks to get round this but sometimes it fails. The usual symptom is a list of similar posts that stays the same from page to page. You can help Similar Posts out by marking the current post manually by adding a line to your theme files. Find the place where the_content(); is used to display the current post and right after it put similar_posts_mark_current();.

Installation Instructions

  1. If upgrading from a previous version, first deactivate the plugin via the Plugins page and delete the plugin folder from your server.
  2. If you have been using the Similar Posts Feed plugin you should deactivate it as it is now obsolete.
  3. Upload the plugin folder to your /wp-content/plugins/ directory. You will also need to install the Post-Plugin Library.
  4. Go to your admin Plugins page and activate Similar Posts. This will automatically add a new table to enable fast, flexible full-text matching. If the plugin reports that there was a problem creating the table first try deactivating and reactivating the plugin.
  5. Put<!--?php similar_posts(); ?--> at the place in your theme files where you want the list of similar posts to appear. Lorelle on WordPress has a good guide to modifying themes for plugins.If you are averse to editing template files you can also place the post listing automatically either as a widget in the sidebar of your widget-aware theme or after each post (from the plugin’s Placement submenu).
  6. Use the admin Settings|Similar Posts pages to set all the available options. Alternatively, the options can be overridden by passing a parameter to the similar_posts template tag.

Usage and Options

The configuration page will help you to set up the plugin to your satisfaction.

The Index Management Page

Using this settings subpage you can re-index your blog. There are two main settings which affect the indexing.

PHP is, by default, not very good at handling text that isn’t in English and you might find Similar Posts mangles extended characters. If so, you can get the plugin to use PHP multi-byte string library if it is available.

The second setting attempts to handle words with related meanings. For example, ‘animal’ and ‘animals’ should probably not count as two distinct words, nor ‘follow’, ‘follows’, ‘following’, etc. You can choose to build the index using a stemming algorithm that groups such words as one (if there is one available for your language) or you can try the fuzzy matching algorithm. Whether it is better to be strict or to be relaxed will depend on your website.

A third setting is for blogs written mainly in Chinese, Korean, or Japanese. The MySQL fulltext index used by Similar Posts has problems with these languages but this setting tries several ways to work around the issues. The setting currently only works when posts are encoded as UTF-8. I would be very glad to get opinions from users familiar with these languages.

To avoid excessive memory use the indexing routine processes posts in batches of 100. This figure can be reduced to shrink the memory consumption even further.

Language Issues

The underlying MySQL full-text indexing is obviously very locale-dependent — how words are divided or punctuation handled, what words are treated as noise, etc. all vary from language to language. For the Similar Posts plugin to work well the version of MySQL on your server must be properly setup in the appropriate language.

Similar Posts generates the terms it matches on by analysing the word frequency of a post while ignoring the most common ‘noise’ words — in English, words like ‘of’, ‘and’, ‘across’, ‘someone’, etc. It uses a so-called ‘stop list’ of common English words to ignore. In fact it uses the stop list a standard English installation of MySQL uses. Obviously this list will be useless for other languages so Similar Posts makes the stop list pluggable.

The Similar_Posts folder contains a subfolder, ‘languages’, with stop lists and stemmers for a German, English, French, Spanish, and Italian. The plugin checks the WPLANG constant (defined in wp-config.php) to see which language WordPress is using and looks for a file on that basis. If WPLANG is undefined or the appropriate file cannot be found the default English list is used.

If you are looking for help setting up a stop list in a language other than English a good resource can be found at http://www.ranks.nl/stopwords. Stemmers in PHP are harder to come by. You can work out how to adapt any you find by inspecting the provided stemming files

414 replies on “Similar Posts”

  2. I’m using this template

    {custom:thumbnail-small} {link}

    How do I check if the custom field is filled or not? If there is not value in the custom field then it makes an unnecessary space before {link}. Thanks.

  3. [/root]: Probably the easiest way is to bypass {custom} and use another {php} tag! Try:

    {php:$thumb=get_post_meta($result->ID, 'thumbnail-small', true);if ($thumb)echo $thumb.' ';}

  4. Upgraded to 2.5b20 from 2.3

    WP 2.5

    “None Found”

    Worked perfect before with 2000+ articles

    Tried evrything

    In the DB (MySQL 5.+) after config plugin no DB tables are populated within anything.

    Went back to 2.3…same problem persists.

  5. David: A number of questions… did you download the Post-Plugin Library which 2.5 needs? Did you use the Manage page to index your posts? Did you get rid of the old version’s file before upgrading?

    Those are the most obvious causes of Similar Posts not showing any posts.

    Please get back to me and we’ll get this sorted out. Similar Posts 2.5 has a bug report screen under the settings menu which might provide some useful clues.

  6. awesome plugin, everything is working fine for me, except today i activated the {views} for the similar posts and its only displaying ARRAY. Post views work fine on my sidebar with another plugin. im using wp 2.5, any advice on why it displays array instead of a number?

  7. rufus: The {postviews} output template tag relies on you having a plugin installed to do the post view counting. It works with several of the best known such plugins but not all. Which one are you using?

    However, if Similar Posts can’t find a plugin to work with it should return zero, not ARRAY, so maybe there is a bug in my code. Let me know which post view plugin you are using and I’ll look into it.

  10. Nathan Cain: Is this a new installation of these plugins? If it was an update make sure you removed the old plugin folders. Have you downloaded and installed the Post-Plugin Library whcih the plugins now require?

    Also try simply deactivating/reactivating the plugins. Remember to go to the Manage >> Similar Posts page to index your blog.

  11. Further to #104/#105

    Entirely my fault. Duh.

    I went to Settings > Similar Posts and set all the options but not clearly realizing I also had to go to Manage > Similar posts.

    May i suggest placing all options within one UI location in future?

  12. Rob – This is a new installation. I downloaded and activated the post-plugin library first. I reindexed under manage>similar posts. I deactivated and reactivated these plugins and I am still getting the errors.

  13. David: I’m glad you solved the problem! I was worried by your initial description–I thought we had obliterated your blog!

    The indexing thing is a chore. Originally, I had the installation routine do a basic index but it failed on some odd systems with very weird and uninformative error messages so now I leave it to the user to do a manual index. As you found out that’s unsatisfactory for several reasons.

  14. Nathan: I am puzzled. Can you tell me anything about your system? Apache? Versions of PHP, MySQL, etc. I have had one previous report of the subpages not working with a certain variety of IIS.

    Also are you using wp-cache or an admin drop menu plugin?

    Can you check to see if a table has been created called ‘wp_similar_posts’ and if that table has been filled with as many rows as the wp_posts table?

    I hope we can resolve this speedily.

  15. Hello Rob,

    it’s not a bug, but a thing that should be changed in the next version:

    If you include the Similar Posts in the rss-feed, there is no option to set (or translate) the string before the tag. It says hard-coded “Similar Posts:”

    It would be great, to have the possibility to change this string like in the other options-page.

    Greetings
    gero

  16. gero: Which version are you using? In v2.5 when you choose to have similar posts’ output to your feeds a new setting menu is created which lets you tailor the output to feed as much as you want, including changing the string before the tag.

  19. rufus: Sorry an email to you must have gone astray. It looks like Lester updated his plugin so the code had to be changed for the next version. In the meantime you can edit post-plugin-library/output_tags.php if you wish and make the patch.

    Round about line 380 you’ll find the code to handle postviews, update it to:

    	else if (function_exists('the_views')) {
		$count = get_post_custom($result->ID);
		$count = intval($count['views'][0]);
	}

    See if that does any good.

  20. Oh, sorry,
    yes, perfect! There are feed option. I just haven’t noticed, that they appear not in the similar feeds option, but in extra settings.
    Everything is perfect 🙂

  21. I placed the recent_posts on the sidebar and all works well. What would I need to do to get the same list to show up on my homepage which is outside of wordpress.
    So say public_html/wordpress –> wordpress blog
    I want to access it from public_html/index.php

  22. john: I’m not sure! You will have to ‘include’ the plugin and at least some of the WordPress files — which I am not sure. Maybe the codex has something on the subject?

  23. The plugin is working, but each post is only picking up that same post as a related site.

    I’m not sure how to set it up really whithi the settings part. I have recreated my index.

  25. I just upgarded to 2.5b22. I followed the steps as stated in the instructions and now no post will show up. It says ‘None Found’ I tried deactivating and reactivating and I still get the same thing. It worked fine before I upgrading to the new version. I went though all the setting, everything is fine. What did I do wrong?

  26. Crystal: Have you indexed the posts? Always worth asking! Perhaps you could submit a bug report from the settings page which will give me some extra information. Could you also tell me what version you upgraded from?

  27. I did index. Honestly I can’t remember what version I had before I upgraded. It was a more recent one, I know that, as I usually check my plugins a few times a month and upgrade them when needed. I just submitted a bug report.

    Crystal

  28. Crystal: I think I know what the problem is. You have the plugin set to match both categories and tags and that is a very strict condition. Try unsetting those options and see if the related posts come back.

    This is a change of behaviour from previous versions where — somewhat illogically — if such matching yielded no hits the plugin reverted to just ignoring the tags and/or categories.

  29. I unset the options you said and it works now. I had those options set-up before and it worked. Oh well….as long as it’s back up and running I’m happy. I’ve used this plugin for all my blogs, I love it. 🙂

    Thanks for your fast response helping me out.

  30. I’m using the plugin on wordpressMU V 1.2.5 I get the same output on every page. I’m not sure where to go form here. Thanks in advance for any help you might be able to provide.

  31. Max: I must confess I have been wanting to explore how to make sure my plugins work properly with WPMU for some time but I have never gotten to it.

    Do you know of any differences I should take into account? Any, perhaps you could make a bug report from the options/settings page as that will at least give me some information that might be useful.

  32. Just upgraded post-plugin-library to the latest, whenever I post or edit a post, I receive this error.
    Warning: exec(): Unable to fork [tasklist /FI “PID eq 10152” /FO LIST] in ….\wp-content\plugins\post-plugin-library\common_functions.php on line 441

    I downgraded to an earlier version, and that resolved this problem.
    btw. great plugins.

  34. Ok, I’ve just tested a couple of things and found that the problem is not caused by wordpressMU it’s actaully theme related. I’ve been testing with an old theme in MU and it works like a charm.

    Now I am not sure how to fix the problem so I have had to disable the plugin again. Why would a theme cause the pugin to pull the same posts on – the theme is quite popular http://www.copyblogger.com/the-copyblogger-theme-for-wordpress/ – try it out and I think you will be able to see the problem straight up.

  35. Max: It could happen if the them has multiple ‘loops’. Similar Posts has to find out somehow which post is the current one so as to compare it to others for similarity. It gets the current post’s ID by looking at the $post variable which is set in the WordPress loop. If theme or plugin or widget runs a second wp-query/loop without restoring its context Similar Posts will have the wrong ID play with. This problem can occur easily with widgets since some of the built-in widgets don’t properly clean up after themselves (though the code has attempted to since 2.5).

    There is no need for themes, plugins etc. to make this mess as Ronald Huereca explains.

  39. I’ve installed Posts Plugin Library on WordPress 2.3.3. But when I try to install similar posts I get the following error

    Warning: require_once(/home/.fenne/dbspin/analoguemagazine.com/wp-content/plugins/similar-posts/languages/en/stemmer.php) [function.require-once]: failed to open stream: No such file or directory in /home/.fenne/dbspin/analoguemagazine.com/wp-content/plugins/similar-posts/similar-posts.php on line 551

Fatal error: require_once() [function.require]: Failed opening required '/home/.fenne/dbspin/analoguemagazine.com/wp-content/plugins/similar-posts/languages/en/stemmer.php' (include_path='.:/usr/local/php5/lib/php:/usr/local/lib/php') in /home/.fenne/dbspin/analoguemagazine.com/wp-content/plugins/similar-posts/similar-posts.php on line 551

  41. Hi Rob,
    I am using the thumbnail option with similar posts and love the result, with one glitch – for youtube posts, there is no thumbnail. Is there a workaround for this?
    Thanks!
    sue

  42. Sue: I’m afraid not since youtube videos are not images. There may be a way extend the {image} output tag without too much trouble or overhead … I’ll look into it.

  43. Sue: There seem to be so many ways to embed videos — many of them non-standard — that I think you are out of luck for the near future.

  45. I have tried everything, DB table is there, i uploaded library plugin first then this one, activated them, put string in my post but it is causing a Fatal error, although its activated and configuring well.. wat to do?

  46. Please! i do everithing that u said but i only find NONE FOUND,
    I go to the edit plugins, and chek the code that you put… but it don{t work… can you please tell me what happend??
    i have the wordpress 2.5

  47. Hello,

    I’m using WP 2.5 and the newest version 2.5b26 of the plugin.

    Since – I can’t say the version exactly – I’ve noticed, that the results of the similar posts are not as good as they used to be, although I have not changed the settings.

    I also noticed a strange thing: if I recreate the index with “manage the index” the results under some postings are quite strange. I know that there are postings, which are quite more similar, but they do not appear.

    If I than open this posting and just save it again (without changing anything) the similar posts shown under it are much better now.

    How can it be, that the results are different?

    Greetings

  48. This plugin is fantastic.

    Only problem… because I have another plugin that modifies my feeds, every time I upgrade your plugin I have to manually edit line 560 (or near that line) from this

    if ($options['feed_active'] === 'true') add_filter('the_content', 'similar_posts_for_feed');

    to this

    if ($options['feed_active'] === 'true') add_filter('the_content', 'similar_posts_for_feed', 99);

    because the other filter must come first (long story).

    well, it would be awesome to have this in the options! 🙂

  49. I was wondering… how could I display similar posts on my 404 page? When Google sends people to a 404 page, I would like the 404 page to display posts similar to the one that was being searched for through Google.

  50. djvibe: There are no stopword/stemmer files for Chinese for a couple of reasons. First, I know no Chinese and don’t even know whether the concept of stopwords etc. is relevant. Second, I do know that the MySQL full-text index the plugin uses has trouble with the Chinese, Japanese, and Korean languages because they don’t use spaces to separate the text into words.

    If you can shed any light on these issues I would be very happy to add support for the CJK languages.

Comments are closed.