Similar Posts

Caveat

Unfortunately, due to ill-health, this plugin has not been developed or supported properly for some years. It works with the latest versions of WordPress (including on this website) but could possibly conflict with any WordPress features added after 2008 — e.g. custom post types — if you use them.

Purpose

This plugin displays a list of posts which are related or similar to the current post.

This is version 2.6.2.0 download latest version. It is compatible with WordPress 1.5–2.6.2.

  • 2.6.2.0 fixes a problem with stemming and stop words and offers a new fuzzy matching capability; supplies a .pot file making internationalisation possible; introduces the {imagealt} output tag and allows {excerpt} to output whole sentences; the content filter and the widget can now take a parameter string; output can be automatically placed after post content without editing theme files.
  • 2.6.1.2 fixes the German-language stemmer which should have been encoded as utf8.
  • 2.6.1.1 fixes the Italian-language stemmer which was crashing under PHP4.
  • 2.6.1.0 allows the current post to be marked manually where the automatic mechanism fails; when used as a widget the plugin now honours the setting to show nothing when there is no output; {commenterlink} now applies the appropriate WordPress filter; and fixed a problem with some installations not finding the right language files.
  • 2.6.0.1 fixes the option to include attachments and adds a parameter to the {imagesrc} output tag to append a suffix to the image name.
  • 2.5.0.11 has a new option to include posts which are attachments; a new output template tag {authorurl} which points to the archive of the author’s posts; new behaviour for the {php} output tag which can now accept other output tags in the code; and includes a fix for MySQL problems in some locales’.
  • 2.5.0.10 provides the ability to select from two algorithms for term extraction; allows you to specify post relationships by hand; and fixes a problem indexing tags in some languages.
  • 2.5.0.9 adds an option to match the current post’s author and extends the options for snippet and excerpt output tags to make the ‘more’ text into a link.
  • 2.5.0.8 adds an option to show posts by status, i.e., published/private/draft/future, changes the {categorynames} and {categorylinks} output tags by applying the ‘single_cat_name’ filter, and fixes a bug in WordPress pre-2.2 that stopped installation code running on Windows servers.
  • 2.5.0 improves the CJK matching algorithm by using bigrams. Also introduces a new output tag {imagesrc}, and adds more parameters to {image}. Fixes bugs with empty categories, excluded posts, and the option to omit current posts.
  • 2.5b28 improves the matching algorithm and adds an experimental mode for blogs in Chinese, Korean, or Japanese.
  • 2.5b27 fixes a bug with the bulk indexing of tags.
  • 2.5b25 makes some important changes: the {image} output tag now serves real thumbnails (couple of bug fixes too); output can now be sorted as you choose with sub-headings included; the {date:raw} tag modifier has been added to help the sorting; the ‘trim_before’ option has been replaced with the more logical ‘divider’.
  • 2.5b24 fixes to stop recursive replacement by content filter, {gravatar} to allow for ‘identicon’ etc., to {commenter} to allow trimming, and to remove a warning in safe mode
  • 2.5b23 brings a new option to filter on custom fields and adds proper nesting of braces in {if}.
  • 2.5b22 moves the manage menu under settings as a subpage, restores automatic indexing on activation, fixes conflicts with the legacy Similar Posts Feed plugin, fixes bugs in several output tags, and introduces the option to show only pages.
  • 2.5b20 doubles the speed of indexing and reduces the memory footprint considerably.
  • 2.5b19 fixes a bug when snippets are stripped of extra tags.
  • 2.5b18 fixes a problem with filtering the output and introduces the conditional tag {if:condition:yes:no}.
  • 2.5b16 fixes a problem with {php}.
  • 2.5b15 fixes for some more installation problems, one or two bugs, and adds the ‘included posts’ setting.
  • 2.5b14 fixes for some of the kinds of installation problems.
  • 2.5b11 fixes some widget problems.
  • 2.5b10 fixes (some?) of the problems folks have been having with no posts found. Most of such errors seem to arise when the proper table is not created and this version addresses that.
  • 2.5b9 has new features and improvements.
  • 2.3.6 restores the widgetiness I managed to remove in 2.3.5!
  • 2.3.5 has been rebuilt to save memory and can match the current post’s tags. It also fixes a bug with categories in WordPress < 2.3.
  • 2.3.4 now works as a widget.
  • 2.3.3 beta adds the ability to include as well as exclude categories and authors and is able to find posts by tag.
  • 2.3.2 beta fixes a conflict between tags and categories.
  • 2.3.1 beta fixes a stupid bug in category exclusion.
  • 2.3.0 beta is compatible with WP 2.3, fixes the {author} bug, and a number of problems related to versions of MySQL.
  • 2.1.1 beta fixes a badly chosen fallback value for the number of terms used to match similar posts.

Ideally, similarity or relatedness would be based on a post’s meaning. Tagging systems try to add meaning after the fact but suffer from two deficiencies, one practical and the other theoretical. When a blog already has many posts it can be impractical to retrofit a tagging system by tagging every post by hand. ‘Automatic’ services, like Yahoo’s, tend to produce too many suggestions which need to be culled, again by hand.

The theoretical problem with tagging is that it tries to pin down a meaning for a post by categorising it under a small number of types, whether those types belong to a predetermined hierarchy or arise by ‘folk’ classification. In fact, a post has a variety of meanings, a multitude of ways it can be related to other posts. Meaning doesn’t just lie in the intention of the author or in the classification of the reader; meaning also inhabits the text itself. Meaning is in the words.

The Similar Posts plugin compares posts by comparing their words. MySQL has a sophisticated full-text searching facility with a carefully tuned algorithm for judging the similarity between texts. Similar Posts extracts representative words from a post’s content, title, and tags and uses the full-text index to find the best matches between posts. This simple approach gives surprisingly good results.

The results can be tweaked in several ways to tailor them for you blog. By default the plugin chooses the 20 most frequent words to make its matches but the number is adjustable. It is worth experimenting to see how many words gives the best results for your blog — it has hardly any impact on speed, even if you set the value high enough to include the whole post. The relative importance given to words in your title may be adjusted so that well-chosen titles can be used to advantage or titles with little relevance downplayed. Similarly, tags can be used to improve matching or not according to your blog and it’s needs.

It is also possible to override the automatic similarity ranking by using a custom field. In the post edit screen create a custom field called ‘sp_similar’ with the ID value of the post to which you wish to ‘link’. You can link to multiple posts by entering a comma-delimited list of IDs.

The plugin has a settings page which lets you change how the output is generated and displayed. There is also a management page where you can change settings which affect the index.

Note: Similar Posts needs to know the ID of the post for which it is generating related posts. WordPress keeps track of that information in a global variable but unfortunately some other plugins can corrupt the data before Similar Posts gets a chance to use it. Similar Posts tries various tricks to get round this but sometimes it fails. The usual symptom is a list of similar posts that stays the same from page to page. You can help Similar Posts out by marking the current post manually by adding a line to your theme files. Find the place where the_content(); is used to display the current post and right after it put similar_posts_mark_current();.

Installation Instructions

  1. If upgrading from a previous version, first deactivate the plugin via the Plugins page and delete the plugin folder from your server.
  2. If you have been using the Similar Posts Feed plugin you should deactivate it as it is now obsolete.
  3. Upload the plugin folder to your /wp-content/plugins/ directory. You will also need to install the Post-Plugin Library.
  4. Go to your admin Plugins page and activate Similar Posts. This will automatically add a new table to enable fast, flexible full-text matching. If the plugin reports that there was a problem creating the table first try deactivating and reactivating the plugin.
  5. Put<!--?php similar_posts(); ?--> at the place in your theme files where you want the list of similar posts to appear. Lorelle on WordPress has a good guide to modifying themes for plugins.If you are averse to editing template files you can also place the post listing automatically either as a widget in the sidebar of your widget-aware theme or after each post (from the plugin’s Placement submenu).
  6. Use the admin Settings|Similar Posts pages to set all the available options. Alternatively, the options can be overridden by passing a parameter to the similar_posts template tag.

Usage and Options

The configuration page will help you to set up the plugin to your satisfaction.

The Index Management Page

Using this settings subpage you can re-index your blog. There are two main settings which affect the indexing.

PHP is, by default, not very good at handling text that isn’t in English and you might find Similar Posts mangles extended characters. If so, you can get the plugin to use PHP multi-byte string library if it is available.

The second setting attempts to handle words with related meanings. For example, ‘animal’ and ‘animals’ should probably not count as two distinct words, nor ‘follow’, ‘follows’, ‘following’, etc. You can choose to build the index using a stemming algorithm that groups such words as one (if there is one available for your language) or you can try the fuzzy matching algorithm. Whether it is better to be strict or to be relaxed will depend on your website.

A third setting is for blogs written mainly in Chinese, Korean, or Japanese. The MySQL fulltext index used by Similar Posts has problems with these languages but this setting tries several ways to work around the issues. The setting currently only works when posts are encoded as UTF-8. I would be very glad to get opinions from users familiar with these languages.

To avoid excessive memory use the indexing routine processes posts in batches of 100. This figure can be reduced to shrink the memory consumption even further.

Language Issues

The underlying MySQL full-text indexing is obviously very locale-dependent — how words are divided or punctuation handled, what words are treated as noise, etc. all vary from language to language. For the Similar Posts plugin to work well the version of MySQL on your server must be properly setup in the appropriate language.

Similar Posts generates the terms it matches on by analysing the word frequency of a post while ignoring the most common ‘noise’ words — in English, words like ‘of’, ‘and’, ‘across’, ‘someone’, etc. It uses a so-called ‘stop list’ of common English words to ignore. In fact it uses the stop list a standard English installation of MySQL uses. Obviously this list will be useless for other languages so Similar Posts makes the stop list pluggable.

The Similar_Posts folder contains a subfolder, ‘languages’, with stop lists and stemmers for a German, English, French, Spanish, and Italian. The plugin checks the WPLANG constant (defined in wp-config.php) to see which language WordPress is using and looks for a file on that basis. If WPLANG is undefined or the appropriate file cannot be found the default English list is used.

If you are looking for help setting up a stop list in a language other than English a good resource can be found at http://www.ranks.nl/stopwords. Stemmers in PHP are harder to come by. You can work out how to adapt any you find by inspecting the provided stemming files

414 replies on “Similar Posts”

  1. I upgraded both post-plugin, and similar posts.
    strange is that I see both No Related Posts, and Similar posts, both printed. Have I missed any thing ?
    +++++++++++++++++++++++++++++
    Related posts

    * No related posts.

    * Travel from Boston to Bangalore
    * 1947 -> 2007
    * science identifies risks…
    +++++++++++++++++++++++++++

  3. Arvind: Looking at the source of your web page in my browser I can see that Similar Posts is being called two times. Is that deliberate?

  5. Arvind: The page source has the two appearances separated by some html. After the first list there are two closing divs one for the post and one for the entry. The second list is prefixed by an opening entry div which never seems to be closed.

    Very strange. Would you be willing to send me the php file by email? I’ll drop you a note with my address.

  7. I am using WordPress MU (1.5.1).

    After adding and activating the plugin (and the post plugin), I can’t seem to get Similar Posts to work. The output says similar posts took these many seconds, but there is no output.

    I checked the database, and there is no wp_similar_posts table. I go no errors when activating the plugins. I can go the administration page, and then set the configuration items, and they seem to change (for example, show 2 posts instead of 5). These configuration items were all set to empty when I got in there the first time.

    Any ideas? It affects all the blogs in this network using WordPress MU.

    Regards,
    Vikram

  8. I tried to installed the plugin but i am getting the “none found” whatever i try.

    i have wordpress 2.5.1 installed and i have installed the 2.6.0 similar post plugin alongwith the post plugin library from your site.i tried to configure the settings but all in vain.i also run the SQL query successfully that you have provided un FAQ no1.

    Please guide me how to set it up in my blog.feeling very frustrated

    many thanks

    Amit

  9. Everyone: This sudden flurry of installation problems are presumably all related. I haven’t tracked down the cause yet but you could all help me by sending in a bug report from the plugin’s settings screen. I might be able to see a common pattern.

    Thanks for your patience and help.

  10. NB: I believe I have found the cause of the installation problems — related to WP 2.6’s capacity to relocate the content folder — and, if I am right, version 2.6.0.1 should fix things.

  14. I was trying to match posts just by giving all weight to the title and 0 to tags and posts. But somehow words that have 2 or 3 characters are taking into account by trying to find matches. Is there any way to apply the same rules like for the post content. Adding the words to the variable $tinywords did not change anything. Thank you very much for your help.

  16. Hi Fr. Marsh.

    I’m getting some strange messages in my error_log file.

    Û¢n—m4ó]yÛOõš®x§€8¯ŠÈ¨¼³zº"Ÿøh™ï솊¦æå‰Èmš_ð¥Ê'µéíþ™n‚)ìþȦŠV«¦‹-³û"š)Z®š,¶Êa¦‰åŠw¶ß­º&évÓO5ם·ôñÏYªçŠx®Ê+µìiyËl¥ªÚ™ë^¯[hmæ«­¬§ºY`Š÷§Šá¢g¿²)þ››–'!¶i—(ž×§·úeº§³û"š)Z®š,¶ÏìŠh¥jºh²Û)†š'–)ÞÛž6蛥ÛM<×^vßÓÇ=f«ž)àjºÚÊÉbqç±¥ç-²–«jg­z½m¡·š®¶²žée‚+Þž)ÿ†‰žþÈh§únnXœ†Ù¥ÿ
¢{^žßé–è"žÏìŠh¥jºh²Û?²)¢•ªé¢Ël¦hžX§{nzÛ¢n—m4ó]yÛOõš®x§€‰ïjXj¸.™éí²êi–'~Šß¢·šr§þ&{û!¢Ÿé¹¹brf—ü)r‰íz{¦[ Š{?²)¢•ªé¢ËlþȦŠV«¦‹-²˜i¢ybí¹

  25. kompass: If your site is using the utf-8 character set and you set the plugin index to use extended characters Similar Posts should work with Russian text. There is no supplied stop-word list or stemmer though — if you can add them it will probably improve accuracy somewhat. (and if you do I will be glad to include them in the package)

  26. Hello.

    I Recently posted a bug in the admin area , output tab.

    this error apears. Fatal error: Call to undefined function ppl_display_trim_before() in /home/.lolli/scyfox/scyfox-site.com/wp/wp-content/plugins/similar-posts/similar-posts-admin.php on line 115

    The plugin still works but dunno what is causing this.

    One other thing. There’s any way I can use the function_exist() function to check if the plugin is active show the content or else… cause I tried with this and didn’t work

    <?php if(function_exists('similar_posts()')){
    similar_posts();}?>

    I place the original code and worked like wonder, but it would be really nice that in case of deactivating the plug it doesn’t break the template.

    Just a thought

    Best of luck

    Silla!

    Best of luck

  33. Hi Bob
    I am still getting none found. I have removed the older versions installed and activated both library and similar, reindexed the database but still getting no similar posts. the tilte of posta re in English though and have some 6000 posts. I am sure there are some similar post title in my database,
    Any help would be approciated.

    DarkRaisin

  34. DarkRaisin: We really need to solve this!!! Can you please send me another bug report from the Similar Posts settings screen and we’ll start from there. Thanks!

  37. Rob
    I was not able to submt a bug report, it gives an error everytime:

    Error 403
    We’re sorry, but we could not fulfill your request for /report-a-bug/ on this server.

    You do not have permission to access this server. Data may not be posted from offsite forms.

    Your technical support key is: 5ab8-7e8d-cd36-1abb

    You can use this key to fix this problem yourself.

    If you are unable to fix the problem yourself, please contact rmarshsj at hotmail.com and be sure to provide the technical support key shown above.

    Any help?

    DarkRaisin

  39. DarkRaisin: I have traced the problem with bug reporting to the latest version of the Bad Behavior anti-spam plugin. I’ve deactivated BB while I look for a workaround so you should once again be able to report bugs from the settings screen.

  40. Carlos: Can you tell me what the link is that seems to be failing?

    I think the problem is due to some intermittent trouble with the wordpress.org/extend servers but it would be good to make sure that is the link you are talking about.

  41. Hi, I’m trying to show similar post after a single post based on Match the current post’s tag. I have only one. I have sett Number of posts to show to 30. I have 21 posts that I want to display but only 15 come up. Any Ideas what’s wrong?

  43. Helena: It may not actually be wrong, strange as it may seem. Similar Posts uses the MySQL full text search facility to query the database and it only returns posts it considers similar in some way. It sounds like you have 6 posts which MySQL is ignoring as too unlike to consider.

  44. Hi.

    I’m only interested of matching by tags, so I have sett “Other Settings→Relative importance of:” to content 0%, title%, 100% tags. Are there any other settings that might be of importance? Or maybe I should do this with manual links in custom field. But I don’t understand how to configure it. Sorry.

    Also I only want to show 1 category name in the output. (I use several categories like tasted, brewage, beer and drink…for layout on index page category etc.) I only want do display beer or red wine if it’s wine. I have bee trying with something like this:
    $os = array({categorynames: , });
    if (in_array(“Vitt vin”, $os)) {
    echo “Vitt vin”;
    }
    if (in_array(“Rött vin”, $os)) {
    echo “Rött vin”;
    }
    if (in_array(“Mousserande vin”, $os)) {
    echo “Mousserande vin”;
    }
    if (in_array(“Öl”, $os)) {
    echo “Öl”;
    }}
    I don’t know if this is to much to ask, but please help.

  45. Hi.

    I’m only interested of matching by tags, so I have sett “Other Settings→Relative importance of:” to content 0%, title%, 100% tags. Are there any other settings that might be of importance? Or maybe I should do this with manual links in custom field. But I don’t understand how to configure it. Sorry.

    Also I only want to show 1 category name in the output. (I use several categories like tasted, brewage, beer and drink…for layout on index page category etc.) I only want do display beer or red wine if it’s wine. I have bee trying post the code but it dosen’t work.
    I don’t know if this is to much to ask, but please help.

  46. I’m so Sorry for the duplicated entries. I got error messages when I posted the comment. Sorry //Helena

  47. Helena: No problem. My ISP has been fiddling with security settings and I’ve been getting strange events like that. I hope it will settle down soon.

    Let me ponder on your question for a little while to see if there’s a simple solution. OK?

  48. Helena: The first part of your question is about how to set up Similar Posts to just match on tags. In addition to biasing the matching algorithm the way you have done you could also set the plugin to match the current post’s tags. In fact, if you are just matching on tags you might find you can use the simpler Recent Posts plugin (with the match tags option) to get your list of related posts. If you don’t like the age-bias of Recent Posts you could go for Random Posts. Similar Posts will work fine — just giving you some other options.

    Now for your other (harder) question. The following bit of code should do it.

    
{php:
$os=explode(',','{categorynames:,}');
if (in_array('Vitt vin', $os)) echo 'Vitt vin';
else if (in_array('Rött vin', $os)) echo 'Rött vin';
else echo 'other';
}

    Your own code had some non-standard quotation marks which would crash it. Give it a try and let me know how it goes.

  49. Rob
    I am writing this comment to show my appriciation – which is nothing in compare with what you have done- .

    Thanks for your great plugin and outstanding support. The plugin works now even for me with my ancient WP, as you nicely put it.

    Thanks again

    Darkraisin

Comments are closed.