Log entry: A Recent Update to XFtoWP...
Over the weekend I released a beta version of
XFtoWP 1.5.2 which adds "major" new feature to the Latest Forum Posts Widget that now allows users to show the latest posts from a specified forum only. On the surface this seems like a tiny feature to add, but for a plugin that shows data from a completely different software through an API (WordPress receiving data from XenForo), there are some difficulties with getting data in such a nuanced way and even more so: keeping it accurate!
After choosing where to add the widget, the user can also differentiate between showing the latest posts from
all forums or select just a single forum to show the posts from:
...and by saving the widget, a request will be made to XenForo that saves the latest posts data from the specified forum to the WP database for easy access. Now on the frontend, a simple widget showing the latest posts from
Forum X will appear on the website like so:
This is a cool feat, but the nature of a
latest posts widget is to show posts that are new. The next time somebody replies to a thread in that forum, this widget will need a way to know there are new posts to show, otherwise it will be outdated. So far the only time the widget goes to get new data is when the save button is pressed, and expecting site owners to do this every time they want the widget to update makes the feature completely unusable.
I won't explain the entire process of how XFtoWP pulls data, but to boil it down the plugin has a "main" refresh routine that runs on a set amount of time and then "sub" processes when dealing with individual pages. For example, the "main" refresh process grabs general data about a forum like posts/threads total, a list of latest posts, newest registered members, and a couple of other global metrics. For performance reasons it is important to keep this process lightweight and the amount of data saved to WordPress minimal so these requests can run fast and reliably.
By the very nature of Widgets, an infinite amount of instances can be created. It is technically possible to create 100 latest forum posts widgets all pulling data from separate nodes, but how you refresh the data of those Widgets later is what will make or break the reliability of the integration.
Playing through this scenario of 100 possible widgets, it makes sense that you
don't want to tie a data refresh process to a "main" routine as that is a
lot of data to crunch during a process that gets run a lot, where most of that data is most likely unneeded at that point in time. Furthermore, you also don't want to update all 100 widgets at a time for the same reason that most of that data may not be needed at the same time, and such a large request could time out or have other unintended effects.
The solution: Save API data to each widget instance away from the "main" data and trigger the "sub" refresh process only when needed (after a specified time limit) and only when any given Widget is active on a page. This will ensure only Widgets that are being seen and used get refreshed regularly, and widgets that are used less get updated less.
I hope someone out there enjoyed this episode of "Simple Things That Aren't So Simple" and I look forward to unveiling the next episode soon™.