The latest version of tt_news (3.0.0 development) has been said to improve performance. However, it still leaves a lot to be desired. Indexing tables can speed up some queries, but the reason why tt_news is so slow is the number of queries it generates, especially in LIST view. Stepping through the extension with the debugger shows how big the problem really is. There are plenty of tips about optimizing tt_news cache in the tt_news news list, but they bypass the main problem – the fact that tt_news is s-l-o-w. Below you will find the way that I have used to speed up significantly the generation of the LIST. It requires modifying tt_news code, and requires some understanding of the structure of the 4,000+ line PHP file. It can be used on any recent version, not just the latest, and requires customization in each case. All changes take place in /pi/class.tx_ttnews.php
The one function in the current development version of tt_news is isRenderField, which (although undocumented) checks if the field is supposed to be displayed. Unfortunately, this function does not work as desired – it requires TypoScript setup, that, again is undocumented in this development version.
I wrote these functions as a replacement. What they do is parse the template part for markers, and later in the functions check if the markers are present. If the marker is not used in the template, why spend time to query the database and run lines of code?
function isRenderMarker($marker) {
if (in_array($marker, $this->renderMarkers))
return true;
return false;
}
function getMarkers($template)
{
preg_match_all('/###(.+)###/Us', $template, $matches);
return $matches[0];
}
When loading the template, it has to be parsed for markers. In tt_news 3.0.0, it is done here:
$t['total'] = $this->getNewsSubpart($this->templateCode, $this->spMarker('###' . $templateName . '###'));
$t['item'] = $this->getLayouts($t['total'], $this->alternatingLayouts, 'NEWS');
// Parse out markers in the templates to prevent unnecessary queries and code from executing
$markers = $this->getMarkers($t['total']);
foreach($t['item'] as $item)
$markers = array_merge($markers, $this->getMarkers($item));
$this->renderMarkers = array_unique($markers);
Now, just go through the rest of the code (specifically getItemMarkerArray), and wrap any part with the if statement:
if($this->isRenderMarker('###MARKER###'))
{
// render the field
}
For example:
// get image markers
$markerArray['###NEWS_IMAGE###'] = '';
if ($this->isRenderField('image') && $this->isRenderMarker('###NEWS_IMAGE###')) {
$markerArray = $this->getImageMarkers($markerArray, $row, $lConf, $textRenderObj);
}
Make sure to go through any function that extend tt_news (comments, tt_ratings, etc) by using the extra marker hook, and add the code there, as these functions are executed for each item in the list and might waste a lot of time and resources if their output is never displayed. Example:
function extraItemMarkerProcessor($markerArray, $row, $lConf, &$pObj) {
if($pObj->isRenderMarker('###TX_COMMENTS_COUNT###'))
{
// Stuff
}
}
Results? LIST view generation time decreased from 20-30 seconds to 0.7-1.4 seconds. I hope this was the intention of the isRenderField function, and will make it to the final release of tt_news 3.0
Patch for testing:tt_news_performance_patch
Ralf Hettinger
Hi Dan,
this sounds reasonable – do you have a patchfile against trunk somewhere? I’d like to test 🙂
admin
I might create a patch – however you would still have to go through all the extensions utilizing the extra marker hook and add the check there, to realize the full benefits.
Ralf Hettinger
That is understood, thanks for highlighting.
Dan Osipov
Patch added
Ralf Hettinger
Thanks 😉
Ries
very nice, well done!!!
Ries
Michael Cannon
Great work Dan. Can you confirm what which version of tt_news the patch is for?
Besides using “a combination of ttnewscache & ttnewscache_clearlike to
minimize cache clearing when new stories are posted” was there any other extensions used to help with performance?
Michael
Dan Osipov
Michael,
The patch is for 3.0.0-dev, but it might not apply cleanly… Rather its better to apply the concept.
For performance we used eaccelerator, as well as a lot of hardware. No magic…
eater
Isn’t it too difficult to parse template for markers and use if statement for each marker?
Maybe it would be easier to generate output based on TypoScript, like in this article?
I guess that tt_news’s logic must be changed a little, but i think i worth it.
Steffen Gebert
If you
– use the tt_news feature “related news by category”
– have an extension hooking into tt_news (extraItemMarkerHook or itemMarkerArrayFunc)
– and want to call $pObj->isRenderMarker() there
you might run into trouble, as the before rendered items for “related by category” will overwrite $this->renderMarkers.
So add
$tmpRenderMarkers = $this->renderMarkers;
near
$tmpcatExclusive = $this->catExclusive;
and
$this->renderMarkers = $tmpRenderMarkers;
near
$this->theCode = $tmpcode;
Hope this saves somebody the hours, i’ve just spent on this… 😉
Steffen
Mazdek
Do you have patch for 2.5.2 version?
Ralf Hettinger
After some testing and thinking I would think it is a very good idea to parse a template for existance of a maybe complex TS marker before passing it to the long run (i.e. tslib.content). Especially if you use tt_news within the same page more than once.
One more idea for that occasion: Store a hash of any loaded template and its markers in GLOBALS[‘TSFE’], and remember the markers within this template by some hash. Result: Template parsing and deciding if markers are available as your patch suggests needs to be done only once no matter how many tt_news sections within the page. I tried that on a tt_news heavy loaded site and had some recognizable performance gain -> works for me.
Thanks for the idea, Dan!
Louis COppola
Has this been integrated into the new typo3 3.0.1 version or is this something still worth doing?
I have a site with over 100,000 tt_news records. Homepage has about 6-8 list elements and as you can imagine load time is horrible. If I could increase speed by the size you are referring to – that would be great.
Thanks
Lou