Memory leak in elgglib.php

[mdupont]

On a 200+ users on ELGG 1.6.1 website, from time to time the front page became unaccessible and we had a PHP fatal error for memory limit exhausted.

Increasing the PHP memory limit fixed the problem for a while, until the same problem appeared (a few days later) and that we had again to increase the memory limit. At a point we had to allocate hundreds of megabytes for ELGG not to throw an error, and PHP was eating so much memory that it could cause the server to crash.

We tried to ajust Apache, PHP, MySQL and Memcache configuration without success.

The PHP error indicates that the culprit is in engine/lib/elgglib.php at line 1795. Here is the block of code :

[Marcus Povey 20090217 : Now retrieving all datalist values on first load as this saves about 9 queries per page]

$result = get_data("SELECT * from {$CONFIG->dbprefix}datalists");

if ($result)
{

foreach ($result as $row)
{

$DATALIST_CACHE[$row->name] = $row->value;

Cache it if memcache is available
if ($datalist_memcache) $datalist_memcache->save($row->name, $row->value);

}

if (isset($DATALIST_CACHE[$name]))

return $DATALIST_CACHE[$name];

}

Just below this block, on line 1808 - 1815 is commented the old version of the code. After uncommenting this block and commenting the one above, ELGG was running again into 64 Mo of PHP memory AND page display was quicker than before. The code is :

if ($row = get_data_row("SELECT value from {$CONFIG->dbprefix}datalists where name = '{$name}' limit 1")) {

$DATALIST_CACHE[$name] = $row->value;

Cache it if memcache is available
if ($datalist_memcache) $datalist_memcache->save($name, $row->value);

return $row->value;

}

It turns out that saving queries wasn't the best way to increase performance and stability. Since this block of code is still here in 1.7, it should be corrected or reverted back to the old version.

[brettp]

How many entries are in your datalists table?

[cash]

I don't see how hundreds of MBs of data could be in the datalist table...

[brettp]

Nor I unless some plugin is using it to store data. Unless we get more info in the next few days we can mark this worksforme.

[mdupont]

Hi,

Thanks for your answer. Actually there are tens of thousands of entries in my datalists table, making it weight more than 250MB, of mostly duplicate entries (thousands of crontrigger_*, of simplecache_*, of view_*?). Why is a new entry created each time and not the existing value updated?

[mdupont]

Additional info : the datalist table is in latin1_swedish encoding and doesn't have an index. Is this normal?

[mdupont]

Update : deleted duplicates in the datalists table (down from 660'000 entries to 26!) and added name as the Primary key. Seems that no more duplicates are created and that ELGG is now fast... I'll wait and see if it remains stable.

No idea why the datalists table was not correct, but the fact that a new entry was created every time a settings was changed (including cron runs) could explain why it was eating more and more memory...

[cash]

Looks like someone must have changed your database schema after Elgg was installed. name is the primary key according to the schema that comes with Elgg.

[mdupont]

Yes, i've reset the database schema following engine/schema/mysql.sql and ever since everything seems to work - no more duplicate keys added. I think I'll never know why and how the schema was modified...

Thanks again for your help!