scavenge

[chrisf]

  I had originally posted this before.  Scavenge is very important in hosting business, without it, we are just giving away resources, and KloxoMR is good for only 'personal' servers.  When I run scavenge from command line I get these first:

[root@cc-server ~]# sh /script/scavenge
ionice: ignoring given class data for idle class
ionice: ignoring given class data for idle class
ionice: ignoring given class data for idle class
ionice: ignoring given class data for idle class
ionice: ignoring given class data for idle class
ionice: ignoring given class data for idle class
ionice: ignoring given class data for idle class

on some servers there may be 10-30 messages like above, then, scavenge always hangs at:

In client-admin cron_manage_flag equals 0 and limit is On
In client-admin backupschedule_flag equals 0 and limit is On
In client-admin backup_flag equals 0 and limit is On
In client-admin dns_manage_flag equals 0 and limit is On
In client-admin webhosting_flag equals 0 and limit is On
Dead Quota: In client-admin clientdisk_usage equals 125.9
Dead Quota: In client-admin traffic_last_usage equals 0
Dead Quota: In client-admin mysqldb_usage equals 2.5

I reproduced this 'hang' on every server, 6 now altogether.  Even on fresh install centos 6 _64.  It will sit there until I ctrl-c to break out of the script.

Please look at scavenge problems Mustafa, very important.

[MRatWork]

Find out linuxfslib.php and modified 'exec("ionice -c2 -n7 du -s {$path}", $out);' to 'exec("ionice -c3 -n4 du -s {$path}", $out);'

[chrisf]

Find out linuxfslib.php and modified 'exec("ionice -c2 -n7 du -s {$path}", $out);' to 'exec("ionice -c3 -n4 du -s {$path}", $out);'

Mine had -c3 -n7.  I changed it to -n4 and there was no change.  I changed it to the -c2 and the ionice warnings stopped (Changed in two places in that file).

Problem is scavenge hangs (Stops) after the above.  I have let it sit for 20 minutes.  What is it doing after the dead quote for admin (where I posted above) how do we get scavenge to run properly?

[MRatWork]

Value '-c3 -n7' that mean '-c3' as 'best effort' and '-n7' is 'when iddle'. So, I think change to '-c3' is solution for this situation.

Or. try remove '-c3 -n7'.

[chrisf]

Value '-c3 -n7' that mean '-c3' as 'best effort' and '-n7' is 'when iddle'. So, I think change to '-c3' is solution for this situation.

Or. try remove '-c3 -n7'.

-c2 is best effort, -c3 is idle.  -nX 1-7 is priority.   In newest release (20140101) you doubled up the code. Look:

Code: [Select]

//      exec("du -sc {$path} | grep -i '{$path}'", $out);
        exec("ionice -c3 -n7 du -s {$path}", $out);
        exec("ionice -c3 -n4 du -s {$path}", $out);
        $os  = preg_replace("/\s+/", ":", $out[0]);
        $t = str_replace(":{$path}", "", $os);

So all things get exec twice!   I changed to -c2 -n7 and warnings stopped.

STILL HANGS.  I let it sit and sh /script/scavenge doesn't complete.  Is this reproducible?

[MRatWork]

Change to:

Code: [Select]

//      exec("du -sc {$path} | grep -i '{$path}'", $out);
//        exec("ionice -c3 -n7 du -s {$path}", $out);
        exec("ionice -c3 -n4 du -s {$path}", $out);
        $os  = preg_replace("/\s+/", ":", $out[0]);
        $t = str_replace(":{$path}", "", $os);
or

Code: [Select]

//      exec("du -sc {$path} | grep -i '{$path}'", $out);
//        exec("ionice -c3 -n7 du -s {$path}", $out);
        exec("du -s {$path}", $out);
        $os  = preg_replace("/\s+/", ":", $out[0]);
        $t = str_replace(":{$path}", "", $os);

[chrisf]

Ah - found the problem.  You can set the io to idle class just remove the -nX here is quote from ionice developer:

The idle class has no class data. It will print a warning if
a prio argument is given for it, since this will be ignored.
Output for idle class will not contain prio data.

So change to:

Code: [Select]

//      exec("du -sc {$path} | grep -i '{$path}'", $out);
        exec("ionice -c3 du -s {$path}", $out);

There is no priority to idle class, all warnings stop.  There is also another ionice exec in that PHP file that needs changed. (-n7 removed)  will find it, and advise.

[chrisf]

These are the changes:

Code: [Select]


//      exec("du -sc {$path} | grep -i '{$path}'", $out);
            exec("ionice -c3 du -s {$path}", $out);
            $os  = preg_replace("/\s+/", ":", $out[0]);
            $t = str_replace(":{$path}", "", $os);

And

From:

Code: [Select]

//      exec($cmd);
        exec("ionice -c3 -n7 " . $cmd);

To:

Code: [Select]

//      exec($cmd);
        exec("ionice -c3 " . $cmd)   ;

All warnings will stop and io set to idle class   please update your code so updates won't break

[chrisf]

Scavenge still stops, doesn't complete, hangs, whatever you want to call it, right here:

In client-admin backup_flag equals 0 and limit is On
In client-admin dns_manage_flag equals 0 and limit is On
In client-admin webhosting_flag equals 0 and limit is On
Dead Quota: In client-admin clientdisk_usage equals 275
Dead Quota: In client-admin traffic_last_usage equals 11.9
Dead Quota: In client-admin mysqldb_usage equals 9.9

It just sits here - this can not be working properly.  I let it sit for 10 mins, then ctrl-c to get the command line.  Why?

[MRatWork]

I think there are something wrong with log file. Try delete certain log files inside /usr/local/lxlabs/kloxo/log and /var/log.

[chrisf]

On ALL my servers?   Which log files Mustafa?

[chrisf]

And this happens even on my fresh install.

[chrisf]

I have been emptying logs, but still seems to stop (hang) always at that spot.  I just reinstalled and tried.  Same result.

Please advise?

[MRatWork]

Did you try 'sh /script/fixtraffic --day=360'?.

[chrisf]

Yes, fixtraffic completed.  Appears scavenge runs at the end of that script and completes.  However, if I run sh /script/scavenge it stops or hangs.  It never completes.  I have tried this across 6 servers.  All hang.  Can you reproduce?

In newest KloxoMR update, changes were not made in /usr/local/lxlabs/kloxo/httpdocs/htmllib/lib/linuxfslib.php as stated above from ionice documentation.  You changed one, but left it doubled up, so it runs twice.  And second ionice reference. Was left untouched.  Please review above.

Other KloxoMR users, please run sh /script/scavenge and advise as to if it completes or hangs.

Thank you