MRatWork Forum by Mustafa Ramadhan

Sawo Project - Kloxo-MR Discussions => Kloxo-MR Development => Topic started by: fossxplorer on 2017-03-30, 09:38:44

Title: Issues with slave
Post by: fossxplorer on 2017-03-30, 09:38:44
I've now installed a new slave and while testing a around, i can see there are issues with the php56s:
 [30-Mar-2017 08:39:34 Europe/London] PHP Warning:  PHP Startup: Unable to load dynamic library '/opt/php56s/usr/lib/php/modules/sphinx.so' - libsphinxclient-0.0.1.so: cannot open shared object file: No such file or directory in Unknown on line 0
[30-Mar-2017 08:39:36 Europe/London] PHP Warning:  PHP Startup: Unable to load dynamic library '/opt/php56s/usr/lib/php/modules/sphinx.so' - libsphinxclient-0.0.1.so: cannot open shared object file: No such file or directory in Unknown on line 0
[root@odcp01 ~]# ll /opt/php56s/usr/lib/php/modules/sphinx.so
-rwxr-xr-x. 1 root root 50336 Jun 16  2016 /opt/php56s/usr/lib/php/modules/sphinx.so
[root@odcp01 ~]# php56s-cli -v
PHP 5.6.30 (cli) (built: Jan 19 2017 10:06:11)
Copyright (c) 1997-2016 The PHP Group
Zend Engine v2.6.0, Copyright (c) 1998-2016 Zend Technologies
[root@odcp01 ~]#

The file is there, but might be compatibility issue with the running php56s version.
Btw, php56s is the "special" PHP version used by Kloxo-MR for administration with lxphp right?

This is a "slave", but i did install it as master and then added it as slave from the real master, i.e i did not run make-slave as you've recommended earlier.





Title: Re: Issues with slave
Post by: MRatWork on 2017-03-30, 10:25:40
Rename sphinx.ini to sphinx_unused.nonini inside /opt/php56s/etc/php.d.
Title: Re: Issues with slave
Post by: fossxplorer on 2017-03-30, 11:12:17
Thnanks.
I have all kinds of problems. I've used 2 days to try to get this up. IIRC, it worked an year ago, but i had to take down that server => lost that config :(
This is a slave running inside an LXC container, does it matter?

It seems, something works when controlling from master, while other things don't.
E.g you can't "webserver configure" from master, nothing changes. Also, "Switch program" doesn't work well from master.
But problem is, some command like /scripts/fixweb --server slave01 must be run from master, it won't directly inside slave, since the client database is at the master.

Should i try to create a new slave with running make-slave? Will that work better from master?


Title: Re: Issues with slave
Post by: fossxplorer on 2017-03-30, 11:15:43
Also, often, using nginx or hiawathaproxy, Apache  is trying to listen to port 80 instead of 30080.
[root@odcp01 ~]# sh /script/restart-web

Stopping php70m-fpm (PHP Used):                            [  OK  ]
Starting php70m-fpm (PHP Used):                            [  OK  ]
-------------------------------------------------------------------

Stopping php56m-fpm (Multiple Php):                        [  OK  ]
Starting php56m-fpm (Multiple Php):                        [  OK  ]
-------------------------------------------------------------------

Stopping php70m-fpm (Multiple Php):                        [  OK  ]
Starting php70m-fpm (Multiple Php):                        [  OK  ]
-------------------------------------------------------------------

*** Process for NGINX service ***
  - Process: restart
Stopping nginx:                                            [FAILED]
Starting nginx:                                            [  OK  ]
-------------------------------------------------------------------

*** Process for HTTPD service ***
  - Process: restart
Stopping httpd:                                            [FAILED]
Starting httpd: (98)Address already in use: AH00072: make_sock: could not bind to address [::]:80
(98)Address already in use: AH00072: make_sock: could not bind to address 0.0.0.0:80
no listening sockets available, shutting down
AH00015: Unable to open logs
                                                           [FAILED]
-------------------------------------------------------------------

- For help, type '/script/restart-web [--help|-h]'
[root@odcp01 ~]#
Title: Re: Issues with slave
Post by: MRatWork on 2017-03-30, 11:20:41
Try using 'sh /script/fixweb --server=all'.
Title: Re: Issues with slave
Post by: fossxplorer on 2017-03-30, 11:54:45
[root@odcp01 ~]# sh /script/fixweb --server=all

- For help, type '/script/fixweb [--help|-h]'

Fixing Web server config

*** Process Time: 00:00:00:00.303 (dd:hh:mm:ss:xxxxxx) ***

So run from the slave, it doesn't do anything.

I can't run it on master, as it's in production, but i need a PROPER way of having and administering a slave.
Only 1 master is not enough for us anymore!

What do you suggest?



Title: Re: Issues with slave
Post by: MRatWork on 2017-03-30, 12:00:48
See 'sh /script/fixweb --help'.
Title: Re: Issues with slave
Post by: fossxplorer on 2017-03-30, 12:47:36
Yes, i've checked that ofc :)
So this MUST be run from master . Won't work on the slave.

Do you still recommend using the slave without running make-slave?

We really need to get this documented. Master-slave is quite an important feature for anyone looking to use the control panel for anything serious :)
I can document it, but problem is that it's quite inconsistent the way it work right now.

Title: Re: Issues with slave
Post by: fossxplorer on 2017-03-30, 12:49:27
I'm installing another slave to test again and i see:
iptables: Unloading modules:  iptable_filter iptable_mangle[FAILED]_raw iptable_security iptable_nat iptable_filter iptable_mangle iptable_raw iptable_security iptable_nat ip_tables
/usr/local/lxlabs/kloxo/install/step2.inc: line 24: mkpasswd: command not found
>>> Modified /etc/sysctl.conf <<<

Iptables error is expected as this in inside one LXC container. But what's mkpasswd error about?
Title: Re: Issues with slave
Post by: MRatWork on 2017-03-30, 13:22:23
mkpasswd is command for generate password. I found this command in centos 5/6/7 under OpenVZ and KVM. Don't know about LXC.

Try 'yum install expect -y'.
Title: Re: Issues with slave
Post by: fossxplorer on 2017-03-30, 13:30:30
Package expect-5.44.1.15-5.el6_4.x86_64 already installed and latest version
[root@odcp01 ~]# type mkpasswd 
mkpasswd is /usr/bin/mkpasswd

I think it was installed later during the installation.

But now that i've installed a new slave (without make-slave), and added to master, i still can't e.g switch program to Nginxproxy FROM the master UI. When i try, i see 2 php<defunct> processes in the slave using top. I think lxphp commands are crashing.
When i do changes from the slave's UI, i *do also see such crashes*.
I've never seen this before.

Title: Re: Issues with slave
Post by: fossxplorer on 2017-03-30, 13:48:48
[root@mail ~]# sh /script/fixweb --server=slave01
- For help, type '/script/fixweb [--help|-h]'

Fixing Web server config
- 'defaults' pages at 'slave01'
- remove all domains configs at 'slave01'
- 'slaveuser.no' ('slaveuser') at 'slave01'

*** Process Time: 00:00:00:02.2110 (dd:hh:mm:ss:xxxxxx) ***

[root@mail ~]# sh /script/fixphp --server=slaveo01

..
it takes ALL , incl. master's web pages!
So fixweb works with slave, but not fixphp, FROM the master!
Title: Re: Issues with slave
Post by: MRatWork on 2017-03-30, 13:53:35
Did you check 'sh /script/fixphp --help'?
Title: Re: Issues with slave
Post by: fossxplorer on 2017-03-30, 13:59:25
Yes, ofc :) It's basically the same arguments as fixweb!
Also, now that's ive manged the slave from it's own UI:
Shutting down MySQL. SUCCESS!
Starting MySQL.170330 13:55:59 mysqld_safe Logging to '/var/lib/mysql/odcp01.linuxwonders.com.err'.
170330 13:55:59 mysqld_safe Starting mysqld daemon with databases from /var/lib/mysql
 SUCCESS!
-------------------------------------------------------------------

Stopping named: .                                          [  OK  ]
Starting named:                                            [  OK  ]
-------------------------------------------------------------------
Stopping PowerDNS authoritative nameserver: Scheduling exit

Stopping php-fpm (PHP Used):                               [  OK  ]
Starting php-fpm (PHP Used):                               [  OK  ]
-------------------------------------------------------------------

Stopping spawn-fcgi:                                       [FAILED]
Starting spawn-fcgi:                                       [  OK  ]
-------------------------------------------------------------------

Stopping nginx:                                            [  OK  ]
Starting nginx:                                            [  OK  ]
-------------------------------------------------------------------

Stopping httpd:                                            [FAILED]
Starting httpd: (98)Address already in use: AH00072: make_sock: could not bind to address [::]:80
(98)Address already in use: AH00072: make_sock: could not bind to address 0.0.0.0:80
no listening sockets available, shutting down
AH00015: Unable to open logs
                                                           [FAILED]
-------------------------------------------------------------------


*** Process for QMAIL service ***
Stopping qmail-toaster: svscan qmail logging.
qmail-send: no process killed
Starting qmail-toaster: svscan.
-------------------------------------------------------------------

- For help, type '/script/restart-mail [--help|-h]'
-------------------------------------------------------------------

Stopping pure-ftpd:                                        [  OK  ]
Starting pure-ftpd:                                        [  OK  ]
-------------------------------------------------------------------
*** Restart services - END ***


*** Process Time: 00:00:01:45.105766 (dd:hh:mm:ss:xxxxxx) ***


REMARK:
 - run 'sh /script/fixdnsaddstatsrecord' to make sure dns records including
   'A record' for 'stats'
 - 'http://domain.com/stats' will be redirect to 'http://stats.domain.com'
[root@odcp01 ~]#

Apache's conf with proxy is screwed up!
I'll do tests with KVM guest rather than LXC container. Ufff, this has taken me 2 days now!





Title: Re: Issues with slave
Post by: MRatWork on 2017-03-30, 14:29:35
Inform here your 'dir -l /usr/local/lxlabs/kloxo/etc/slavedb'
Title: Re: Issues with slave
Post by: fossxplorer on 2017-03-30, 14:50:20
Slave:
[root@odcp01 ~]# dir -l /usr/local/lxlabs/kloxo/etc/slavedb
total 12
-rw-r--r--. 1 lxlabs root  79 Mar 30 13:54 contactemail
-rw-r--r--. 1 lxlabs root  89 Mar 30 13:54 dbadmin
-rw-r--r--. 1 lxlabs root 186 Mar 30 13:54 driver

Master:
[root@mail ~]# dir -l /usr/local/lxlabs/kloxo/etc/slavedb
total 12
-rw-r--r--. 1 lxlabs root  79 Mar 29 20:16 contactemail
-rw-r--r--. 1 lxlabs root  86 Mar 29 20:16 dbadmin
-rw-r--r--. 1 lxlabs root 186 Mar 29 20:58 driver


Title: Re: Issues with slave
Post by: MRatWork on 2017-03-30, 15:18:27
inform here 'cat /usr/local/lxlabs/kloxo/etc/slavedb/driver'.
Title: Re: Issues with slave
Post by: fossxplorer on 2017-03-30, 15:29:48
root@mail ~]#  cat /usr/local/lxlabs/kloxo/etc/slavedb/driver
O:6:"Remote":1:{s:4:"data";a:6:{s:3:"web";s:10:"nginxproxy";s:8:"webcache";s:4:"none";s:3:"dns";s:4:"bind";s:4:"spam";s:10:"bogofilter";s:4:"pop3";s:7:"courier";s:4:"smtp";s:5:"qmail";}}
Title: Re: Issues with slave
Post by: MRatWork on 2017-03-30, 16:21:10
Inform here 'cat /opt/configs/apache/conf/defaults/init.conf'.
Title: Re: Issues with slave
Post by: fossxplorer on 2017-03-30, 17:02:31
[root@odcp01 ~]# cat /opt/configs/apache/conf/defaults/init.conf
### begin - web of initial - do not remove/modify this line


<IfVersion < 2.4>
   Define global::port 80
   Define global::portssl 443
   Define global::ip *

   Define port ${global::port}
   Define portssl ${global::portssl}
   Define ip ${global::ip}
</IfVersion>

<IfVersion >= 2.4>
   Define port 80
   Define portssl 443
   Define ip *
</IfVersion>

Listen ${ip}:${port}
Listen ${ip}:${portssl}

<IfVersion < 2.4>
   NameVirtualHost ${ip}:${port}
   NameVirtualHost ${ip}:${portssl}
</IfVersion>


### end - web of initial - do not remove/modify this line

This is wrong when in proxy mode ofc.

I've now confirmed using KVM guest, same problem here too! Master-slave has exact the same issue.

Title: Re: Issues with slave
Post by: fossxplorer on 2017-03-31, 10:34:58
It seems the following is not being checked somehow or not working:
https://github.com/mustafaramadhan/kloxo/blob/dev/kloxo/file/apache/tpl/defaults.conf.tpl#L173

Are you able to find out why? We really need a slave up and running.


Title: Re: Issues with slave
Post by: MRatWork on 2017-03-31, 10:50:50
Nothing wrong with defaults.conf.tpl code.

Solution for your issue is copy default.conf.tpl to custom.defaults.conf.tpl and change:
Code: [Select]
if ($reverseproxy) {
$ports[] = '30080';
$ports[] = '30443';
} else {
if (($webcache === 'none') || (!$webcache)) {
$ports[] = '80';
$ports[] = '443';
} else {
$ports[] = '8080';
$ports[] = '8443';
}
}

to:
Code: [Select]
/*
if ($reverseproxy) {
$ports[] = '30080';
$ports[] = '30443';
} else {
if (($webcache === 'none') || (!$webcache)) {
$ports[] = '80';
$ports[] = '443';
} else {
$ports[] = '8080';
$ports[] = '8443';
}
}
*/
$ports[] = '30080';
$ports[] = '30443';

and then run fixweb again.
Title: Re: Issues with slave
Post by: fossxplorer on 2017-03-31, 14:01:18
Ok, i did the changes, but still init.conf is wrong! Not using 30080 and 300443!


Trying to change to PHP7 on the slave directly, i get:
[root@odcp01 defaults]# tail -f /usr/local/lxlabs/kloxo/log/php-error.log
[31-Mar-2017 12:00:31 UTC] PHP Warning:  PHP Startup: Unable to load dynamic library '/opt/php56s/usr/lib/php/modules/gearman.so' - /opt/php56s/usr/lib/php/modules/gearman.so: undefined symbol: zend_objects_store_get_refcount in Unknown on line 0
[31-Mar-2017 12:00:31 UTC] PHP Warning:  PHP Startup: Unable to load dynamic library '/opt/php56s/usr/lib/php/modules/geoip.so' - /opt/php56s/usr/lib/php/modules/geoip.so: undefined symbol: zval_used_for_init in Unknown on line 0
[31-Mar-2017 12:00:31 UTC] PHP Warning:  PHP Startup: Unable to load dynamic library '/opt/php56s/usr/lib/php/modules/igbinary.so' - /opt/php56s/usr/lib/php/modules/igbinary.so: undefined symbol: zval_used_for_init in Unknown on line 0
[31-Mar-2017 12:00:31 UTC] PHP Warning:  PHP Startup: Unable to load dynamic library '/opt/php56s/usr/lib/php/modules/json.so' - /opt/php56s/usr/lib/php/modules/json.so: undefined symbol: zend_object_store_get_object in Unknown on line 0
[31-Mar-2017 12:00:31 UTC] PHP Warning:  PHP Startup: Unable to load dynamic library '/opt/php56s/usr/lib/php/modules/krb5.so' - /opt/php56s/usr/lib/php/modules/krb5.so: undefined symbol: zval_used_for_init in Unknown on line 0
[31-Mar-2017 12:00:31 UTC] PHP Warning:  PHP Startup: Unable to load dynamic library '/opt/php56s/usr/lib/php/modules/oauth.so' - /opt/php56s/usr/lib/php/modules/oauth.so: undefined symbol: zval_used_for_init in Unknown on line 0
[31-Mar-2017 12:00:31 UTC] PHP Warning:  PHP Startup: Unable to load dynamic library '/opt/php56s/usr/lib/php/modules/rrd.so' - /opt/php56s/usr/lib/php/modules/rrd.so: undefined symbol: zend_object_store_get_object in Unknown on line 0
[31-Mar-2017 12:00:31 UTC] PHP Warning:  PHP Startup: Unable to load dynamic library '/opt/php56s/usr/lib/php/modules/ssh2.so' - /opt/php56s/usr/lib/php/modules/ssh2.so: undefined symbol: zval_used_for_init in Unknown on line 0
[31-Mar-2017 12:00:31 UTC] PHP Warning:  PHP Startup: Unable to load dynamic library '/opt/php56s/usr/lib/php/modules/twig.so' - /opt/php56s/usr/lib/php/modules/twig.so: undefined symbol: zval_used_for_init in Unknown on line 0
[31-Mar-2017 12:00:31 UTC] PHP Warning:  PHP Startup: Unable to load dynamic library '/opt/php56s/usr/lib/php/modules/yaml.so' - /opt/php56s/usr/lib/php/modules/yaml.so: undefined symbol: zval_used_for_init in Unknown on line 0



Has it remove modules for php56s mistakenly?

Title: Re: Issues with slave
Post by: fossxplorer on 2017-03-31, 14:33:27
[31-Mar-2017 13:36:52 Europe/London] PHP Fatal error:  Call to a member function fetch_array() on boolean in /usr/local/lxlabs/kloxo/bin/fix/fix-qmail-assign.php on line 31


*** Process for SPAWN-FCGI service ***
  - Process 'spawn-fcgi': restart
Stopping spawn-fcgi:                                       [FAILED]
Starting spawn-fcgi:                                       [FAILED]
-------------------------------------------------------------------

*** Process for NGINX service ***
  - Process: restart
Stopping nginx:                                            [  OK  ]
Starting nginx:                                            [  OK  ]
-------------------------------------------------------------------

*** Process for HTTPD service ***
  - Process: restart
Stopping httpd:                                            [FAILED]
Starting httpd: (98)Address already in use: AH00072: make_sock: could not bind to address [::]:80
(98)Address already in use: AH00072: make_sock: could not bind to address 0.0.0.0:80
no listening sockets available, shutting down
AH00015: Unable to open logs
                                                           [FAILED]
-------------------------------------------------------------------

- For help, type '/script/restart-web [--help|-h]'
[root@odcp01 php]#