[HAIWATHA] How to stop Haiwatha troubles!

[Kloxo-DR]

Hello Mustafa,

For some reasons Haiwatha got messy. ALL THE WEBSITES WERE SHOWING HAIWATHA "INSTALLATION SUCCESSFUL" PAGE.

So I issued:

# service httpd start

Response:

[root@domain ~]# service httpd start
httpd start (98) Address already in use: make_sock: could not bind to address   [::]:80
(98)Address already in use: make_sock: could not bind to address 0.0.0.0:80 no listening sockets available, shutting down Unable to open logs [FAILED]

I did:

# sudo netstat -ltnp | grep ':80'

It gave:

tcp     0     0 0.0.0.0:80     0.0.0.0:*     LISTEN     1187/hiawatha

Then:

# kill 1187

Then:

#service httpd start

Only then httpd started. /script/restart-all gave - ofcourse - the same result because httpd could not bind to port 80, which was occupied by Haiwatha.

It there a way to have Haiwatha under control or not to do this problem?

[MRatWork]

Still something wrong with disable hiawatha.

Just running 'chkconfig hiawatha off; sh /script/restart-all'.

This issue appear when update/reinstall/downgrade hiawatha

[Kloxo-DR]

Hi Mustafa,

Thats true. I got with /script/restart-all:

httpd terminating:                                             [  OK  ]
httpd starting:                                             [  OK  ]
Stopping Hiawatha web server:                              [FAILED]
Starting Hiawatha web server: Error binding 0.0.0.0:80
                                                           [FAILED]
Stopping qmail-toaster: svscan qmail logging.
qmail-send: No Process interrupted

I did not make an update/reinstall/downgrade hiawatha. Something happened without my knowledge.

Perhaphs your suggestion could be integrated in the /script/restart-all so that it does not fail.

[Kloxo-DR]

Hello Mustafa,

It is the third time all the webpages stopped working.

Why did you introduce starting of Haiwatha to initiate at boot process in sh /script/cleanup togather and parallel with httpd?

This was not before. Perhaphs that the problem.

[MRatWork]

Let me tell you something.

Hiawatha (not special; just generic/regular) using by Kloxo-MR itself and also as web server in 6.5.1.

When hiawatha update, it's also set hiawatha service as 'on'. Restart-all script detect where services in 'state' status (on or off). If 'on', restart-all (also restart-web in web context) will try to restart services.

Cleanup just checking hiawatha as webserver or not and if not as webserver cleanup will running 'chkconfig hiawatha off'.

[Kloxo-DR]

Mustafa,

Can you answer me one question:

Who configured installation scripts of Haiwatha on my server?

THATS YOU! And now this is not working.

[chrisf]

@Kloxo-DR, it is easy to get frustrated,  but Mustafa answered your question.  When the last updates there was a small
 Hiawatha update.  (Yum update)  after this Hiawatha have 'on' status due to the update.   Nothing Mustafa did.  I manage 5 production servers now, and ALL of them did this after last update.  Takes just a few key strokes to fix.

Code: [Select]

chkconfig hiawatha off
sh /script/restart-all

If httpd can't bind to the port due to a conflict at /script/cleanup, kill the pid like you did, then execute above commands.  Always execute updates on a non production server first.

Being system admin is a learning experience for all, and I know you were frustrated, but Mustafa is the holder of the keys if you choose to use KloxoMR, and he in my opinion is doing a great job.  I am currently having all kinds of conflicts and problems on my new server trying to bring it online with KloxoMR, however, it is with a different company and the configuration is slightly different, but conflict after conflict.  I took my test server and fresh installed, mariadb, no problems.  What a headache!  But it is not Mustafa's fault.... he can not possibly know every single server configuration - but he helps, and he tells us what to do, sometimes better to listen than talk.

I enjoy working with KloxoMR.

[Spacedust]

My Hiawatha won't even start, but I've added port 81 to the config and chattr +i it so it won't ge overwritten.

/etc/init.d/hiawatha start
Starting Hiawatha web server: The default website has no hostname.
                                                           [FAILED]

[MRatWork]

My Hiawatha won't even start, but I've added port 81 to the config and chattr +i it so it won't ge overwritten.

/etc/init.d/hiawatha start
Starting Hiawatha web server: The default website has no hostname.
                                                           [FAILED]

Two option if not using hiawatha as webserver:
1. run 'service hiawatha stop' and then delete '/etc/rc.d/init.d/hiawatha'
2. run 'chkconfig hiawatha off; service hiawatha stop'

Yes, have a trouble if hiawatha updated/downgraded because hiawatha always create '/etc/rc.d/init.d/hiawatha' with 'on' state. Restart-all/restart-web all check this status in 'on' or 'off'. If 'on', restart process will restart hiawatha service. The same mechanism for other services.

[Kloxo-DR]

Hello Mustafa,

I did not make an explicit update of Haiwatha. I think this was done somethere during execution of any one of the /scripts.

Two option if not using hiawatha as webserver:
1. run 'service hiawatha stop' and then delete '/etc/rc.d/init.d/hiawatha'
2. run 'chkconfig hiawatha off; service hiawatha stop'

If this is possible to execute manually, why should this not be integrated somewhere in the right place to prevent this happening on production servers.

There must be a solution on a general level to not to allow things like this to happen in the future with Haiwatha, or any other software update.

Yes, have a trouble if hiawatha updated/downgraded because hiawatha always create '/etc/rc.d/init.d/hiawatha' with 'on' state. Restart-all/restart-web all check this status in 'on' or 'off'. If 'on', restart process will restart hiawatha service. The same mechanism for other services.

How about modifying a general code in Kloxo-MR /script/fix-all to integrate your suggestion.

If there is some garbage in the system somewhere, which creates some nonesense and forces system on production servers, then it should be able to fix it automatically.

Thats what fix-all is there and should fix it, right, everyone?

[Kloxo-DR]

Hello chrisf,

@Kloxo-DR, it is easy to get frustrated,  but Mustafa answered your question.

It is not the question of "easy to get frustrated" but a legal responsibility to run and manage a production webserver of a non-profit association, which I undertake. You know, may be, how association laws works in western countries.

If an official invitation by the association needs to go out and members of the association cannot access the webpage they are oblige to visit, then there may be lawsuits. Appearing before a court in western country is more frustrating than kissing feets of anyone here in this forum.

Just beware what you are saying, because it is not just emotions here. Just be reminded that there are rights and responsibilities attached with a webserver and email server, which Mustafa is indirectly associated with.

Ofourse, Mustafa undertakes no legal responsibility, but I do. I have legal responsibilities. I undertake legal obligation against others by using his code.

Further, chrisf, I did not agree, in any aspect, to undertake any beta testing of Mustafa's code on my production servers, where there are such serious legal responsibilities attached.

Qmail logs content of all emails with the -v switch, or something like that. I stopped this by -QVH switch. I have inserted -QVH switch in the qmail everywhere because Mustafa uses -v in all the run script to prevent message content logging. And chrisf, it is very wrong to not to recognize legal obligation of privacy laws of western countries.

When the last updates there was a small Hiawatha update.  (Yum update)  after this Hiawatha have 'on' status due to the update.

I have not installed Haiwatha as a webserver.

I am using Kloxo-MR 6.5.0f. If I want to change Haiwatha webserver on 6.5.0f then it will not change to it and give a notice that it is not ready for use with 6.5.0f.

Haiwatha is not ready for 6.5.0f, so I use apache.

Nothing Mustafa did.

Nothing Mustafa did NOT.

Why is Haiwatha getting updated on my webserver, when Kloxo-MR does not allow installation of Haiwatha at all?

Why is Haiwatha getting updated on my webserver, when I did not install Haiwatha at all?

I have no reason to blame Mustafa, or any one here. I am also respecting Mustafa's ambitious work. However, I want to make clear that no problems may enter on a production server in the velocity at which Mustafa is working and pouring THREE TIMES PROBLEMS THAN OFFICIAL KLOXO.

After i upgraded to kloxo-MR, I just have nightmares of studying everything to solve tiny little problems, which occurs when a programmer was not attentive or agreeing to those problems.

The stituation should be understood much more at a responsibility level than frustration of a user.

I was getting frusted learning fotran 77 and cobol  in 1983, when the size of my harddrive was 16 Megabytes. Since then, I learned not to get frustrated. I have no reason to get frustrated now. With this hint, is it clear to you, chrisf?

[aphanciale]

Kloxo-DR

Hiawatha shouldn't have been updated if you did not run yum update. However if you have installed and have yum-cron running your server will automatically update, thus update Hiawatha as it is installed to run Kloxo-MR.  For production servers I would not recommend using yum-cron as it can cause issues like this.

I would first check to see if you have it installed

Code: [Select]

yum list yum-cronCheck wether its in list of installed packages.

If it is installed I would then check if its running.

Code: [Select]

/etc/init.d/yum-cron status
If it is running I would stop it and prevent it from starting again.

Code: [Select]

/etc/init.d/yum-cron stop;chkconfig yum-cron off
Then in the future run "yum update" on development server before attempting on production server. Or if this isn't an option run in off hours were you can hopefully quickly recover the system. 

On a side note the Hiawatha update has been a plague for us as well and I am considering taking the time to move Kloxo-MR to use NGINX instead of Hiawatha for all the servers that I run Kloxo-MR on.