MRatWork Forum by Mustafa Ramadhan

Sawo Project - Kloxo-MR Discussions => Kloxo-MR Tips and Tricks => Topic started by: chrisf on 2014-01-17, 21:41:58

Title: sendmail userid usage limits [script v1.0b]
Post by: chrisf on 2014-01-17, 21:41:58
Major battle we must fight as hosting providers are spammers ruining our servers IP credibility.  KloxoMR uses qmail-toaster, which is a very efficient MTA, it lacks a few features.

Major problem is sendmail.  It can forge domain, to/from, the only thing that is a result... IP ban.

Sendmail limiter is a wrapper script which uses mysql to keep track of sendmail, and limit by hour, half day, and full day.  You can even completely ban a client/user from sendmail.

Full logging abilities, even able to let you know which directory the script which is sending mail is at.

You can use for just limiting,  just logging, or both. ;)

Install is easy with my auto installer, you have a few things to do manually (mysql database, cron).

SSH:
Code: [Select]
cd /tmp
wget http://cc-server1.us/sendmail_wrapper.zip
unzip sendmail_wrapper.zip

Read sendmail-chrisf.txt, full instructions :)

Please report performance issues or bugs.  I am developing other scripts to help manage the client settings, and a auto group script to increase group by days active.

:)
Title: Re: sendmail userid usage limits [script v1.0b]
Post by: Spacedust on 2014-01-18, 00:25:34
Testing tomorrow :)
Title: Re: sendmail userid usage limits [script v1.0b]
Post by: Spacedust on 2014-01-18, 19:53:42
I've installed it. I can tell if it's working, because there is no much mails in queue.
Title: Re: sendmail userid usage limits [script v1.0b]
Post by: Spacedust on 2014-01-18, 20:13:10
It seems to be working:

The following clients have violated sender limits: (Hourly ~ groups 1 and 2)  1:gregor76:7848 (773/50) - 2014-01-18 20:02:24

Does this mean user tried to send 773 mails while he is actually able to send 50 mails per hour ?
Title: Re: sendmail userid usage limits [script v1.0b]
Post by: Spacedust on 2014-01-18, 22:56:08
This script is sending me mails each hour which is a bit annoying.
Title: Re: sendmail userid usage limits [script v1.0b]
Post by: chrisf on 2014-01-18, 23:51:03
/usr/sbin/sendmailwrapper/smw_config.pl
Turn reporting off, emails will stop. ;)  also, with reporting on, it only sends an email if there are violators.

Yes, that means he TRIED to send 773 for that hour, with 50 being the maximum per hour, and any mail over that was dropped and not sent.

Look at /var/log/sendmail-limits.log for more information.
Title: Re: sendmail userid usage limits [script v1.0b]
Post by: Spacedust on 2014-01-19, 00:37:15
Great. It works fantastic !
Title: Re: sendmail userid usage limits [script v1.0b]
Post by: Spacedust on 2014-02-03, 23:48:31
Kloxo shouldn't be blocked !

/usr/local/lxlabs/kloxo/httpdocs - lxlabs:501 0/100 (OK)
Title: Re: sendmail userid usage limits [script v1.0b]
Post by: chrisf on 2014-02-04, 01:41:19
this is new because of spawn-fcgi, kloxo was sending under root.

I have been so busy programming,  I need to write a commandline utility.  goto the database and under kloxo set the ignore flag to 1.

I will update the sendmail-wrapper as well. ;)
Title: Re: sendmail userid usage limits [script v1.0b]
Post by: Spacedust on 2014-02-04, 18:41:02
this is new because of spawn-fcgi, kloxo was sending under root.

I have been so busy programming,  I need to write a commandline utility.  goto the database and under kloxo set the ignore flag to 1.

I will update the sendmail-wrapper as well. ;)


That's why I don't get any e-mails from Kloxo now. This is really urgent, because I'm sending messages to my customers via Kloxo. Do you confirm this ?
Title: Re: sendmail userid usage limits [script v1.0b]
Post by: MRatWork on 2014-02-04, 19:56:29
@Spacedust,

Try change 'phpcgi_user="lxlabs"' in /etc/init.d/kloxo
Title: Re: sendmail userid usage limits [script v1.0b]
Post by: Spacedust on 2014-02-04, 20:05:08
@Spacedust,

Try change 'phpcgi_user="lxlabs"' in /etc/init.d/kloxo

It's already set this way.

sendmail_limits.log shows this:

[Tue, 04 Feb 2014 20:04:14 +0100] /usr/local/lxlabs/kloxo/httpdocs - lxlabs:501 1/100 (OK)

but there is no mail and maillog shows nothing :/
Title: Re: sendmail userid usage limits [script v1.0b]
Post by: MRatWork on 2014-02-04, 20:14:30
So, it's not related to spawn-fcgi.
Title: Re: sendmail userid usage limits [script v1.0b]
Post by: chrisf on 2014-02-04, 21:10:25
@spacedust, it says in that log it is (OK) which means it got sent.  I am checking and fixing right now.
Title: Re: sendmail userid usage limits [script v1.0b]
Post by: Spacedust on 2014-02-04, 21:59:30
@spacedust, it says in that log it is (OK) which means it got sent.  I am checking and fixing right now.

So why no logs in maillog for such e-mail and no e-mail received ? (spam folder checked).
Title: Re: sendmail userid usage limits [script v1.0b]
Post by: chrisf on 2014-02-04, 22:55:56
here is the fix for lxlabs:
SSH:
Code: [Select]
cd /usr/sbin/sendmailwrapper
rm -f sendmail-wrapper
wget http://cc-server1.us/sendmail-wrapper
chmod 755 sendmail-wrapper

so root and lxlabs will never be limited.  I can't reproduce mail not delivering, tell me the steps.


* UPDATING the zip so if fresh install this step is not needed
Title: Re: sendmail userid usage limits [script v1.0b]
Post by: Spacedust on 2014-02-05, 13:46:42
here is the fix for lxlabs:
SSH:
Code: [Select]
cd /usr/sbin/sendmailwrapper
rm -f sendmail-wrapper
wget http://cc-server1.us/sendmail-wrapper
chmod 755 sendmail-wrapper

so root and lxlabs will never be limited.  I can't reproduce mail not delivering, tell me the steps.


* UPDATING the zip so if fresh install this step is not needed

It doesn't work.

To reproduce - add new customer with a valid e-mail.

Go to admin home -> messages -> select this customer -> click Send mail field -> enter subject and text description and click Send.

sendmail_limits.log shows:

[Wed, 05 Feb 2014 13:44:17 +0100] /usr/local/lxlabs/kloxo/httpdocs - lxlabs:501 0/100 (OK)

maillog does not show anything

and no mail !!!
Title: Re: sendmail userid usage limits [script v1.0b]
Post by: chrisf on 2014-02-05, 17:31:07
Update again, root and lxlabs will never be blocked, or even hold a count.

Second, this is a KloxoMR bug, it is not sending the message to sendmail.  I turned on sendmail logging, (4) and the first entry is from panel, but it didn't send a message with it, right after is csf sending mail concerning my SSH login.

So, this is KloxoMR bug, not my script.


[Wed, 05 Feb 2014 11:20:36 -0500] /usr/local/lxlabs/kloxo/httpdocs - lxlabs:498 / (OK)
sendmail called using args: -t -i

[Wed, 05 Feb 2014 11:20:53 -0500] /root - root:0 / (OK)
sendmail called using args: -f root -t
From: root
To: <removed>@gmail.com
Subject: lfd on server: SSH login alert for user root from x.x.x.x (US/United States)

The second message gets delivered, because there IS a message, the panel message contains no body.  It doesn't have anything to send, therefore, no mail logs.
Title: Re: sendmail userid usage limits [script v1.0b]
Post by: MRatWork on 2014-02-05, 18:07:49
The question is how about without 'script v1.0b'. Also didn't able to send mail?.

What about server setting. Follow http://forum.mratwork.com/kloxo-mr-tips-and-tricks/(tip)-how-to-setup-qmail-on-kloxo-mr/ or not?.

So, don't easy to judge as Kloxo-MR bug.
Title: Re: sendmail userid usage limits [script v1.0b]
Post by: Spacedust on 2014-02-05, 18:35:19
You really need to fix this somehow. It never happened in the old versions. The only concern was no polish special characters.
Title: Re: sendmail userid usage limits [script v1.0b]
Post by: chrisf on 2014-02-05, 18:48:50
True.  I removed wrapper and admin mail was sent.

I am investigating.  Wrapper works with joomla, WordPress, dolphin, custom php scripts, and other cms scripts tested.

My script logs all arguments passed to it.  I am investigating.

@spacedust, if you want to disable for now,
SSH:
Code: [Select]
cd /usr/sbin
rm -f sendmail
ln -sf ../../var/qmail/bin/sendmail sendmail

I will figure out why any php script I try works, but KloxoMR doesn't.

Title: Re: sendmail userid usage limits [script v1.0b]
Post by: chrisf on 2014-02-05, 22:27:16
I have not been able to get ANY other php script to hang on sending mail.  KloxoMR just won't send.  I do not want to have sendmail limiting off so this is the quickest fix.

Code: [Select]
cd /usr/sbin
rm -f sendmail
ln -sf ../../var/qmail/bin/sendmail sendmail_nowrap
ln -sf sendmailwrapper/sendmail-wrapper sendmail
vim /opt/php52s/etc/php.ini

find
Code: [Select]
[mail function]
SMTP = localhost
smtp_port = 25

Change to
Code: [Select]
[mail function]
SMTP = localhost
smtp_port = 25
sendmail_path = /usr/sbin/sendmail_nowrap -t -i

Then
Code: [Select]
sh /script/restart-all

That will force mail sent through KloxoMR to go straight through sendmail, without the wrapper.

Still investigating as to why KloxoMR is only script not sending through the wrapper.  Mustafa, what file does KloxoMR use to send these messages?
Title: Re: sendmail userid usage limits [script v1.0b]
Post by: Spacedust on 2014-02-05, 22:34:41
It works again... Thanks :)
Title: Re: sendmail userid usage limits [script v1.0b]
Post by: Spacedust on 2014-02-12, 12:57:32
I think I need to order additional IP for mail server usage, but I need this script to help stop these abusing customers or it will be banned again by Google...
Title: Re: sendmail userid usage limits [script v1.0b]
Post by: Spacedust on 2014-02-17, 21:51:55
I had to restore normal sendmail from my backup server because I was unable to send mails from php scripts ;)
Title: Re: sendmail userid usage limits [script v1.0b]
Post by: chrisf on 2014-02-18, 00:31:36
no scripts?  Did the test script work? 
Title: Re: sendmail userid usage limits [script v1.0b]
Post by: Spacedust on 2014-02-18, 12:55:45
no scripts?  Did the test script work? 

I was talking about your last instructions how to disable it.
Title: Re: sendmail userid usage limits [script v1.0b]
Post by: chrisf on 2014-02-18, 17:12:52
I am rewriting the script now and should have it ready by tonight, tomorrow.  I am going to use a dual script approach and try to eliminate any hangups.

I'll post here when completed.
Title: Re: sendmail userid usage limits [script v1.0b]
Post by: Spacedust on 2014-02-18, 18:48:55
I am rewriting the script now and should have it ready by tonight, tomorrow.  I am going to use a dual script approach and try to eliminate any hangups.

I'll post here when completed.

Finally ;) I need to lock some customers otherwise I will get another ban on gmail.com ;)
Title: Re: sendmail userid usage limits [script v1.0b]
Post by: bigdigillc on 2015-01-12, 10:15:37
It seems my servers are sending out mail that isn't passing through the sendmail wrapper. Any thoughts on how this might be happening?
Title: Re: sendmail userid usage limits [script v1.0b]
Post by: chrisf on 2015-01-12, 20:02:07
I am sorry for my absence from posting and general help, I had family problems that consumed my time.

Yes, Spacedust posted a script which a hacker was using on his server.  It was checking for qmail and then sending mail directly through qmail.

I had a fix, but again, posting the fix gives the hacker the key as well.  I am working on a fix now.  I actually have it done, just need to test a bit more.  The script is going to essentially rename the qmail sendmail original to a random name, and save this into the new sendmail wrapper conf automatically.  This will stop any direct use of qmail sendmail.  I also optimized the code again, since I hit about 2000 mail in/out for a legitimate customer, and found a place in the code that needed it.

I will post the new files and information as soon as possible.  Again, sorry for the delay and my absence.
Title: Re: sendmail userid usage limits [script v1.0b]
Post by: Spacedust on 2015-03-23, 12:39:37
Chris - links are down !