MRatWork Forum by Mustafa Ramadhan

Sawo Project - Kloxo-MR Discussions => Kloxo-MR Bugs and Requests => Topic started by: Spacedust on 2015-04-22, 19:35:34

Title: DMARC txt records does not work properly ?
Post by: Spacedust on 2015-04-22, 19:35:34
Please see: http://www.dnsinspect.com/forum.mratwork.com

"NOTICE: Domain doesn't have DMARC record. DMARC (Domain-based Message Authentication, Reporting & Conformance) helps reducing potential of email-based abuse such as spam and phishing e-mails."

I'm using pdns and also the same ;(

This is my entry:

Quote
v=DMARC1; p=none; ruf=mailto:d@ruf.agari.com; rua=mailto:d@rua.agari.com; fo=1
Title: Re: DMARC txt records does not work properly ?
Post by: Spacedust on 2015-04-22, 22:47:45
It seems DMARC is being messed up with SPF:

 Identical TXT records
FAIL: Your name servers returned different TXT records:

Quote
    37.187.123.56 ? v=spf1 a mx ~all v=spf1 a mx -all v=DMARC1; p=none; ruf=mailto:d@ruf.agari.com; rua=mailto:d@rua.agari.com; fo=1
    78.46.85.86 ? v=DMARC1; p=none; ruf=mailto:d@ruf.agari.com; rua=mailto:d@rua.agari.com; fo=1 v=spf1 a mx -all v=spf1 a mx ~all
    78.46.85.90 ? v=spf1 a mx -all v=spf1 a mx ~all v=DMARC1; p=none; ruf=mailto:d@ruf.agari.com; rua=mailto:d@rua.agari.com; fo=1

Only SPF records are compared, all name servers should return identical SPF records.
Identical SPF records
FAIL: Your name servers returned different SPF records:

Quote
    37.187.123.56 ? v=spf1 a mx -all v=spf1 a mx ~all
    78.46.85.86 ? v=spf1 a mx ~all v=spf1 a mx -all
    78.46.85.90 ? v=spf1 a mx ~all v=spf1 a mx -all

All name servers should return identical SPF records.
Title: Re: DMARC txt records does not work properly ?
Post by: MRatWork on 2015-04-22, 23:15:58
Try:
Code: [Select]
dig hostspectra.com TXT @ns1.hostspectra.com
dig hostspectra.com TXT @ns2.hostspectra.com
dig hostspectra.com TXT @ns3.hostspectra.com
Title: Re: DMARC txt records does not work properly ?
Post by: Spacedust on 2015-04-23, 00:34:57
See:

https://dmarcian.com/dmarc-inspector/hostspectra.com
http://www.dnsinspect.com/hostspectra.com#mail
Title: Re: DMARC txt records does not work properly ?
Post by: MRatWork on 2015-04-23, 01:24:30
No it's work.

Previous problem because using 'domain.com TXT "v=DMARC1;..."' instead '_dmarc.domain.com TXT "v=DMARC1;..."'

Will be update in next upload. Thanks.
Title: Re: DMARC txt records does not work properly ?
Post by: Spacedust on 2015-04-23, 13:34:24
No it's work.

Previous problem because using 'domain.com TXT "v=DMARC1;..."' instead '_dmarc.domain.com TXT "v=DMARC1;..."'

Will be update in next upload. Thanks.

Nice found ! Yes we were using just domain instead of _dmarc ;)
Title: Re: DMARC txt records does not work properly ?
Post by: Spacedust on 2015-04-23, 16:27:07
After readding DMARC now I got two records on all of my domains:

Quote
   __base__    TXT    -    v=DMARC1; p=none; ruf=mailto:d@ruf.agari.com; rua=...
   _dmarc    TXT    -    v=DMARC1; p=none; ruf=mailto:d@ruf.agari.com; rua=...

How to remove the upper one from all domains ?
Title: Re: DMARC txt records does not work properly ?
Post by: MRatWork on 2015-04-23, 16:36:54
Use dmarc via 'email auth' only (new default rui using 'admin@__base__') with click 'update all'.
Title: Re: DMARC txt records does not work properly ?
Post by: Spacedust on 2015-04-23, 18:47:29
Use dmarc via 'email auth' only (new default rui using 'admin@__base__') with click 'update all'.

I got my own entries in DNS template. Rebuild the template does not remove these records ;(
Title: Re: DMARC txt records does not work properly ?
Post by: MRatWork on 2015-04-23, 19:10:33
Use dmarc via 'email auth' only (new default rui using 'admin@__base__') with click 'update all'.

I got my own entries in DNS template. Rebuild the template does not remove these records ;(
Check dmarc for google.com. Their dmarc is more simple than Kloxo-MR do.
Title: Re: DMARC txt records does not work properly ?
Post by: Spacedust on 2015-04-23, 19:12:28
I know, DMARC is working fine for me.

Now I only want to remove these entries for all domains (about 1300 domains):

 __base__    TXT    -    v=DMARC1; p=none; ruf=mailto:d@ruf.agari.com; rua=...

Also please make DNS fields a bit longer... ;)
Title: Re: DMARC txt records does not work properly ?
Post by: MRatWork on 2015-04-23, 19:48:54
Create/modified template without entry for 'dmarc' and then rebuild in 'manage dns' with this template.
Title: Re: DMARC txt records does not work properly ?
Post by: Spacedust on 2015-04-23, 23:02:39
Create/modified template without entry for 'dmarc' and then rebuild in 'manage dns' with this template.

The wrong TXT entry remains - only new or the same entires are being modifed.

Quote
__base__    TXT    -    v=DMARC1; p=none; ruf=mailto:d@ruf.agari.com; rua=...
Title: Re: DMARC txt records does not work properly ?
Post by: Spacedust on 2015-04-23, 23:16:50
That's weird all other entries which does not exists in a new template (A, NS, FCNAME etc.) are removed but this wrong TXT is still untouched :(
Title: Re: DMARC txt records does not work properly ?
Post by: MRatWork on 2015-04-24, 05:45:29
I will try to emulate your setting.
Title: Re: DMARC txt records does not work properly ?
Post by: Spacedust on 2015-04-24, 12:11:57
I will try to emulate your setting.

Yes. Just add TXT record to the template with __base__ at the beginning:

Code: [Select]
v=DMARC1; p=none; ruf=mailto:d@ruf.agari.com; rua=mailto:d@rua.agari.com; fo=1
Rebuild the DNS for domain.

Then remove the entry in DNS template and add correct TXT record with _dmarc:

Code: [Select]
v=DMARC1; p=none; ruf=mailto:d@ruf.agari.com; rua=mailto:d@rua.agari.com; fo=1
Then rebuild again.

You will have two records then:

one with __base__ and one with _dmarc which is incorrect.
Title: Re: DMARC txt records does not work properly ?
Post by: MRatWork on 2015-04-24, 12:24:49
I am not sure but possible in dns table of kloxo database, 'dmac' and '_dmarc' save in the same variable (where variable is 'dmarc'). And it's make Kloxo-MR panel confuse with this situation.

By default, Kloxo not accept variable like '_dmarc' (include '_' in front). In Kloxo-MR, '_dmarc' still accept but need to convert '_dmarc' to 'dmarc' and then save as 'hostname'.
Title: Re: DMARC txt records does not work properly ?
Post by: Spacedust on 2015-04-24, 14:07:11
I am not sure but possible in dns table of kloxo database, 'dmac' and '_dmarc' save in the same variable (where variable is 'dmarc'). And it's make Kloxo-MR panel confuse with this situation.

By default, Kloxo not accept variable like '_dmarc' (include '_' in front). In Kloxo-MR, '_dmarc' still accept but need to convert '_dmarc' to 'dmarc' and then save as 'hostname'.

Latest update to kloxo/httpdocs/driver/dns/dnsbaselib.php does not fix it.

Just add simple entry where reloading DNS template removes all old TXT records with __base__
Title: Re: DMARC txt records does not work properly ?
Post by: Spacedust on 2015-04-24, 15:00:37
Maybe first remove all DNS records before reloading the domain:

sh /script/update --class=dns --name=$DOMAIN --subaction=general --action=delete

It kills domainkeys too ;(
Title: Re: DMARC txt records does not work properly ?
Post by: Spacedust on 2015-04-24, 20:38:46
How to add action delete to this:

Quote
https://domain.com:7777/display.php?frm_action=updateform&frm_subaction=edit&frm_o_o[0][class]=client&frm_o_o[0][nname]=agona2&frm_o_o[1][class]=domain&frm_o_o[1][nname]=3dgsm.pl&frm_o_o[2][class]=dns&frm_o_o[3][class]=dns_record_a&frm_o_o[3][nname]=txt_base
Title: Re: DMARC txt records does not work properly ?
Post by: Spacedust on 2015-04-24, 23:33:17
Any ideas ?
Title: Re: DMARC txt records does not work properly ?
Post by: MRatWork on 2015-04-25, 07:14:13
I don't testing this codes but you can try in your testing server.

Try to copy fixdnsremoverecord to /usr/local/lxlabs/kloxo/pscript and fixdnsremoverecord.php to /usr/local/lxlabs/kloxo/bin/misc
Title: Re: DMARC txt records does not work properly ?
Post by: Spacedust on 2015-04-25, 11:40:17
Something is wrong:

Quote
sh /script/fixdnsremoverecord --ttype=TXT --hostname=localhost
/script/fixdnsremoverecord: line 29: [: missing `]'
Remove DNS record for 'localhost' hostname in 'TXT' ttype
- For 'test.xxx.pl' ('admin') at 'localhost'
Title: Re: DMARC txt records does not work properly ?
Post by: MRatWork on 2015-04-25, 12:00:44
Wait for next update (also introduce for thehostingtool).
Title: Re: DMARC txt records does not work properly ?
Post by: Spacedust on 2015-04-25, 12:05:26
Wait for next update (also introduce for thehostingtool).

OK. So far tested again on 7.0.0.b-2015042502

No errors, but no action were made:

Quote
sh /script/fixdnsremoverecord --ttype=TXT --hostname=localhost
Remove DNS record for 'localhost' hostname in 'TXT' ttype
- For 'test.xxx.pl' ('admin') at 'localhost'
Title: Re: DMARC txt records does not work properly ?
Post by: Spacedust on 2015-04-26, 14:30:05
Does it work now ?
Title: Re: DMARC txt records does not work properly ?
Post by: MRatWork on 2015-04-26, 14:32:49
I am not testing fixdnsremoverecord intensively.
Title: Re: DMARC txt records does not work properly ?
Post by: Spacedust on 2015-04-26, 20:11:27
I am not testing fixdnsremoverecord intensively.

It doesn't work at all. Please test it. I really need to have it working.
Title: Re: DMARC txt records does not work properly ?
Post by: MRatWork on 2015-04-26, 20:54:54
Replace content of fixdnsremoverecord.php with:
Code: [Select]
<?php 

include_once "lib/html/include.php"

initProgram('admin');

$login->loadAllObjects('client');
$list $login->getList('client');

$par parse_opt($argv);

if (isset(
$par['ttype'])) {
$ttype $par['ttype'];
}

if (isset(
$par['hostname'])) {
$hostname $par['hostname'];
}

$nolog false;

log_cleanup("Remove DNS record for '{$hostname}' hostname in '{$ttype}' ttype"$nolog);

foreach(
$list as $c) {
$dlist $c->getList('domain');

foreach($dlist as $l) {
$dns $l->getObject('dns');
$dns->setUpdateSubaction('full_update');

print("- For '{$dns->nname}' ('{$c->nname}') at '{$c->syncserver}'\n");

foreach($dns->dns_record_a as $drec) {
if (($drec->ttype === $ttype) && ($drec->hostname === $hostname)) {
print("-- remove '{$drec->hostname}' hostname in '{$drec->ttype}'\n");
} else {
$x[] = $drec;
}
}

$dns->dns_record_a $x;

$dns->was();
}
}

Title: Re: DMARC txt records does not work properly ?
Post by: Spacedust on 2015-04-26, 21:42:11
Done, but nothing has changed. All TXT records are untouched.

Code: [Select]
sh /script/fixdnsremoverecord --ttype=TXT --hostname=localhost
Remove DNS record for 'localhost' hostname in 'TXT' ttype
- For 'test.xxx.pl' ('admin') at 'localhost'
Title: Re: DMARC txt records does not work properly ?
Post by: MRatWork on 2015-04-26, 22:55:08
Wrong input. It's must like 'sh /script/fixdnsremoverecord --ttype=txt --hostname=dmarc'.
Title: Re: DMARC txt records does not work properly ?
Post by: Spacedust on 2015-04-27, 12:12:34
My fixdnsremoverecord is blank...
Title: Re: DMARC txt records does not work properly ?
Post by: Spacedust on 2015-04-27, 12:13:59
Restored and it worked !

Quote
sh /script/fixdnsremoverecord --ttype=txt --hostname=__base__
Remove DNS record for '__base__' hostname in 'txt' ttype
- For 'tester.pl' ('admin') at 'localhost'
-- remove '__base__' hostname in 'txt'
-- remove '__base__' hostname in 'txt'
-- remove '__base__' hostname in 'txt'
-- remove '__base__' hostname in 'txt'
-- remove '__base__' hostname in 'txt'
-- remove '__base__' hostname in 'txt'
-- remove '__base__' hostname in 'txt'
-- remove '__base__' hostname in 'txt'
-- remove '__base__' hostname in 'txt'
-- remove '__base__' hostname in 'txt'
-- remove '__base__' hostname in 'txt'
-- remove '__base__' hostname in 'txt'
-- remove '__base__' hostname in 'txt'
-- remove '__base__' hostname in 'txt'
-- remove '__base__' hostname in 'txt'
-- remove '__base__' hostname in 'txt'
-- remove '__base__' hostname in 'txt'
-- remove '__base__' hostname in 'txt'
-- remove '__base__' hostname in 'txt'
-- remove '__base__' hostname in 'txt'
-- remove '__base__' hostname in 'txt'
-- remove '__base__' hostname in 'txt'
-- remove '__base__' hostname in 'txt'
-- remove '__base__' hostname in 'txt'
-- remove '__base__' hostname in 'txt'
-- remove '__base__' hostname in 'txt'
-- remove '__base__' hostname in 'txt'
-- remove '__base__' hostname in 'txt'
-- remove '__base__' hostname in 'txt'
-- remove '__base__' hostname in 'txt'
-- remove '__base__' hostname in 'txt'
-- remove '__base__' hostname in 'txt'
-- remove '__base__' hostname in 'txt'
-- remove '__base__' hostname in 'txt'
-- remove '__base__' hostname in 'txt'
-- remove '__base__' hostname in 'txt'
-- remove '__base__' hostname in 'txt'
-- remove '__base__' hostname in 'txt'
-- remove '__base__' hostname in 'txt'
-- remove '__base__' hostname in 'txt'
-- remove '__base__' hostname in 'txt'
-- remove '__base__' hostname in 'txt'
-- remove '__base__' hostname in 'txt'
-- remove '__base__' hostname in 'txt'
-- remove '__base__' hostname in 'txt'
-- remove '__base__' hostname in 'txt'
-- remove '__base__' hostname in 'txt'
-- remove '__base__' hostname in 'txt'
-- remove '__base__' hostname in 'txt'
-- remove '__base__' hostname in 'txt'
-- remove '__base__' hostname in 'txt'
-- remove '__base__' hostname in 'txt'
-- remove '__base__' hostname in 'txt'
-- remove '__base__' hostname in 'txt'
-- remove '__base__' hostname in 'txt'
-- remove '__base__' hostname in 'txt'
-- remove '__base__' hostname in 'txt'
-- remove '__base__' hostname in 'txt'
-- remove '__base__' hostname in 'txt'
-- remove '__base__' hostname in 'txt'
-- remove '__base__' hostname in 'txt'
-- remove '__base__' hostname in 'txt'
-- remove '__base__' hostname in 'txt'
-- remove '__base__' hostname in 'txt'
-- remove '__base__' hostname in 'txt'
-- remove '__base__' hostname in 'txt'
-- remove '__base__' hostname in 'txt'
-- remove '__base__' hostname in 'txt'
-- remove '__base__' hostname in 'txt'
-- remove '__base__' hostname in 'txt'
-- remove '__base__' hostname in 'txt'
-- remove '__base__' hostname in 'txt'
-- remove '__base__' hostname in 'txt'
-- remove '__base__' hostname in 'txt'
-- remove '__base__' hostname in 'txt'
-- remove '__base__' hostname in 'txt'
-- remove '__base__' hostname in 'txt'
-- remove '__base__' hostname in 'txt'
-- remove '__base__' hostname in 'txt'
-- remove '__base__' hostname in 'txt'
-- remove '__base__' hostname in 'txt'
-- remove '__base__' hostname in 'txt'
-- remove '__base__' hostname in 'txt'
-- remove '__base__' hostname in 'txt'
-- remove '__base__' hostname in 'txt'
-- remove '__base__' hostname in 'txt'
-- remove '__base__' hostname in 'txt'
Title: Re: DMARC txt records does not work properly ?
Post by: Spacedust on 2015-04-27, 12:25:09
It's very slow (1-2 domains per minute) but it works ;) Removing in progress...

I will have to rebuild it anyway, because it also removes my SPF records.
Title: Re: DMARC txt records does not work properly ?
Post by: MRatWork on 2015-04-27, 12:42:31
It's slow but it works ;) Removing in progress...

I will have to rebuild it anyway, because it also removes my SPF records.
Yes, because remove with 2 variables. You are lucky because all default 'txt record' (spf, dkim, dmarc and domainkeys) not insert directly to dns setting.
Title: Re: DMARC txt records does not work properly ?
Post by: Spacedust on 2015-04-27, 14:06:50
Sometimes it gets faster, then it's about 3-5 domains per minute. I'm leaving this on "screen" ;)
Title: Re: DMARC txt records does not work properly ?
Post by: Spacedust on 2015-04-28, 12:09:11
I have to restart my Kloxo because of new SSL cert and it stopped on letter "B" and not moving.

How to resolve this ?
Title: Re: DMARC txt records does not work properly ?
Post by: MRatWork on 2015-04-28, 12:19:46
Some users tell that nginx and hiawatha (that mean including their proxy) have a problem with ssl from comodo. I don't know what's a problem is.
Title: Re: DMARC txt records does not work properly ?
Post by: Spacedust on 2015-04-28, 12:41:44
Some users tell that nginx and hiawatha (that mean including their proxy) have a problem with ssl from comodo. I don't know what's a problem is.

No troubles with SSL. I mean your fixdnsremoverecord is not working properly. It's way too slow with many domains. It will take ages to complete in master-slave environment where DNS is also on slave.
Title: Re: DMARC txt records does not work properly ?
Post by: MRatWork on 2015-04-28, 13:00:22
Fixdnsremoverecord just remove record from dns table of kloxo database and don't care for master or slave.

After remove need fixdns dan restart-dns.

Yes, this script is slow because reading object instead read database directly where object tend to more need process and memory.
Title: Re: DMARC txt records does not work properly ?
Post by: Spacedust on 2015-04-28, 14:58:29
Fixdnsremoverecord just remove record from dns table of kloxo database and don't care for master or slave.

After remove need fixdns dan restart-dns.

Yes, this script is slow because reading object instead read database directly where object tend to more need process and memory.

Now even reloading DNS template takes ages to complete. It wasn't that slow before !
Title: Re: DMARC txt records does not work properly ?
Post by: Spacedust on 2015-04-28, 17:44:44
After upgrade to latest Kloxo-MR. It stopped working again:

Quote
sh /script/fixdnsremoverecord --ttype=txt --hostname=__base__
Remove DNS record for '' key in '' type
- For 'infotryb.pl' domain ('adi2577' client) at 'localhost' server
-- NO exists of '' key in '' type
Title: Re: DMARC txt records does not work properly ?
Post by: MRatWork on 2015-04-28, 18:00:19
change parameter, from 'hostname' to 'key' and 'ttype' to 'type'. Try execute 'sh /script/fixdnsremoverecord' and the you see what's you want.
Title: Re: DMARC txt records does not work properly ?
Post by: Spacedust on 2015-04-28, 18:04:54
change parameter, from 'hostname' to 'key' and 'ttype' to 'type'. Try execute 'sh /script/fixdnsremoverecord' and the you see what's you want.

It works again, but ultra slow. Can we skip clients which are already fixed ? Maybe add --client switch ?
Title: Re: DMARC txt records does not work properly ?
Post by: MRatWork on 2015-04-28, 18:18:05
Try replace fixdnsremoverecord.php with:
Code: [Select]
<?php 

include_once "lib/html/include.php"

initProgram('admin');

$login->loadAllObjects('client');
$list $login->getList('client');

$par parse_opt($argv);

if (isset(
$par['type'])) {
$ttype strtolower($par['type']);
}

if (isset(
$par['key'])) {
$hostname strtolower($par['key']);
}

$client = (isset($list['client'])) ? $list['client'] : null;
$clist = array();

$nolog false;

log_cleanup("Remove DNS record for '{$hostname}' key in '{$ttype}' type"$nolog);

foreach(
$list as $c) {
if ($client) {
$ca explode(","$client);

if (!in_array($c->nname$ca)) { continue; }
}

$dlist $c->getList('domain');

foreach($dlist as $l) {
$dns $l->getObject('dns');
$dns->setUpdateSubaction('full_update');

print("- For '{$dns->nname}' domain ('{$c->nname}' client) at '{$c->syncserver}' server\n");

$removed false;

foreach($dns->dns_record_a as $drec) {
if (($drec->ttype === $ttype) && ($drec->hostname === $hostname)) {
print("-- remove '{$drec->hostname}' key in '{$drec->ttype}' type\n");
$removed true;
} else {
$x[] = $drec;
}
}

if ($removed === false) {
print("-- NO exists of '{$hostname}' key in '{$ttype}' type\n");
}

$dns->dns_record_a $x;

$dns->was();
}
}


You can set like '--client=clientA,clientB'
Title: Re: DMARC txt records does not work properly ?
Post by: Spacedust on 2015-04-28, 18:53:39
It does not work ;(
Title: Re: DMARC txt records does not work properly ?
Post by: MRatWork on 2015-04-28, 19:16:25
Try this:
Code: [Select]
<?php 

include_once "lib/html/include.php"

initProgram('admin');

$login->loadAllObjects('client');
$list $login->getList('client');

$par parse_opt($argv);

if (isset(
$par['type'])) {
$ttype strtolower($par['type']);
}

if (isset(
$par['key'])) {
$hostname strtolower($par['key']);
}

$client = (isset($par['client'])) ? $par['client'] : null;
$clist = array();

$nolog false;

log_cleanup("Remove DNS record for '{$hostname}' key in '{$ttype}' type"$nolog);

foreach(
$list as $c) {
if ($client) {
$ca explode(","$client);

if (!in_array($c->nname$ca)) { continue; }
}

$dlist $c->getList('domain');

foreach($dlist as $l) {
$dns $l->getObject('dns');
$dns->setUpdateSubaction('full_update');

print("- For '{$dns->nname}' domain ('{$c->nname}' client) at '{$c->syncserver}' server\n");

$removed false;

foreach($dns->dns_record_a as $drec) {
if (($drec->ttype === $ttype) && ($drec->hostname === $hostname)) {
print("-- remove '{$drec->hostname}' key in '{$drec->ttype}' type\n");
$removed true;
} else {
$x[] = $drec;
}
}

if ($removed === false) {
print("-- NO exists of '{$hostname}' key in '{$ttype}' type\n");
}

$dns->dns_record_a $x;

$dns->was();
}
}

Title: Re: DMARC txt records does not work properly ?
Post by: Spacedust on 2015-04-28, 19:19:46
Ok. First I need to investigate why Kloxo is such slow and it wasn't before.
Title: Re: DMARC txt records does not work properly ?
Post by: MRatWork on 2015-04-28, 19:37:25
Try restart mysql and kloxo itself.