[Opendnssec-user] *****SPAM***** ODS 2.0.1 did not start after reboot.

[Fred.Zwarts]

Spam detection software, running on the system "dicht.nlnetlabs.nl",
has identified this incoming email as possible spam. The original
message has been attached to this so you can view it or label
similar future email. If you have any questions, see
The administrator of that system for details.

Content preview: ODS 2.0.1 has now been running satisfactory on our test system
for several weeks. However, recently we noticed that each time we reboot
the system, ods does not startup properly. It turns out that after each reboot,
the directory /var/run/opendnssec has disappeared, so opendnssec can not
start, because it wants to create sock and pid files in this directory. I
have worked around this problem, by modifying /usr/local/sbin/ods-control,
where I added a mkdir for this directory just before the startup of the enforcer.
[...]

Content analysis details: (5.5 points, 5.0 required)

pts rule name description
---- ---------------------- --------------------------------------------------
0.2 STOX_REPLY_TYPE No description available.
1.3 RDNS_NONE Delivered to internal network by a host with no rDNS
1.9 STOX_REPLY_TYPE_WITHOUT_QUOTES No description available.
2.2 TO_NO_BRKTS_MSFT To: lacks brackets and supposed Microsoft tool

[Berry A.W. van Halderen]

Post by Fred.Zwarts
ODS 2.0.1 has now been running satisfactory on our test system
for several weeks. However, recently we noticed that each time we reboot
the system, ods does not startup properly. It turns out that after each reboot,
the directory /var/run/opendnssec has disappeared, so opendnssec can not
start, because it wants to create sock and pid files in this directory. I
have worked around this problem, by modifying /usr/local/sbin/ods-control,
where I added a mkdir for this directory just before the startup of the enforcer.
[...]

Thanks for your feedback, good to see 2.0.1 in the real world!

Unfortunately, this is specific to how some Linux distributions work,
and it is not easy to create a consistent fix for it.
The best way is for package providers for the distributions should
create a fix in their system.

The problem lies in the fact that some distributions regard /var/run to
be completely volatile. Any content in this directory is removed on a
reboot.

Now OpenDNSSEC uses multiple files that should go into /var/run.
Multiple files containing PIDs and sockets. It is not uncommon to
want to bundle them into a directory below /var/run.

Now you could say just create the directory if it does not exist.
However here the trouble starts. The directory is not always writable
for any other user. And if you do not start the daemon as root,
the directory cannot be created. Giving a bad user experience to
others. More over, since this directory can also be located elsewhere
we cannot assume at all to have write permissions to create the
directory.

What normally happens, is that package providers for such distributions
either make an exception for /var/run/opendnssec to not be removed, or
add the sequence to create the directory after a reboot.
We however cannot make such a patch to the system ourselved, afaik.

It is also possible for you to locate the /var/run/opendnssec directory
elsewhere. It is part of the standard options of the autoconf when
compiling.

With kind regards,
Berry van Halderen

PS: a better warning concerning this problem during reboot should be
outputted by OpenDNSSEC. That is very much true.

[Berry A.W. van Halderen]

I'm resending the message because it may have been missed by people
on the user list as it was the subject contained the word spam.
Unfortunately the original message was marked as spam because of
high rating in bad To: address.

\Berry

Post by Berry A.W. van Halderen
Thanks for your feedback, good to see 2.0.1 in the real world!
Unfortunately, this is specific to how some Linux distributions work,
and it is not easy to create a consistent fix for it.
The best way is for package providers for the distributions should
create a fix in their system.
The problem lies in the fact that some distributions regard /var/run to
be completely volatile. Any content in this directory is removed on a
reboot.
Now OpenDNSSEC uses multiple files that should go into /var/run.
Multiple files containing PIDs and sockets. It is not uncommon to
want to bundle them into a directory below /var/run.
Now you could say just create the directory if it does not exist.
However here the trouble starts. The directory is not always writable
for any other user. And if you do not start the daemon as root,
the directory cannot be created. Giving a bad user experience to
others. More over, since this directory can also be located elsewhere
we cannot assume at all to have write permissions to create the
directory.
What normally happens, is that package providers for such distributions
either make an exception for /var/run/opendnssec to not be removed, or
add the sequence to create the directory after a reboot.
We however cannot make such a patch to the system ourselved, afaik.
It is also possible for you to locate the /var/run/opendnssec directory
elsewhere. It is part of the standard options of the autoconf when
compiling.
With kind regards,
Berry van Halderen
PS: a better warning concerning this problem during reboot should be
outputted by OpenDNSSEC. That is very much true.

[Wytze van der Raay]

ODS 2.0.1 has now been running satisfactory on our test system for several
weeks. However, recently we noticed that each time we reboot the system,
ods does not startup properly. It turns out that after each reboot, the
directory /var/run/opendnssec has disappeared, so opendnssec can not start,
because it wants to create sock and pid files in this directory. I have
worked around this problem, by modifying /usr/local/sbin/ods-control, where
I added a mkdir for this directory just before the startup of the
enforcer.
It is not clear to me how this directory disappears. The test system runs
SLES 12 SP1.
I hope that this information helps to improve this software.

I recognize the problem ... we are running ODS 2.0.1 on OpenSUSE 13.2,
which is presumably quite similar to SLES 12 SP1. The /var/run directory
is a symbolic link to the /run directory. /run is created as tmpfs on
every reboot, ie its content does not persist. This change has been made in
SUSE and several other Linux distros to accomodate systemd, see for example
http://www.h-online.com/open/news/item/Linux-distributions-to-include-run-directory-1219006.html
for some background.

I *think* that if one uses a proper systemd script for starting/stopping ODS,
it is possible to have the opendnssec directory created automatically more or
less, but my current hack is to put a "mkdir -p /var/run/opendnssec" in its
(oldfashioned init-style) startup script. The same issue exists with nsd,
and I solved it in the same way.

In any case, this is not something that should be solved in
/usr/local/sbin/ods-control, but rather in the distro-specific packaging scripts
for ODS.

Regards,
Wytze van der Raay

[Petr Spacek]

Post by Wytze van der Raay

ODS 2.0.1 has now been running satisfactory on our test system for several
weeks. However, recently we noticed that each time we reboot the system,
ods does not startup properly. It turns out that after each reboot, the
directory /var/run/opendnssec has disappeared, so opendnssec can not start,
because it wants to create sock and pid files in this directory. I have
worked around this problem, by modifying /usr/local/sbin/ods-control, where
I added a mkdir for this directory just before the startup of the
enforcer.
It is not clear to me how this directory disappears. The test system runs
SLES 12 SP1.
I hope that this information helps to improve this software.

I recognize the problem ... we are running ODS 2.0.1 on OpenSUSE 13.2,
which is presumably quite similar to SLES 12 SP1. The /var/run directory
is a symbolic link to the /run directory. /run is created as tmpfs on
every reboot, ie its content does not persist. This change has been made in
SUSE and several other Linux distros to accomodate systemd, see for example
http://www.h-online.com/open/news/item/Linux-distributions-to-include-run-directory-1219006.html
for some background.
I *think* that if one uses a proper systemd script for starting/stopping ODS,
it is possible to have the opendnssec directory created automatically more or
less, but my current hack is to put a "mkdir -p /var/run/opendnssec" in its
(oldfashioned init-style) startup script. The same issue exists with nsd,
and I solved it in the same way.

"Proper systemd-way" is to use tmpfiles.d config file:
https://www.freedesktop.org/software/systemd/man/tmpfiles.d.html

It allows to specify what directories and files should be re-created as
needed. I'm attaching config file which is shipped right now by Fedora 24.

It would be awesome if OpenDNSSEC could ship its tmpfiles.d config file
somewhere in contrib directory in source tree or so. It would allow all
systemd-based distros to use the proper config file instead of inventing own
buggy ones.

Post by Wytze van der Raay
In any case, this is not something that should be solved in
/usr/local/sbin/ods-control, but rather in the distro-specific packaging scripts
for ODS.

[Fred.Zwarts]

Post by Petr Spacek

Post by Wytze van der Raay

ODS 2.0.1 has now been running satisfactory on our test system for several
weeks. However, recently we noticed that each time we reboot the system,
ods does not startup properly. It turns out that after each reboot, the
directory /var/run/opendnssec has disappeared, so opendnssec can not start,
because it wants to create sock and pid files in this directory. I have
worked around this problem, by modifying /usr/local/sbin/ods-control, where
I added a mkdir for this directory just before the startup of the
enforcer.
It is not clear to me how this directory disappears. The test system runs
SLES 12 SP1.
I hope that this information helps to improve this software.

I recognize the problem ... we are running ODS 2.0.1 on OpenSUSE 13.2,
which is presumably quite similar to SLES 12 SP1. The /var/run directory
is a symbolic link to the /run directory. /run is created as tmpfs on
every reboot, ie its content does not persist. This change has been made in
SUSE and several other Linux distros to accomodate systemd, see for example
http://www.h-online.com/open/news/item/Linux-distributions-to-include-run-directory-1219006.html
for some background.
I *think* that if one uses a proper systemd script for starting/stopping ODS,
it is possible to have the opendnssec directory created automatically more or
less, but my current hack is to put a "mkdir -p /var/run/opendnssec" in its
(oldfashioned init-style) startup script. The same issue exists with nsd,
and I solved it in the same way.

https://www.freedesktop.org/software/systemd/man/tmpfiles.d.html
It allows to specify what directories and files should be re-created as
needed. I'm attaching config file which is shipped right now by Fedora 24.
It would be awesome if OpenDNSSEC could ship its tmpfiles.d config file
somewhere in contrib directory in source tree or so. It would allow all
systemd-based distros to use the proper config file instead of inventing own
buggy ones.

Thanks. I made a conf file in /etc/tmpfiles.d and it works! No messing in
files that are overwritten with new versions of opendnssec.