Discussion:
pkgbase: Move of head/etc/ files
Brad Davis
2018-08-01 23:54:10 UTC
Permalink
Hello,

I want to give some information on what is happening to files in head/etc/ and open a discussion.

I am trying to move all config files out of head/etc/ and place them with the source files for the particular use. For example, some of these are obvious moves like moving say portsnap.conf to usr.sbin/portsnap/portsnap.conf. The review for this move shows exactly what would happen here: https://reviews.freebsd.org/D16554. For these cases, there are some easy benefits, like:
1) Keeping all files as part of a program together, which makes maintenance for maintainers a little easier.
2) Simplicity, we do not need all the .if ${MK_FOO} lines in etc/Makefile because that is handled a level above the programs Makefile and so one CONFS= line is all that is needed.
3) Easier to avoid bugs where we install things that should not be installed. Like for apmd, we installed apmd.conf on all archs even though it only works on i386. i.e. we don't have to keep knobs in sync in multiple locations.

Some of the more complicated things to move like etc/shells and the init scripts. I picked the location of etc/shells based on it being used by getusershell(3) in libc, so it ended up in lib/libc/gen/shells. The move of the init scripts was based on a few things, including the reasons above, but additionally:
1) Isolating a subcomponent, for example work is happening to bring in OpenRC, so we will have multiple init systems with different types of scripts, so moving them away from head/etc/ keeps them isolated.

This work was driven by needing to tag these files as config files in the METALOG so that pkg knows what is a config file so that pkg can 3-way merge them.

More info on pkgbase: https://wiki.freebsd.org/PkgBase


Regards,
Brad Davis
Simon J. Gerraty
2018-08-02 00:11:11 UTC
Permalink
Post by Brad Davis
I am trying to move all config files out of head/etc/ and place them
with the source files for the particular use. For example, some of
these are obvious moves like moving say portsnap.conf to
usr.sbin/portsnap/portsnap.conf. The review for this move shows
In the case of foo.conf for foo, this is a reasonable approach.

There are however many files in etc/ that do not fit that model.

IMO one of the nice things about the BSD src tree, is that the location
of a file in the src tree can generally be inferred from its installed
location on the system.
This property should not be lost without substantial countervailing
benefit.
Brad Davis
2018-08-02 00:30:11 UTC
Permalink
Post by Simon J. Gerraty
Post by Brad Davis
I am trying to move all config files out of head/etc/ and place them
with the source files for the particular use. For example, some of
these are obvious moves like moving say portsnap.conf to
usr.sbin/portsnap/portsnap.conf. The review for this move shows
In the case of foo.conf for foo, this is a reasonable approach.
There are however many files in etc/ that do not fit that model.
IMO one of the nice things about the BSD src tree, is that the location
of a file in the src tree can generally be inferred from its installed
location on the system.
This property should not be lost without substantial countervailing
benefit.
Hi Simon,

I generally agree with you, but I think where to draw that line is different based on who is looking at it at the moment.

Lots, not all, but lots, of our config files have the $FreeBSD$ line that shows the path of where the file came from in the source tree, so that helps some. Using /etc/shells as an example, the first line is:

# $FreeBSD: head/lib/libc/gen/shells 336840 2018-07-28 20:21:23Z brd $


Regards,
Brad Davis
Guido Falsi
2018-08-03 10:43:12 UTC
Permalink
Post by Brad Davis
Hello,
I want to give some information on what is happening to files in head/etc/ and open a discussion.
Today while updating a pkgbase machine I got this error(with trimmed
debug output):

The following 1 package(s) will be affected (of 0 checked):

Installed packages to be UPGRADED:
FreeBSD-runtime: 12.0.s20180725172903 -> 12.0.s20180803092917
[mpnet-base]

Number of packages to be upgraded: 1

Proceed with this action? [y/N]: y
DBG(1)[71153]> want to upgrade advisory to exclusive lock
[1/1] Upgrading FreeBSD-runtime from 12.0.s20180725172903 to
12.0.s20180803092917...
[1/1] Extracting FreeBSD-runtime-12.0.s20180803092917: 1%
DBG(1)[71153]> Populating config_file /etc/blacklistd.conf
[1/1] Extracting FreeBSD-runtime-12.0.s20180803092917: 1%
DBG(1)[71153]> Populating config_file /etc/defaults/rc.conf
Segmentation fault (core dumped)

I'm not sure if it's related to r336845 and r336847, but since those are
the only one touching rc.conf recently it looks possible.

Anyone has an idea what I've stumbled upon?

I'm investigating this, but if I can't fix it shortly I'll revert to my
previous pkg set and try again at another time. If some tests are needed
I'll try to perform them as requested.

Thanks in advance!

BTW I have no objection to moving the configuration files if it's needed
for code maintainability.
--
Guido Falsi <***@FreeBSD.org>
Brad Davis
2018-08-04 16:50:45 UTC
Permalink
Post by Guido Falsi
Post by Brad Davis
Hello,
I want to give some information on what is happening to files in head/etc/ and open a discussion.
Today while updating a pkgbase machine I got this error(with trimmed
FreeBSD-runtime: 12.0.s20180725172903 -> 12.0.s20180803092917
[mpnet-base]
Number of packages to be upgraded: 1
Proceed with this action? [y/N]: y
DBG(1)[71153]> want to upgrade advisory to exclusive lock
[1/1] Upgrading FreeBSD-runtime from 12.0.s20180725172903 to
12.0.s20180803092917...
[1/1] Extracting FreeBSD-runtime-12.0.s20180803092917: 1%
DBG(1)[71153]> Populating config_file /etc/blacklistd.conf
[1/1] Extracting FreeBSD-runtime-12.0.s20180803092917: 1%
DBG(1)[71153]> Populating config_file /etc/defaults/rc.conf
Segmentation fault (core dumped)
I'm not sure if it's related to r336845 and r336847, but since those are
the only one touching rc.conf recently it looks possible.
Anyone has an idea what I've stumbled upon?
I'm investigating this, but if I can't fix it shortly I'll revert to my
previous pkg set and try again at another time. If some tests are needed
I'll try to perform them as requested.
This is a bug in pkg and has to do with these files transitioning from normal files to config files.

I will try and work on this in the next week.


Regards,
Brad Davis
Guido Falsi
2018-08-04 17:21:55 UTC
Permalink
Post by Brad Davis
Post by Guido Falsi
Post by Brad Davis
Hello,
I want to give some information on what is happening to files in head/etc/ and open a discussion.
Today while updating a pkgbase machine I got this error(with trimmed
FreeBSD-runtime: 12.0.s20180725172903 -> 12.0.s20180803092917
[mpnet-base]
Number of packages to be upgraded: 1
Proceed with this action? [y/N]: y
DBG(1)[71153]> want to upgrade advisory to exclusive lock
[1/1] Upgrading FreeBSD-runtime from 12.0.s20180725172903 to
12.0.s20180803092917...
[1/1] Extracting FreeBSD-runtime-12.0.s20180803092917: 1%
DBG(1)[71153]> Populating config_file /etc/blacklistd.conf
[1/1] Extracting FreeBSD-runtime-12.0.s20180803092917: 1%
DBG(1)[71153]> Populating config_file /etc/defaults/rc.conf
Segmentation fault (core dumped)
I'm not sure if it's related to r336845 and r336847, but since those are
the only one touching rc.conf recently it looks possible.
Anyone has an idea what I've stumbled upon?
I'm investigating this, but if I can't fix it shortly I'll revert to my
previous pkg set and try again at another time. If some tests are needed
I'll try to perform them as requested.
This is a bug in pkg and has to do with these files transitioning from normal files to config files.
I will try and work on this in the next week.
Thanks in advance!

For me there's no hurry. I have reverted my machine and will not be
updating for at least a full week anyway.

If you need testing I'm available though.
--
Guido Falsi <***@FreeBSD.org>
Guido Falsi
2018-08-18 17:34:13 UTC
Permalink
Post by Guido Falsi
Post by Brad Davis
Post by Guido Falsi
Post by Brad Davis
Hello,
I want to give some information on what is happening to files in head/etc/ and open a discussion.
Today while updating a pkgbase machine I got this error(with trimmed
FreeBSD-runtime: 12.0.s20180725172903 -> 12.0.s20180803092917
[mpnet-base]
Number of packages to be upgraded: 1
Proceed with this action? [y/N]: y
DBG(1)[71153]> want to upgrade advisory to exclusive lock
[1/1] Upgrading FreeBSD-runtime from 12.0.s20180725172903 to
12.0.s20180803092917...
[1/1] Extracting FreeBSD-runtime-12.0.s20180803092917: 1%
DBG(1)[71153]> Populating config_file /etc/blacklistd.conf
[1/1] Extracting FreeBSD-runtime-12.0.s20180803092917: 1%
DBG(1)[71153]> Populating config_file /etc/defaults/rc.conf
Segmentation fault (core dumped)
I'm not sure if it's related to r336845 and r336847, but since those are
the only one touching rc.conf recently it looks possible.
Anyone has an idea what I've stumbled upon?
I'm investigating this, but if I can't fix it shortly I'll revert to my
previous pkg set and try again at another time. If some tests are needed
I'll try to perform them as requested.
This is a bug in pkg and has to do with these files transitioning from normal files to config files.
I will try and work on this in the next week.
Thanks in advance!
For me there's no hurry. I have reverted my machine and will not be
updating for at least a full week anyway.
If you need testing I'm available though.
Hi I just updated a machine using pkg with this patch added:

https://github.com/freebsd/pkg/commit/30644237c1c655e43911110198fe23a9c835aa24

and it now works fine, so thanks a lot!
--
Guido Falsi <***@FreeBSD.org>
Brad Davis
2018-08-20 17:09:52 UTC
Permalink
Post by Guido Falsi
Post by Guido Falsi
Post by Brad Davis
Post by Guido Falsi
Post by Brad Davis
Hello,
I want to give some information on what is happening to files in head/etc/ and open a discussion.
Today while updating a pkgbase machine I got this error(with trimmed
FreeBSD-runtime: 12.0.s20180725172903 -> 12.0.s20180803092917
[mpnet-base]
Number of packages to be upgraded: 1
Proceed with this action? [y/N]: y
DBG(1)[71153]> want to upgrade advisory to exclusive lock
[1/1] Upgrading FreeBSD-runtime from 12.0.s20180725172903 to
12.0.s20180803092917...
[1/1] Extracting FreeBSD-runtime-12.0.s20180803092917: 1%
DBG(1)[71153]> Populating config_file /etc/blacklistd.conf
[1/1] Extracting FreeBSD-runtime-12.0.s20180803092917: 1%
DBG(1)[71153]> Populating config_file /etc/defaults/rc.conf
Segmentation fault (core dumped)
I'm not sure if it's related to r336845 and r336847, but since those are
the only one touching rc.conf recently it looks possible.
Anyone has an idea what I've stumbled upon?
I'm investigating this, but if I can't fix it shortly I'll revert to my
previous pkg set and try again at another time. If some tests are needed
I'll try to perform them as requested.
This is a bug in pkg and has to do with these files transitioning from normal files to config files.
I will try and work on this in the next week.
Thanks in advance!
For me there's no hurry. I have reverted my machine and will not be
updating for at least a full week anyway.
If you need testing I'm available though.
https://github.com/freebsd/pkg/commit/30644237c1c655e43911110198fe23a9c835aa24
and it now works fine, so thanks a lot!
Great, and I just committed this to ports as pkg 1.10.5_2.


Regards,
Brad Davis

Chris H
2018-08-04 09:43:44 UTC
Permalink
Post by Brad Davis
Hello,
I want to give some information on what is happening to files in head/etc/
and open a discussion.
I am trying to move all config files out of head/etc/ and place them with the
source files for the particular use. For example, some of these are obvious
moves like moving say portsnap.conf to usr.sbin/portsnap/portsnap.conf. The
https://reviews.freebsd.org/D16554. For these cases, there are some easy
1) Keeping all files as part of a program together, which makes maintenance
for maintainers a little easier.
2) Simplicity, we do not need all the .if ${MK_FOO} lines in etc/Makefile
because that is handled a level above the programs Makefile and so one CONFS=
line is all that is needed.
3) Easier to avoid bugs where we install things that should not be installed.
Like for apmd, we installed apmd.conf on all archs even though it only works
on i386. i.e. we don't have to keep knobs in sync in multiple locations.
Some of the more complicated things to move like etc/shells and the init
scripts. I picked the location of etc/shells based on it being used by
getusershell(3) in libc, so it ended up in lib/libc/gen/shells. The move of
the init scripts was based on a few things, including the reasons above, but
1) Isolating a subcomponent, for example work is happening to bring in
OpenRC, so we will have multiple init systems with different types of
scripts, so moving them away from head/etc/ keeps them isolated.
Is the current rc(8) broken? Is OpenRC an intended replacement? How is this
going to work?
Post by Brad Davis
This work was driven by needing to tag these files as config files in the
METALOG so that pkg knows what is a config file so that pkg can 3-way merge
them.
More info on pkgbase: https://wiki.freebsd.org/PkgBase
Regards,
Brad Davis
--Chris
Brad Davis
2018-08-04 16:51:39 UTC
Permalink
Post by Chris H
Post by Brad Davis
Hello,
I want to give some information on what is happening to files in head/etc/
and open a discussion.
I am trying to move all config files out of head/etc/ and place them with the
source files for the particular use. For example, some of these are obvious
moves like moving say portsnap.conf to usr.sbin/portsnap/portsnap.conf. The
https://reviews.freebsd.org/D16554. For these cases, there are some easy
1) Keeping all files as part of a program together, which makes maintenance
for maintainers a little easier.
2) Simplicity, we do not need all the .if ${MK_FOO} lines in etc/Makefile
because that is handled a level above the programs Makefile and so one CONFS=
line is all that is needed.
3) Easier to avoid bugs where we install things that should not be installed.
Like for apmd, we installed apmd.conf on all archs even though it only works
on i386. i.e. we don't have to keep knobs in sync in multiple locations.
Some of the more complicated things to move like etc/shells and the init
scripts. I picked the location of etc/shells based on it being used by
getusershell(3) in libc, so it ended up in lib/libc/gen/shells. The move of
the init scripts was based on a few things, including the reasons above, but
1) Isolating a subcomponent, for example work is happening to bring in
OpenRC, so we will have multiple init systems with different types of
scripts, so moving them away from head/etc/ keeps them isolated.
Is the current rc(8) broken? Is OpenRC an intended replacement? How is this
going to work?
I am not sure what the status or progress is for this. Probably worth contacting Kris Moore.


Regards,
Brad Davis
Loading...