metalog -> makefs is broken again in -HEAD

Post by Adrian Chadd
hi,
warning: tags: unsupported keyword
warning: duplicate definition of mailer.conf
warning: duplicate definition of .
warning: duplicate definition of fstab
warning: duplicate definition of rc.conf
warning: duplicate definition of autosize
makefs: 3 error(s) and 25012 warning(s) in mtree manifest
* the tags=<blah> isn't supported by makefs, which warsn out
* there are now double installs of a handful of files in /etc
Can someone please give me a hand in figuring out how to fix these?
I'm swamped doing other things so doing tree surgery isn't an option.
:(
thanks,
-adrian

Hrm ... I don't even know if I get that far with the wifi build tools:

*** Target Done.
*** Target : fullroot
*** Deleting old file system..
*** Creating new filesystem...
**** Done.
*** Target Done.
*** Target : fullfsimage
*** Running makefs to build image ..
*** from /home/sbruno/bsd/fbsd_head/../mfsroot/malta ..
makefs: Can't stat
`/home/sbruno/bsd/fbsd_head/../mfsroot/METALOG.malta': No such file or
directory

Post by Adrian Chadd
_______________________________________________
https://lists.freebsd.org/mailman/listinfo/freebsd-arch

Adrian Chadd

2016-08-18 14:44:38 UTC

update; just merged the fix.

-a

*** Target Done.
*** Target : fullroot
*** Deleting old file system..
*** Creating new filesystem...
**** Done.
*** Target Done.
*** Target : fullfsimage
*** Running makefs to build image ..
*** from /home/sbruno/bsd/fbsd_head/../mfsroot/malta ..
makefs: Can't stat
`/home/sbruno/bsd/fbsd_head/../mfsroot/METALOG.malta': No such file or
directory

Post by Adrian Chadd
_______________________________________________
https://lists.freebsd.org/mailman/listinfo/freebsd-arch

Sean Bruno

2016-08-19 16:03:35 UTC

Post by Adrian Chadd
update; just merged the fix.
-a

Hrm ... doesn't seem to be working ... if I understand what to do.

===> etc/sendmail (installconfig)
*** Target Done.
*** Target : fullroot
*** Deleting old file system..
install: open /home/sbruno/bsd/fbsd_head/../mfsroot/METALOG.malta.tmp:
No such file or directory
*** Creating new filesystem...
**** Done.
*** Target Done.
*** Target : fullfsimage
*** Running makefs to build image ..
*** from /home/sbruno/bsd/fbsd_head/../mfsroot/malta ..
/home/sbruno/bsd/fbsd_head/../mfsroot/METALOG.malta:54: warning:
duplicate definition of var
/home/sbruno/bsd/fbsd_head/../mfsroot/METALOG.malta:103: warning:
duplicate definition of usr
/home/sbruno/bsd/fbsd_head/../mfsroot/METALOG.malta:848: warning:
duplicate definition of include
/home/sbruno/bsd/fbsd_head/../mfsroot/METALOG.malta:1020: warning:
duplicate definition of lib
/home/sbruno/bsd/fbsd_head/../mfsroot/METALOG.malta:1049: warning:
duplicate definition of usr
/home/sbruno/bsd/fbsd_head/../mfsroot/METALOG.malta:1050: warning:
duplicate definition of share
/home/sbruno/bsd/fbsd_head/../mfsroot/METALOG.malta:1070: warning:
duplicate definition of usr
/home/sbruno/bsd/fbsd_head/../mfsroot/METALOG.malta:1074: warning:
duplicate definition of usr
/home/sbruno/bsd/fbsd_head/../mfsroot/METALOG.malta:1413: warning:
duplicate definition of tests
/home/sbruno/bsd/fbsd_head/../mfsroot/METALOG.malta:1748: warning:
duplicate definition of .
/home/sbruno/bsd/fbsd_head/../mfsroot/METALOG.malta:1749: warning:
duplicate definition of var
/home/sbruno/bsd/fbsd_head/../mfsroot/METALOG.malta:1750: warning:
duplicate definition of spool
/home/sbruno/bsd/fbsd_head/../mfsroot/METALOG.malta:1777: warning: tags:
unsupported keyword
...
/home/sbruno/bsd/fbsd_head/../mfsroot/METALOG.malta:26713: error: time:
invalid value '1471621820'
/home/sbruno/bsd/fbsd_head/../mfsroot/METALOG.malta:26714: error: time:
invalid value '1471621820'
/home/sbruno/bsd/fbsd_head/../mfsroot/METALOG.malta:26715: error: time:
invalid value '1471621820'
....
/home/sbruno/bsd/fbsd_head/../mfsroot/METALOG.malta:27046: warning:
tags: unsupported keyword
/home/sbruno/bsd/fbsd_head/../mfsroot/METALOG.malta:27079: warning:
duplicate definition of mailer.conf
makefs: 3 error(s) and 25009 warning(s) in mtree manifest

Adrian Chadd

2016-08-19 16:11:19 UTC

Right, that's the bit that is failing right there.

-adrian

Post by Adrian Chadd
update; just merged the fix.
-a

Hrm ... doesn't seem to be working ... if I understand what to do.
===> etc/sendmail (installconfig)
*** Target Done.
*** Target : fullroot
*** Deleting old file system..
No such file or directory
*** Creating new filesystem...
**** Done.
*** Target Done.
*** Target : fullfsimage
*** Running makefs to build image ..
*** from /home/sbruno/bsd/fbsd_head/../mfsroot/malta ..
duplicate definition of var
duplicate definition of usr
duplicate definition of include
duplicate definition of lib
duplicate definition of usr
duplicate definition of share
duplicate definition of usr
duplicate definition of usr
duplicate definition of tests
duplicate definition of .
duplicate definition of var
duplicate definition of spool
unsupported keyword
...
invalid value '1471621820'
invalid value '1471621820'
invalid value '1471621820'
....
tags: unsupported keyword
duplicate definition of mailer.conf
makefs: 3 error(s) and 25009 warning(s) in mtree manifest

Bryan Drewery

2016-08-19 17:22:41 UTC

Post by Adrian Chadd
update; just merged the fix.
-a

Hrm ... doesn't seem to be working ... if I understand what to do.
===> etc/sendmail (installconfig)
*** Target Done.
*** Target : fullroot
*** Deleting old file system..

What tool is this? I don't find 'Deleting old file system' in the tree.

What flags are being passed to makefs?

Post by Sean Bruno
No such file or directory
*** Creating new filesystem...
**** Done.
*** Target Done.
*** Target : fullfsimage
*** Running makefs to build image ..
*** from /home/sbruno/bsd/fbsd_head/../mfsroot/malta ..
duplicate definition of var
duplicate definition of usr
duplicate definition of include
duplicate definition of lib
duplicate definition of usr
duplicate definition of share
duplicate definition of usr
duplicate definition of usr
duplicate definition of tests
duplicate definition of .
duplicate definition of var
duplicate definition of spool
unsupported keyword
...
invalid value '1471621820'
invalid value '1471621820'
invalid value '1471621820'
....
tags: unsupported keyword
duplicate definition of mailer.conf
makefs: 3 error(s) and 25009 warning(s) in mtree manifest

As of r304288 I get none of this, tags is in the METALOG and makefs
recognizes it. What is the os rel of the host running makefs?

I do have some duplicates in the METALOG but makefs is not warning about
them or failing. I'm guessing -D is being passed.

--
Regards,
Bryan Drewery

Sean Bruno

2016-08-19 17:35:42 UTC

Post by Bryan Drewery
As of r304288 I get none of this, tags is in the METALOG and makefs
recognizes it. What is the os rel of the host running makefs?
I do have some duplicates in the METALOG but makefs is not warning about
them or failing. I'm guessing -D is being passed.

FreeBSD bruno 12.0-CURRENT FreeBSD 12.0-CURRENT #0 r303327: Mon Jul 25
20:54:25 MDT 2016 ***@bruno:/usr/obj/usr/src/sys/BRUNO amd64

Yeah, its definitely passing -D ... at least I see it being passed to
install.

+ INSTALL_PROG='install -U -M
/home/sbruno/bsd/fbsd_head/../mfsroot/METALOG.malta.tmp -D
/home/sbruno/bsd/fbsd_head/../mfsroot/malta'
+ echo '*** Deleting old file system..'
*** Deleting old file system..
+ install -U -M /home/sbruno/bsd/fbsd_head/../mfsroot/METALOG.malta.tmp
-D /home/sbruno/bsd/fbsd_head/../mfsroot/malta -d
/home/sbruno/bsd/fbsd_head/../mfsroot/malta
+ chflags -R noschg /home/sbruno/bsd/fbsd_head/../mfsroot/malta
+ rm -rf /home/sbruno/bsd/fbsd_head/../mfsroot/malta
+ rm -rf /home/sbruno/bsd/fbsd_head/../tmp/malta
+ echo '*** Creating new filesystem...'
*** Creating new filesystem...
+ mkdir -p /home/sbruno/bsd/fbsd_head/../mfsroot/malta
+ mkdir -p /home/sbruno/bsd/fbsd_head/../tmp/malta
+ rsync -arH /home/sbruno/bsd/fbsd_head/../root/mips/
/home/sbruno/bsd/fbsd_head/../mfsroot/malta
+ echo '/dev/ada0 / ufs rw 1 1'
+ install -U -M /home/sbruno/bsd/fbsd_head/../mfsroot/METALOG.malta.tmp
-D /home/sbruno/bsd/fbsd_head/../mfsroot/malta -m 644 -o root -g wheel
/home/sbruno/bsd/fbsd_head/../tmp/malta/fstab
/home/sbruno/bsd/fbsd_head/../mfsroot/malta/etc/
+ echo 'autosize_enable="YES"'
+ echo 'autosize_rootfs="/dev/ada0"'
+ echo 'sshd_enable="YES"'
+ install -U -M /home/sbruno/bsd/fbsd_head/../mfsroot/METALOG.malta.tmp
-D /home/sbruno/bsd/fbsd_head/../mfsroot/malta -m 644 -o root -g wheel
/home/sbruno/bsd/fbsd_head/../tmp/malta/rc.conf
/home/sbruno/bsd/fbsd_head/../mfsroot/malta/etc/
+ install -U -M /home/sbruno/bsd/fbsd_head/../mfsroot/METALOG.malta.tmp
-D /home/sbruno/bsd/fbsd_head/../mfsroot/malta -m 755 -o root -g wheel
../freebsd-wifi-build/build/bin/..//files.full/autosize
/home/sbruno/bsd/fbsd_head/../mfsroot/malta/etc/rc.d/
+ ../freebsd-wifi-build/build/bin/filter.pl
/home/sbruno/bsd/fbsd_head/../root/mips/METALOG
/home/sbruno/bsd/fbsd_head/../mfsroot/METALOG.malta.tmp
/home/sbruno/bsd/fbsd_head/../mfsroot/METALOG.malta.mfsroot
+ echo '**** Done.'
**** Done.
+ echo '*** Target Done.'
*** Target Done.
+ echo $'*** Target\t\t: fullfsimage'
*** Target : fullfsimage
+ ../freebsd-wifi-build/build/bin/build_fullfsimage malta
*** Running makefs to build image ..
*** from /home/sbruno/bsd/fbsd_head/../mfsroot/malta ..
/home/sbruno/bsd/fbsd_head/../mfsroot/METALOG.malta:54: warning:
duplicate definition of var

Bryan Drewery

2016-08-19 17:38:21 UTC

FreeBSD bruno 12.0-CURRENT FreeBSD 12.0-CURRENT #0 r303327: Mon Jul 25
Yeah, its definitely passing -D ... at least I see it being passed to
install.
+ INSTALL_PROG='install -U -M
/home/sbruno/bsd/fbsd_head/../mfsroot/METALOG.malta.tmp -D
/home/sbruno/bsd/fbsd_head/../mfsroot/malta'
+ echo '*** Deleting old file system..'
*** Deleting old file system..
+ install -U -M /home/sbruno/bsd/fbsd_head/../mfsroot/METALOG.malta.tmp
-D /home/sbruno/bsd/fbsd_head/../mfsroot/malta -d
/home/sbruno/bsd/fbsd_head/../mfsroot/malta
+ chflags -R noschg /home/sbruno/bsd/fbsd_head/../mfsroot/malta
+ rm -rf /home/sbruno/bsd/fbsd_head/../mfsroot/malta
+ rm -rf /home/sbruno/bsd/fbsd_head/../tmp/malta
+ echo '*** Creating new filesystem...'
*** Creating new filesystem...
+ mkdir -p /home/sbruno/bsd/fbsd_head/../mfsroot/malta
+ mkdir -p /home/sbruno/bsd/fbsd_head/../tmp/malta
+ rsync -arH /home/sbruno/bsd/fbsd_head/../root/mips/
/home/sbruno/bsd/fbsd_head/../mfsroot/malta
+ echo '/dev/ada0 / ufs rw 1 1'
+ install -U -M /home/sbruno/bsd/fbsd_head/../mfsroot/METALOG.malta.tmp
-D /home/sbruno/bsd/fbsd_head/../mfsroot/malta -m 644 -o root -g wheel
/home/sbruno/bsd/fbsd_head/../tmp/malta/fstab
/home/sbruno/bsd/fbsd_head/../mfsroot/malta/etc/
+ echo 'autosize_enable="YES"'
+ echo 'autosize_rootfs="/dev/ada0"'
+ echo 'sshd_enable="YES"'
+ install -U -M /home/sbruno/bsd/fbsd_head/../mfsroot/METALOG.malta.tmp
-D /home/sbruno/bsd/fbsd_head/../mfsroot/malta -m 644 -o root -g wheel
/home/sbruno/bsd/fbsd_head/../tmp/malta/rc.conf
/home/sbruno/bsd/fbsd_head/../mfsroot/malta/etc/
+ install -U -M /home/sbruno/bsd/fbsd_head/../mfsroot/METALOG.malta.tmp
-D /home/sbruno/bsd/fbsd_head/../mfsroot/malta -m 755 -o root -g wheel
../freebsd-wifi-build/build/bin/..//files.full/autosize
/home/sbruno/bsd/fbsd_head/../mfsroot/malta/etc/rc.d/
+ ../freebsd-wifi-build/build/bin/filter.pl
/home/sbruno/bsd/fbsd_head/../root/mips/METALOG
/home/sbruno/bsd/fbsd_head/../mfsroot/METALOG.malta.tmp
/home/sbruno/bsd/fbsd_head/../mfsroot/METALOG.malta.mfsroot
+ echo '**** Done.'
**** Done.
+ echo '*** Target Done.'
*** Target Done.
+ echo $'*** Target\t\t: fullfsimage'
*** Target : fullfsimage
+ ../freebsd-wifi-build/build/bin/build_fullfsimage malta
*** Running makefs to build image ..
*** from /home/sbruno/bsd/fbsd_head/../mfsroot/malta ..
duplicate definition of var

Missing from this is an answer to any of my questions!

What script is this?

What flags are passed to makefs? (The -D above is to install). Passing
-D to my makefs doesn't yield any error anyhow.

What makefs "version" / What OS rel is the host?

--
Regards,
Bryan Drewery

Sean Bruno

2016-08-19 17:59:30 UTC

Missing from this is an answer to any of my questions!

HA! Well, I'm not quite awake as it is. I'll try to do better.

Post by Bryan Drewery
What script is this?

From the FreeBSD wifi scripts (not in tree).

https://github.com/freebsd/freebsd-wifi-build

I believe the makefs command we're looking at is the one in
build/bin/build_fullfsimage

Post by Bryan Drewery
What flags are passed to makefs? (The -D above is to install). Passing
-D to my makefs doesn't yield any error anyhow.

+ cp -f /home/sbruno/bsd/fbsd_head/../mfsroot/METALOG.malta.mfsroot
/home/sbruno/bsd/fbsd_head/../mfsroot/METALOG.malta
+ echo '*** Running makefs to build image .. '
*** Running makefs to build image ..
+ echo '*** from /home/sbruno/bsd/fbsd_head/../mfsroot/malta ..'
*** from /home/sbruno/bsd/fbsd_head/../mfsroot/malta ..
+ mkdir -p /home/sbruno/bsd/fbsd_head/../img/
+ cd /home/sbruno/bsd/fbsd_head/../mfsroot/malta
+ makefs -D -t ffs -M 4294967296 -B be -o 'version=2' -f 1048576
/home/sbruno/bsd/fbsd_head/../img//fullroot-malta.img
/home/sbruno/bsd/fbsd_head/../mfsroot/METALOG.malta
/home/sbruno/bsd/fbsd_head/../mfsroot/METALOG.malta:54: warning:
duplicate definition of var
/home/sbruno/bsd/fbsd_head/../mfsroot/METALOG.malta:103: warning:
duplicate definition of usr
/home/sbruno/bsd/fbsd_head/../mfsroot/METALOG.malta:848: warning:
duplicate definition of include
/home/sbruno/bsd/fbsd_head/../mfsroot/METALOG.malta:1020: warning:
duplicate definition of lib
/home/sbruno/bsd/fbsd_head/../mfsroot/METALOG.malta:1049: warning:
duplicate definition of usr

Post by Bryan Drewery
What makefs "version" / What OS rel is the host?

FreeBSD bruno 12.0-CURRENT FreeBSD 12.0-CURRENT #0 r303327: Mon Jul 25
20:54:25 MDT 2016 ***@bruno:/usr/obj/usr/src/sys/BRUNO amd64

Bryan Drewery

2016-08-19 18:04:25 UTC

Post by Sean Bruno
+ makefs -D -t ffs -M 4294967296 -B be -o 'version=2' -f 1048576
/home/sbruno/bsd/fbsd_head/../img//fullroot-malta.img
/home/sbruno/bsd/fbsd_head/../mfsroot/METALOG.malta

Here's the problem. The METALOG is a mtree file, not a makefs manifest.

It needs to pass it as -F and use '.' or full path for the directory.

Such as:

makefs -D -t ffs -M 4294967296 -B be -o 'version=2' -f 1048576
-F /home/sbruno/bsd/fbsd_head/../mfsroot/METALOG.malta
/home/sbruno/bsd/fbsd_head/../img//fullroot-malta.img
.

I get the million errors with the quoted form and none with the
suggested form.

--
Regards,
Bryan Drewery

Sean Bruno

2016-08-19 19:48:20 UTC

Post by Sean Bruno
+ makefs -D -t ffs -M 4294967296 -B be -o 'version=2' -f 1048576
/home/sbruno/bsd/fbsd_head/../img//fullroot-malta.img
/home/sbruno/bsd/fbsd_head/../mfsroot/METALOG.malta

Here's the problem. The METALOG is a mtree file, not a makefs manifest.
It needs to pass it as -F and use '.' or full path for the directory.
makefs -D -t ffs -M 4294967296 -B be -o 'version=2' -f 1048576
-F /home/sbruno/bsd/fbsd_head/../mfsroot/METALOG.malta
/home/sbruno/bsd/fbsd_head/../img//fullroot-malta.img
.
I get the million errors with the quoted form and none with the
suggested form.

Indeed. I've fixed this up in the freebsd wifi scripts and fired off a
pull request for it.

Has "install" lost the ability to do relative directories completely?

install: open /home/sbruno/bsd/fbsd_head/../mfsroot/METALOG.malta.tmp:
No such file or directory

ls -l /home/sbruno/bsd/fbsd_head/../mfsroot/METALOG.malta.tmp
-rw-r--r-- 1 sbruno sbruno 203 Aug 19 13:30
/home/sbruno/bsd/fbsd_head/../mfsroot/METALOG.malta.tmp

--- full script output ---
+ X_BASEDIR=../freebsd-wifi-build/build/bin/../
+ INSTALL_PROG='install -U -M
/home/sbruno/bsd/fbsd_head/../mfsroot/METALOG.malta.tmp -D
/home/sbruno/bsd/fbsd_head/../mfsroot/malta'
+ echo '*** Deleting old file system..'
*** Deleting old file system..
+ install -U -M /home/sbruno/bsd/fbsd_head/../mfsroot/METALOG.malta.tmp
-D /home/sbruno/bsd/fbsd_head/../mfsroot/malta -d
/home/sbruno/bsd/fbsd_head/../mfsroot/malta
install: open /home/sbruno/bsd/fbsd_head/../mfsroot/METALOG.malta.tmp:
No such file or directory
+ chflags -R noschg /home/sbruno/bsd/fbsd_head/../mfsroot/malta
+ rm -rf /home/sbruno/bsd/fbsd_head/../mfsroot/malta
+ rm -rf /home/sbruno/bsd/fbsd_head/../tmp/malta
+ echo '*** Creating new filesystem...'
*** Creating new filesystem...
+ mkdir -p /home/sbruno/bsd/fbsd_head/../mfsroot/malta
+ mkdir -p /home/sbruno/bsd/fbsd_head/../tmp/malta
+ rsync -arH /home/sbruno/bsd/fbsd_head/../root/mips/
/home/sbruno/bsd/fbsd_head/../mfsroot/malta
+ echo '/dev/ada0 / ufs rw 1 1'
+ install -U -M /home/sbruno/bsd/fbsd_head/../mfsroot/METALOG.malta.tmp
-D /home/sbruno/bsd/fbsd_head/../mfsroot/malta -m 644 -o root -g wheel
/home/sbruno/bsd/fbsd_head/../tmp/malta/fstab /ho
me/sbruno/bsd/fbsd_head/../mfsroot/malta/etc/
+ echo 'autosize_enable="YES"'
+ echo 'autosize_rootfs="/dev/ada0"'
+ echo 'sshd_enable="YES"'
+ install -U -M /home/sbruno/bsd/fbsd_head/../mfsroot/METALOG.malta.tmp
-D /home/sbruno/bsd/fbsd_head/../mfsroot/malta -m 644 -o root -g wheel
/home/sbruno/bsd/fbsd_head/../tmp/malta/rc.conf /
home/sbruno/bsd/fbsd_head/../mfsroot/malta/etc/
+ install -U -M /home/sbruno/bsd/fbsd_head/../mfsroot/METALOG.malta.tmp
-D /home/sbruno/bsd/fbsd_head/../mfsroot/malta -m 755 -o root -g wheel
../freebsd-wifi-build/build/bin/..//files.full/au
tosize /home/sbruno/bsd/fbsd_head/../mfsroot/malta/etc/rc.d/
+ ../freebsd-wifi-build/build/bin/filter.pl
/home/sbruno/bsd/fbsd_head/../root/mips/METALOG
/home/sbruno/bsd/fbsd_head/../mfsroot/METALOG.malta.tmp
/home/sbruno/bsd/fbsd_head/../mfsroot/METALOG.malta.mfsroot
+ echo '**** Done.'
**** Done.
*** Target Done.
*** Target : fullfsimage
*** Running makefs to build image ..
*** from /home/sbruno/bsd/fbsd_head/../mfsroot/malta ..
makefs: ./usr/local/bin: missing directory in specification
makefs: failed at line 26660 of the specification

Bryan Drewery

2016-08-19 19:54:11 UTC

Post by Sean Bruno
+ makefs -D -t ffs -M 4294967296 -B be -o 'version=2' -f 1048576
/home/sbruno/bsd/fbsd_head/../img//fullroot-malta.img
/home/sbruno/bsd/fbsd_head/../mfsroot/METALOG.malta

Indeed. I've fixed this up in the freebsd wifi scripts and fired off a
pull request for it.
Has "install" lost the ability to do relative directories completely?
No such file or directory

...

Post by Sean Bruno
ls -l /home/sbruno/bsd/fbsd_head/../mfsroot/METALOG.malta.tmp
-rw-r--r-- 1 sbruno sbruno 203 Aug 19 13:30
/home/sbruno/bsd/fbsd_head/../mfsroot/METALOG.malta.tmp
--- full script output ---
+ X_BASEDIR=../freebsd-wifi-build/build/bin/../
+ INSTALL_PROG='install -U -M
/home/sbruno/bsd/fbsd_head/../mfsroot/METALOG.malta.tmp -D
/home/sbruno/bsd/fbsd_head/../mfsroot/malta'
+ echo '*** Deleting old file system..'
*** Deleting old file system..
+ install -U -M /home/sbruno/bsd/fbsd_head/../mfsroot/METALOG.malta.tmp
-D /home/sbruno/bsd/fbsd_head/../mfsroot/malta -d
/home/sbruno/bsd/fbsd_head/../mfsroot/malta
No such file or directory

My guess is that at this point in the install the directory for
METALOG.malta.tmp is likely missing and not created yet. In the failing
command -d is being used to create mfsroot/malta. -d creates parents,
so perhaps mfsroot/ is not present yet either.

Post by Sean Bruno
+ chflags -R noschg /home/sbruno/bsd/fbsd_head/../mfsroot/malta
+ rm -rf /home/sbruno/bsd/fbsd_head/../mfsroot/malta
+ rm -rf /home/sbruno/bsd/fbsd_head/../tmp/malta
+ echo '*** Creating new filesystem...'
*** Creating new filesystem...
+ mkdir -p /home/sbruno/bsd/fbsd_head/../mfsroot/malta
+ mkdir -p /home/sbruno/bsd/fbsd_head/../tmp/malta
+ rsync -arH /home/sbruno/bsd/fbsd_head/../root/mips/
/home/sbruno/bsd/fbsd_head/../mfsroot/malta
+ echo '/dev/ada0 / ufs rw 1 1'
+ install -U -M /home/sbruno/bsd/fbsd_head/../mfsroot/METALOG.malta.tmp
-D /home/sbruno/bsd/fbsd_head/../mfsroot/malta -m 644 -o root -g wheel
/home/sbruno/bsd/fbsd_head/../tmp/malta/fstab /ho
me/sbruno/bsd/fbsd_head/../mfsroot/malta/etc/
+ echo 'autosize_enable="YES"'
+ echo 'autosize_rootfs="/dev/ada0"'
+ echo 'sshd_enable="YES"'
+ install -U -M /home/sbruno/bsd/fbsd_head/../mfsroot/METALOG.malta.tmp
-D /home/sbruno/bsd/fbsd_head/../mfsroot/malta -m 644 -o root -g wheel
/home/sbruno/bsd/fbsd_head/../tmp/malta/rc.conf /
home/sbruno/bsd/fbsd_head/../mfsroot/malta/etc/
+ install -U -M /home/sbruno/bsd/fbsd_head/../mfsroot/METALOG.malta.tmp
-D /home/sbruno/bsd/fbsd_head/../mfsroot/malta -m 755 -o root -g wheel
../freebsd-wifi-build/build/bin/..//files.full/au
tosize /home/sbruno/bsd/fbsd_head/../mfsroot/malta/etc/rc.d/
+ ../freebsd-wifi-build/build/bin/filter.pl
/home/sbruno/bsd/fbsd_head/../root/mips/METALOG
/home/sbruno/bsd/fbsd_head/../mfsroot/METALOG.malta.tmp
/home/sbruno/bsd/fbsd_head/../mfsroot/METALOG.malta.mfsroot
+ echo '**** Done.'
**** Done.
*** Target Done.
*** Target : fullfsimage
*** Running makefs to build image ..
*** from /home/sbruno/bsd/fbsd_head/../mfsroot/malta ..
makefs: ./usr/local/bin: missing directory in specification
makefs: failed at line 26660 of the specification

--
Regards,
Bryan Drewery

Sean Bruno

2016-08-19 20:11:46 UTC

Post by Sean Bruno
--- full script output ---
+ X_BASEDIR=../freebsd-wifi-build/build/bin/../
+ INSTALL_PROG='install -U -M
/home/sbruno/bsd/fbsd_head/../mfsroot/METALOG.malta.tmp -D
/home/sbruno/bsd/fbsd_head/../mfsroot/malta'
+ echo '*** Deleting old file system..'
*** Deleting old file system..
+ install -U -M /home/sbruno/bsd/fbsd_head/../mfsroot/METALOG.malta.tmp
-D /home/sbruno/bsd/fbsd_head/../mfsroot/malta -d
/home/sbruno/bsd/fbsd_head/../mfsroot/malta
No such file or directory

Hrm ... I added a "ls -ld" to the script immediately preceding the
install directive, it appears to be there.

+ INSTALL_PROG='install -U -M
/home/sbruno/bsd/fbsd_head/../mfsroot/METALOG.malta.tmp -D
/home/sbruno/bsd/fbsd_head/../mfsroot/malta'
+ echo '*** Deleting old file system..'
*** Deleting old file system..
+ ls -ld /home/sbruno/bsd/fbsd_head/../mfsroot/METALOG.malta.tmp
-rw-r--r-- 1 sbruno sbruno 203 Aug 19 13:45
/home/sbruno/bsd/fbsd_head/../mfsroot/METALOG.malta.tmp
+ install -U -M /home/sbruno/bsd/fbsd_head/../mfsroot/METALOG.malta.tmp
-D /home/sbruno/bsd/fbsd_head/../mfsroot/malta -d
/home/sbruno/bsd/fbsd_head/../mfsroot/malta
+ chflags -R noschg /home/sbruno/bsd/fbsd_head/../mfsroot/malta
+ rm -rf /home/sbruno/bsd/fbsd_head/../mfsroot/malta
+ rm -rf /home/sbruno/bsd/fbsd_head/../tmp/malta
+ echo '*** Creating new filesystem...'
*** Creating new filesystem...
+ mkdir -p /home/sbruno/bsd/fbsd_head/../mfsroot/malta
+ mkdir -p /home/sbruno/bsd/fbsd_head/../tmp/malta
+ rsync -arH /home/sbruno/bsd/fbsd_head/../root/mips/
/home/sbruno/bsd/fbsd_head/../mfsroot/malta
+ echo '/dev/ada0 / ufs rw 1 1'
+ install -U -M /home/sbruno/bsd/fbsd_head/../mfsroot/METALOG.malta.tmp
-D /home/sbruno/bsd/fbsd_head/../mfsroot/malta -m 644 -o root -g wheel
/home/sbruno/bsd/fbsd_head/../tmp/malta/fstab
/home/sbruno/bsd/fbsd_head/../mfsroot/malta/etc/
+ echo 'autosize_enable="YES"'
+ echo 'autosize_rootfs="/dev/ada0"'
+ echo 'sshd_enable="YES"'
+ install -U -M /home/sbruno/bsd/fbsd_head/../mfsroot/METALOG.malta.tmp
-D /home/sbruno/bsd/fbsd_head/../mfsroot/malta -m 644 -o root -g wheel
/home/sbruno/bsd/fbsd_head/../tmp/malta/rc.conf
/home/sbruno/bsd/fbsd_head/../mfsroot/malta/etc/
+ install -U -M /home/sbruno/bsd/fbsd_head/../mfsroot/METALOG.malta.tmp
-D /home/sbruno/bsd/fbsd_head/../mfsroot/malta -m 755 -o root -g wheel
../freebsd-wifi-build/build/bin/..//files.full/autosize
/home/sbruno/bsd/fbsd_head/../mfsroot/malta/etc/rc.d/
+ ../freebsd-wifi-build/build/bin/filter.pl
/home/sbruno/bsd/fbsd_head/../root/mips/METALOG
/home/sbruno/bsd/fbsd_head/../mfsroot/METALOG.malta.tmp
/home/sbruno/bsd/fbsd_head/../mfsroot/METALOG.malta.mfsroot
+ echo '**** Done.'
**** Done.
*** Target Done.
*** Target : fullfsimage
*** Running makefs to build image ..
*** from /home/sbruno/bsd/fbsd_head/../mfsroot/malta ..
makefs: ./usr/local/bin: missing directory in specification
makefs: failed at line 26660 of the specification

Bryan Drewery

2016-08-19 20:14:32 UTC

There seems to be no error now?

Post by Sean Bruno
+ chflags -R noschg /home/sbruno/bsd/fbsd_head/../mfsroot/malta
+ rm -rf /home/sbruno/bsd/fbsd_head/../mfsroot/malta
+ rm -rf /home/sbruno/bsd/fbsd_head/../tmp/malta
+ echo '*** Creating new filesystem...'
*** Creating new filesystem...
+ mkdir -p /home/sbruno/bsd/fbsd_head/../mfsroot/malta
+ mkdir -p /home/sbruno/bsd/fbsd_head/../tmp/malta
+ rsync -arH /home/sbruno/bsd/fbsd_head/../root/mips/
/home/sbruno/bsd/fbsd_head/../mfsroot/malta
+ echo '/dev/ada0 / ufs rw 1 1'
+ install -U -M /home/sbruno/bsd/fbsd_head/../mfsroot/METALOG.malta.tmp
-D /home/sbruno/bsd/fbsd_head/../mfsroot/malta -m 644 -o root -g wheel
/home/sbruno/bsd/fbsd_head/../tmp/malta/fstab
/home/sbruno/bsd/fbsd_head/../mfsroot/malta/etc/
+ echo 'autosize_enable="YES"'
+ echo 'autosize_rootfs="/dev/ada0"'
+ echo 'sshd_enable="YES"'
+ install -U -M /home/sbruno/bsd/fbsd_head/../mfsroot/METALOG.malta.tmp
-D /home/sbruno/bsd/fbsd_head/../mfsroot/malta -m 644 -o root -g wheel
/home/sbruno/bsd/fbsd_head/../tmp/malta/rc.conf
/home/sbruno/bsd/fbsd_head/../mfsroot/malta/etc/
+ install -U -M /home/sbruno/bsd/fbsd_head/../mfsroot/METALOG.malta.tmp
-D /home/sbruno/bsd/fbsd_head/../mfsroot/malta -m 755 -o root -g wheel
../freebsd-wifi-build/build/bin/..//files.full/autosize
/home/sbruno/bsd/fbsd_head/../mfsroot/malta/etc/rc.d/
+ ../freebsd-wifi-build/build/bin/filter.pl
/home/sbruno/bsd/fbsd_head/../root/mips/METALOG
/home/sbruno/bsd/fbsd_head/../mfsroot/METALOG.malta.tmp
/home/sbruno/bsd/fbsd_head/../mfsroot/METALOG.malta.mfsroot
+ echo '**** Done.'
**** Done.
*** Target Done.
*** Target : fullfsimage
*** Running makefs to build image ..
*** from /home/sbruno/bsd/fbsd_head/../mfsroot/malta ..
makefs: ./usr/local/bin: missing directory in specification
makefs: failed at line 26660 of the specification

--
Regards,
Bryan Drewery

Sean Bruno

2016-08-19 20:19:00 UTC

Post by Sean Bruno
Hrm ... I added a "ls -ld" to the script immediately preceding the
install directive, it appears to be there.
+ INSTALL_PROG='install -U -M
/home/sbruno/bsd/fbsd_head/../mfsroot/METALOG.malta.tmp -D
/home/sbruno/bsd/fbsd_head/../mfsroot/malta'
+ echo '*** Deleting old file system..'
*** Deleting old file system..
+ ls -ld /home/sbruno/bsd/fbsd_head/../mfsroot/METALOG.malta.tmp
-rw-r--r-- 1 sbruno sbruno 203 Aug 19 13:45
/home/sbruno/bsd/fbsd_head/../mfsroot/METALOG.malta.tmp
+ install -U -M /home/sbruno/bsd/fbsd_head/../mfsroot/METALOG.malta.tmp
-D /home/sbruno/bsd/fbsd_head/../mfsroot/malta -d
/home/sbruno/bsd/fbsd_head/../mfsroot/malta

There seems to be no error now?

Yeah, this looks like a error in the scripts. Cleaned and removed the
mfsroot dir, and the "ls -ld" and the install error appear to be in
correspondence.

+ INSTALL_PROG='install -U -M
/home/sbruno/bsd/fbsd_head/../mfsroot/METALOG.malta.tmp -D
/home/sbruno/bsd/fbsd_head/../mfsroot/malta'
+ echo '*** Deleting old file system..'
*** Deleting old file system..
+ ls -ld /home/sbruno/bsd/fbsd_head/../mfsroot/METALOG.malta.tmp
ls: /home/sbruno/bsd/fbsd_head/../mfsroot/METALOG.malta.tmp: No such
file or directory
+ install -U -M /home/sbruno/bsd/fbsd_head/../mfsroot/METALOG.malta.tmp
-D /home/sbruno/bsd/fbsd_head/../mfsroot/malta -d
/home/sbruno/bsd/fbsd_head/../mfsroot/malta
install: open /home/sbruno/bsd/fbsd_head/../mfsroot/METALOG.malta.tmp:
No such file or directory
+ chflags -R noschg /home/sbruno/bsd/fbsd_head/../mfsroot/malta
+ rm -rf /home/sbruno/bsd/fbsd_head/../mfsroot/malta
+ rm -rf /home/sbruno/bsd/fbsd_head/../tmp/malta
+ echo '*** Creating new filesystem...'

Sean Bruno

2016-08-19 20:34:27 UTC

There seems to be no error now?

Yeah, this looks like a error in the scripts. Cleaned and removed the
mfsroot dir, and the "ls -ld" and the install error appear to be in
correspondence.
+ INSTALL_PROG='install -U -M
/home/sbruno/bsd/fbsd_head/../mfsroot/METALOG.malta.tmp -D
/home/sbruno/bsd/fbsd_head/../mfsroot/malta'
+ echo '*** Deleting old file system..'
*** Deleting old file system..
+ ls -ld /home/sbruno/bsd/fbsd_head/../mfsroot/METALOG.malta.tmp
ls: /home/sbruno/bsd/fbsd_head/../mfsroot/METALOG.malta.tmp: No such
file or directory
+ install -U -M /home/sbruno/bsd/fbsd_head/../mfsroot/METALOG.malta.tmp
-D /home/sbruno/bsd/fbsd_head/../mfsroot/malta -d
/home/sbruno/bsd/fbsd_head/../mfsroot/malta
No such file or directory
+ chflags -R noschg /home/sbruno/bsd/fbsd_head/../mfsroot/malta
+ rm -rf /home/sbruno/bsd/fbsd_head/../mfsroot/malta
+ rm -rf /home/sbruno/bsd/fbsd_head/../tmp/malta
+ echo '*** Creating new filesystem...'

adrian:

Hmmm ... what should the METALOG.malta.tmp file contain at this point,
it looks like only a fewlines once its created, which doesn't happen at
this point.

./etc//fstab type=file uname=root gname=wheel mode=0644 size=23
./etc//rc.conf type=file uname=root gname=wheel mode=0644 size=68
./etc/rc.d//autosize type=file uname=root gname=wheel mode=0755 size=653
. type=dir mode=0755

Adrian Chadd

2016-08-19 21:27:38 UTC

hi,

ok, so here's the hilarity.

We don't really have tools for consistently manipulating
metalog/manifest files. So, in order to do things like
add/delete/modify things in the manifest file, we need to do some ..
hackery.

So that's what filter.pl does. It takes in the metalog file and does a
replace or append. It currently doesn't have a "delete" method.

the .tmp file is "the extra files that we add /after/ the system
installs". Ie, it's the customisations done. It goes through filter.pl
and then the resultant metalog file has the complete thing that's
going into the image.

When I get around to adding package support for mips (not mips_ap)
then it'll also have a second intermediary metalog file that has the
combination of all of the package installs, so makefs knows what to
do.

Ok, so for metalog versus manifest. I remember doing it one way and
then brooks saying "Don't do that, here's the bugs". I'll have to go
over that and see.

Thanks,

-adrian

There seems to be no error now?

Yeah, this looks like a error in the scripts. Cleaned and removed the
mfsroot dir, and the "ls -ld" and the install error appear to be in
correspondence.
+ INSTALL_PROG='install -U -M
/home/sbruno/bsd/fbsd_head/../mfsroot/METALOG.malta.tmp -D
/home/sbruno/bsd/fbsd_head/../mfsroot/malta'
+ echo '*** Deleting old file system..'
*** Deleting old file system..
+ ls -ld /home/sbruno/bsd/fbsd_head/../mfsroot/METALOG.malta.tmp
ls: /home/sbruno/bsd/fbsd_head/../mfsroot/METALOG.malta.tmp: No such
file or directory
+ install -U -M /home/sbruno/bsd/fbsd_head/../mfsroot/METALOG.malta.tmp
-D /home/sbruno/bsd/fbsd_head/../mfsroot/malta -d
/home/sbruno/bsd/fbsd_head/../mfsroot/malta
No such file or directory
+ chflags -R noschg /home/sbruno/bsd/fbsd_head/../mfsroot/malta
+ rm -rf /home/sbruno/bsd/fbsd_head/../mfsroot/malta
+ rm -rf /home/sbruno/bsd/fbsd_head/../tmp/malta
+ echo '*** Creating new filesystem...'

Hmmm ... what should the METALOG.malta.tmp file contain at this point,
it looks like only a fewlines once its created, which doesn't happen at
this point.
./etc//fstab type=file uname=root gname=wheel mode=0644 size=23
./etc//rc.conf type=file uname=root gname=wheel mode=0644 size=68
./etc/rc.d//autosize type=file uname=root gname=wheel mode=0755 size=653
. type=dir mode=0755

Warner Losh

2016-08-19 21:38:14 UTC

Post by Adrian Chadd
hi,
ok, so here's the hilarity.
We don't really have tools for consistently manipulating
metalog/manifest files. So, in order to do things like
add/delete/modify things in the manifest file, we need to do some ..
hackery.
So that's what filter.pl does. It takes in the metalog file and does a
replace or append. It currently doesn't have a "delete" method.
the .tmp file is "the extra files that we add /after/ the system
installs". Ie, it's the customisations done. It goes through filter.pl
and then the resultant metalog file has the complete thing that's
going into the image.
When I get around to adding package support for mips (not mips_ap)
then it'll also have a second intermediary metalog file that has the
combination of all of the package installs, so makefs knows what to
do.
Ok, so for metalog versus manifest. I remember doing it one way and
then brooks saying "Don't do that, here's the bugs". I'll have to go
over that and see.

NanoBSD will eventually follow suit as well. I really want to be able to
create entire images, including installing packages, as a normal user.
Making sure all this works is on that path. I have similar scripts to Adrian's
filter script as well for manipulating mtree files (and makefs does take mtree
files, it's just picky about the exact form, sadly).

Warner

Adrian Chadd

2016-08-21 21:30:20 UTC

----------------------------------------------------------------------
diff --git a/build/bin/build_freebsd b/build/bin/build_freebsd
index d7fe4c2..04169b4 100755
--- a/build/bin/build_freebsd
+++ b/build/bin/build_freebsd
@@ -112,7 +112,7 @@ fi
while [ -n "$1" ]; do
echo "*** Stage : $1"
- if [ "$1" = "installworld" ]; then
+ if [ "$1" = "installworld" -a ${X_BUILD_BASE_CFG} = "mips_ap" ];
then
mkdir -p ${X_DESTDIR}/usr/local/bin/
mkdir -p ${X_DESTDIR}/bsdbox/
mkdir -p ${X_DESTDIR}/usr/local/libdata/athprom/

This happens because when adding LOCAL_TOOL_DIRS to install the ath
bits, it's installing it in /usr/local/* without doing an install -d
/usr/local/bin , /usr/local/libexec/, etc beforehand.

I bet doing a "make installworld" without /usr/local/ fully populated
(ie, no ports) won't create the /usr/local/ subdirectories.

---------------------------------------------------------------------
*** Target : fullroot
*** Deleting old file system..
No such file or directory
*** Creating new filesystem...
**** Done.
*** Target Done.
*** Target : fullfsimage
*** Running makefs to build image ..
*** from /home/sbruno/bsd/fbsd_head/../mfsroot/malta ..
makefs: ./boot/kernel.MALTA: missing directory in specification
makefs: failed at line 26663 of the specification

This happens because the option that sets the kernel directory for the
installworld/installkernel pass (KODIR and KMODDIR) doesn't first do
an install -d /boot/kernel.XXX/, so the metalog entry isn't populated.

I could write another tool like filter.pl that allows for entries to
be inserted before other entries, so we could say "hey, please
populate /boot/kernel.XXX before you populate /boot/kernel.XXX/YYY",
but I think I'd rather just fix it in -HEAD.

-adrian

Adrian Chadd

2016-08-21 21:32:57 UTC