xstr, mkstr

Post by Eitan Adler
Why are these tools in base? As far as I could tell these tools are
un-used in the build process and otherwise specialized tools. Is there
any reason we still have them in the toolchain or in base?

Anyone? They don't cause acute problems, but having specialized and
specific tools like this don't seem like they belong.

--
Eitan Adler

Gary Jennejohn

2018-07-03 09:33:45 UTC

On Mon, 2 Jul 2018 21:58:43 -0700

Post by Eitan Adler

Anyone? They don't cause acute problems, but having specialized and
specific tools like this don't seem like they belong.

Reading the man pages it appears that these are old baggage left
over from the PDP/VAX legacy of BSD. Their whole prupose was to
reduce the memory consumed by strings in binaries on machines
with little RAM.

In fact, the BUGS section of mkstr(1) makes this very clear.

xstr is actually used in /usr/src/contrib/bmake/mk/prog.mk. I
can't tell whether this file is ever invoked.

It's commented out in some other makefiles.

There's a test for mkstr in
/usr/src/contrib/netbsd-tests/lib/libc/regex/t_exhaust.c.

Both are in /usr/src/usr.sbin/crunch/examples/really-big.conf.

--
Gary Jennejohn

Rodney W. Grimes

2018-07-03 14:45:19 UTC

Post by Eitan Adler

Anyone? They don't cause acute problems, but having specialized and
specific tools like this don't seem like they belong.

Since when was any of that the criteria for if something should
or should not be in a BSD operating system? One could apply that
type of critera from difference perspectives to almost anything
in the base system and ask for its removal.

tip/cu no longer belong, who has serial ports? (A modern MACbook
user who hasnt seen a serial port on a machine in a decade).
ME, screams an embeded board user.

What appears to be unused or specialized to one user can be
very differeent to another. It is best to keep an open mind
that there are many very diverse users of BSD.

These tools are "in base" because 1BSD and 3BSD put them there
as useful tools. And given that age they probably got used
a fair bit. So that answers the question as to why they are
there.

"As far as I can tell" needs to be a bit stronger, have you
removed them from the system and had an exp run done to make
sure they are not used?

Rather than trying to micro minimize the size of FreeBSD base
realizes that package base should completly optionalize
these types of things and then users can decide if they need/want
them. Then perhaps at that point start a large and complete
sweep of the system to try and decide what is and is not in
base. Doing this one little thing at a time is just a PITA
and IMHO a waste of time.

--
Rod Grimes ***@freebsd.org

Eitan Adler

2018-07-03 14:55:50 UTC

On 3 July 2018 at 07:45, Rodney W. Grimes

Post by Eitan Adler

Anyone? They don't cause acute problems, but having specialized and
specific tools like this don't seem like they belong.

What appears to be unused or specialized to one user can be
very differeent to another. It is best to keep an open mind
that there are many very diverse users of BSD.

<snark>
I absolutely agree with you! I did not know much about their history,
beyond them being used in low-memory environments.
In fact, I agree with you so much so that I think that before doing
anything about them, I should email an appropriate list to find out
more about them. Perhaps arch@ is a good list? hackers@ might also be
good, but tends to have less of a reply-rate.
</snark>

Post by Rodney W. Grimes
"As far as I can tell" needs to be a bit stronger,

Agreed. But before doing work we should first make sure that the work
will be productive.

Post by Rodney W. Grimes
have you
removed them from the system and had an exp run done to make
sure they are not used?

I have locally built a system without them and built several ports.
This is certainly not an exp-run, but want to hold off on filing for
an exp-run before this conversation about their usefulness is
complete.

--
Eitan Adler

Warner Losh

2018-07-03 15:26:30 UTC

Post by Rodney W. Grimes
"As far as I can tell" needs to be a bit stronger,
Agreed. But before doing work we should first make sure that the work
will be productive.

Xstr and most are best thought of as a final pass of the old pcc compiler.
This functionality moved into gcc before the egcs split back in the 90s.
The compiler has done this work for two decades now. Nobody uses it, or
needs it today.

Unlike tip or cu, nobody at all uses these.

Warner

John Baldwin

2018-07-03 16:05:51 UTC

Post by Rodney W. Grimes
"As far as I can tell" needs to be a bit stronger,
Agreed. But before doing work we should first make sure that the work
will be productive.

Agreed, these tools are quite obscure and I don't think _FreeBSD_ has ever
used them.

--
John Baldwin

Warner Losh

2018-07-03 16:35:10 UTC

Post by Rodney W. Grimes
"As far as I can tell" needs to be a bit stronger,
Agreed. But before doing work we should first make sure that the work
will be productive.

Xstr and most are best thought of as a final pass of the old pcc

compiler.

Post by Warner Losh
This functionality moved into gcc before the egcs split back in the 90s.
The compiler has done this work for two decades now. Nobody uses it, or
needs it today.
Unlike tip or cu, nobody at all uses these.

Agreed, these tools are quite obscure and I don't think _FreeBSD_ has ever
used them.

If we did, we had stopped by 3.x for sure.

Warner

Rodney W. Grimes

2018-07-03 16:41:44 UTC

[ Charset UTF-8 unsupported, converting... ]

Post by Rodney W. Grimes
"As far as I can tell" needs to be a bit stronger,
Agreed. But before doing work we should first make sure that the work
will be productive.

Agreed, these tools are quite obscure and I don't think _FreeBSD_ has ever
used them.

I thought someone already pointed out places in the src tree
that had references to them, though they may be commented out.

--
Rod Grimes ***@freebsd.org

Warner Losh

2018-07-03 16:56:38 UTC

On Tue, Jul 3, 2018, 11:41 AM Rodney W. Grimes <

Post by Rodney W. Grimes
[ Charset UTF-8 unsupported, converting... ]

Post by Rodney W. Grimes
"As far as I can tell" needs to be a bit stronger,
Agreed. But before doing work we should first make sure that the work
will be productive.

Xstr and most are best thought of as a final pass of the old pcc

compiler.

Post by Warner Losh
This functionality moved into gcc before the egcs split back in the

90s.

Post by Warner Losh
The compiler has done this work for two decades now. Nobody uses it, or
needs it today.
Unlike tip or cu, nobody at all uses these.

Agreed, these tools are quite obscure and I don't think _FreeBSD_ has

ever

Post by John Baldwin
used them.

I thought someone already pointed out places in the src tree
that had references to them, though they may

You removed the SHAREDSTRINGS support in r3493 back in 1994. It's been
unused since then. There are comments that suggest it never worked in
FreeBSD, but that may be 2.0 and newer only.

Warner

Rodney W. Grimes

2018-07-03 17:56:22 UTC

[ Charset UTF-8 unsupported, converting... ]

Post by Warner Losh
On Tue, Jul 3, 2018, 11:41 AM Rodney W. Grimes <

Post by Rodney W. Grimes
[ Charset UTF-8 unsupported, converting... ]

Post by Rodney W. Grimes
"As far as I can tell" needs to be a bit stronger,
Agreed. But before doing work we should first make sure that the work
will be productive.

Xstr and most are best thought of as a final pass of the old pcc

compiler.

Post by Warner Losh
This functionality moved into gcc before the egcs split back in the

90s.

Post by Warner Losh
The compiler has done this work for two decades now. Nobody uses it, or
needs it today.
Unlike tip or cu, nobody at all uses these.

Agreed, these tools are quite obscure and I don't think _FreeBSD_ has

ever

Post by John Baldwin
used them.

I thought someone already pointed out places in the src tree
that had references to them, though they may

You removed the SHAREDSTRINGS support in r3493 back in 1994. It's been
unused since then. There are comments that suggest it never worked in
FreeBSD, but that may be 2.0 and newer only.

Removed it, not commented it out. Someone else on this thread made
statement that stuff is refering to these tools, that could not been
what I removed in r3493.

I do see that some of what I removed in r3493 is now back as bmake
has moved to contrib/ and these strings are now there:
./contrib/bmake/mk/prog.mk:.if defined(SHAREDSTRINGS)
./contrib/bmake/mk/README:SHAREDSTRINGS If defined, a new .c.o rule is used that results in shared

Those are parts that I totally removed from
share/mk/{bsd.README,bsd.prog.mk} in r3493.

The only note I see about "never worked" is that prog.mk stuff
is suppose to have issues with Parallel make, thats in the
contrib/bmake/mk/README. Where do you have a reference to
never worked in FreeBSD?

--
Rod Grimes ***@freebsd.org

Bryan Drewery

2018-07-04 00:38:11 UTC

Post by Rodney W. Grimes
[ Charset UTF-8 unsupported, converting... ]

Post by Warner Losh
On Tue, Jul 3, 2018, 11:41 AM Rodney W. Grimes <

Post by Rodney W. Grimes
[ Charset UTF-8 unsupported, converting... ]

Post by Rodney W. Grimes
"As far as I can tell" needs to be a bit stronger,
Agreed. But before doing work we should first make sure that the work
will be productive.

Xstr and most are best thought of as a final pass of the old pcc

compiler.

Post by Warner Losh
This functionality moved into gcc before the egcs split back in the

90s.

Post by Warner Losh
The compiler has done this work for two decades now. Nobody uses it, or
needs it today.
Unlike tip or cu, nobody at all uses these.

Agreed, these tools are quite obscure and I don't think _FreeBSD_ has

ever

Post by John Baldwin
used them.

I thought someone already pointed out places in the src tree
that had references to them, though they may

You removed the SHAREDSTRINGS support in r3493 back in 1994. It's been
unused since then. There are comments that suggest it never worked in
FreeBSD, but that may be 2.0 and newer only.

Removed it, not commented it out. Someone else on this thread made
statement that stuff is refering to these tools, that could not been
what I removed in r3493.
I do see that some of what I removed in r3493 is now back as bmake
./contrib/bmake/mk/prog.mk:.if defined(SHAREDSTRINGS)
./contrib/bmake/mk/README:SHAREDSTRINGS If defined, a new .c.o rule is used that results in shared

These files (contrib/bmake/mk/*) are not used anywhere for FreeBSD. It's
just part of the bmake distribution.

On the otherhand I don't see the point in removing something that isn't
harming anything, is not worth talking about in terms of size or build
time, and is unlikely to need maintenance.

This argument of "not useful" is a bit subjective. Is cut useful when we
have awk? I won't go on.

Post by Rodney W. Grimes
Those are parts that I totally removed from
share/mk/{bsd.README,bsd.prog.mk} in r3493.
The only note I see about "never worked" is that prog.mk stuff
is suppose to have issues with Parallel make, thats in the
contrib/bmake/mk/README. Where do you have a reference to
never worked in FreeBSD?

--
Regards,
Bryan Drewery

Poul-Henning Kamp

2018-07-04 06:52:02 UTC

--------

Post by Bryan Drewery
On the otherhand I don't see the point in removing something that isn't
harming anything, is not worth talking about in terms of size or build
time, and is unlikely to need maintenance.

Source code is a liability, it is not an investment.

If the code is useless and not part of a published API, then it should
be removed.

Post by Bryan Drewery
This argument of "not useful" is a bit subjective. Is cut useful when we
have awk? I won't go on.

cut(1) & awk(1) are both part of the published API, so that comparison gets
you nowhere.

--
Poul-Henning Kamp | UNIX since Zilog Zeus 3.20
***@FreeBSD.ORG | TCP/IP since RFC 956
FreeBSD committer | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.

Warner Losh

2018-07-04 04:03:57 UTC

On Tue, Jul 3, 2018 at 11:56 AM, Rodney W. Grimes <

Post by Rodney W. Grimes
[ Charset UTF-8 unsupported, converting... ]

Post by Warner Losh
On Tue, Jul 3, 2018, 11:41 AM Rodney W. Grimes <

Post by Rodney W. Grimes
[ Charset UTF-8 unsupported, converting... ]

Post by Rodney W. Grimes
"As far as I can tell" needs to be a bit stronger,
Agreed. But before doing work we should first make sure that the

work

Post by Rodney W. Grimes
will be productive.

Xstr and most are best thought of as a final pass of the old pcc

compiler.

Post by Warner Losh
This functionality moved into gcc before the egcs split back in the

90s.

Post by Warner Losh
The compiler has done this work for two decades now. Nobody uses

it, or

Post by Warner Losh
needs it today.
Unlike tip or cu, nobody at all uses these.

Agreed, these tools are quite obscure and I don't think _FreeBSD_ has

ever

Post by John Baldwin
used them.

I thought someone already pointed out places in the src tree
that had references to them, though they may

You removed the SHAREDSTRINGS support in r3493 back in 1994. It's been
unused since then. There are comments that suggest it never worked in
FreeBSD, but that may be 2.0 and newer only.

Removed it, not commented it out. Someone else on this thread made
statement that stuff is refering to these tools, that could not been
what I removed in r3493.
I do see that some of what I removed in r3493 is now back as bmake
./contrib/bmake/mk/prog.mk:.if defined(SHAREDSTRINGS)
./contrib/bmake/mk/README:SHAREDSTRINGS If defined, a new .c.o rule is
used that results in shared
Those are parts that I totally removed from
share/mk/{bsd.README,bsd.prog.mk} in r3493.

Right, it hsn't been relevant since then, and hasn't worked in the build
system. the actual programs may or may not still work, since they were
never updated past the K&R era level C language. There are a number of
special cases I don't think they handle, including // comments. I doubt
they are still useful.

Post by Rodney W. Grimes
The only note I see about "never worked" is that prog.mk stuff
is suppose to have issues with Parallel make, thats in the
contrib/bmake/mk/README. Where do you have a reference to
never worked in FreeBSD?

This was reimported after the original stuff was removed. I should have
said unused rather than not working since it was 'not working with the
build'.

Warner

Rodney W. Grimes

2018-07-04 14:25:02 UTC

Post by Poul-Henning Kamp
--------

Source code is a liability, it is not an investment.

Source code is both a liability and an investment, sometimes even
a very large investment of time or money or both.

Post by Poul-Henning Kamp
If the code is useless and not part of a published API, then it should
be removed.

Useless is another subjective term, what you find useless I may find
very useful. API is also probably a poor use of that term here,
none of these have "API", not even your cut(1) or awk(1) below.

Post by Poul-Henning Kamp

Post by Bryan Drewery
This argument of "not useful" is a bit subjective. Is cut useful when we
have awk? I won't go on.

cut(1) & awk(1) are both part of the published API, so that comparison gets
you nowhere.

I'll note that xstr is documented in PSD/make, and has a man page,
making it have an API as much as cut(1) or awk(1).

But, as I said early, lets just shelf all this "lets remove this or
that cause of foo" until package base is done, and then a group of
people should create a documented critiria list of what should or
should not be in the base system, and do a tree wide sweep rather
than have 100+ threads over the next 5 years about this or that.

--
Rod Grimes ***@freebsd.org

Warner Losh

2018-07-04 14:27:44 UTC

On Wed, Jul 4, 2018 at 8:25 AM, Rodney W. Grimes <

Post by Rodney W. Grimes
But, as I said early, lets just shelf all this "lets remove this or
that cause of foo" until package base is done, and then a group of
people should create a documented critiria list of what should or
should not be in the base system, and do a tree wide sweep rather
than have 100+ threads over the next 5 years about this or that.

It does tie into the deprecation stuff too.

xstr, at least, needs major work to support modern C dialects. It's not
just old, it's also broken. A quick stroll through the code shows this. It
should go because it's not worth taking with us to pkg base.

Warner

Rodney W. Grimes

2018-07-04 14:49:21 UTC

Post by Warner Losh
On Wed, Jul 4, 2018 at 8:25 AM, Rodney W. Grimes <

It does tie into the deprecation stuff too.

Then shall this be tabled until that is finished please?

Post by Warner Losh
xstr, at least, needs major work to support modern C dialects. It's not
just old, it's also broken. A quick stroll through the code shows this. It
should go because it's not worth taking with us to pkg base.

So add "Old", "Broken" to the list of critera and lets move on?
Cost of taking this to pkg base should be 0, cost of this
conversation is already >>> 0.

--
Rod Grimes ***@freebsd.org

Ian Lepore

2018-07-04 15:30:44 UTC