upstream for contrib/tzcode/stdtme?

Post by Eitan Adler
What's the upstream for contrib/tzcode/stdtme?
There are a couple of PRs for it and I'd like to either update, fix
upstream, or move to non-contrib & fix.
--
Eitan Adler

Hi,

I think it's here

https://www.iana.org/time-zones

or here (depending on how much bleeding edge you want to go):

https://github.com/eggert/tz

I recall we have old versions of the same files in libc too, plus some
modifications we got from Apple and Illumos.

NetBSD also has some updates of their own.

Pedro..

Warner Losh

2018-06-11 02:52:21 UTC

Post by Pedro Giffuni

Post by Eitan Adler
What's the upstream for contrib/tzcode/stdtme?
There are a couple of PRs for it and I'd like to either update, fix
upstream, or move to non-contrib & fix.
--
Eitan Adler

Hi,
I think it's here
https://www.iana.org/time-zones
https://github.com/eggert/tz
I recall we have old versions of the same files in libc too, plus some
modifications we got from Apple and Illumos.
NetBSD also has some updates of their own.

Plus some fixes of our own...

Warner

Pedro..

Post by Pedro Giffuni
_______________________________________________
https://lists.freebsd.org/mailman/listinfo/freebsd-arch

Mehmet Erol Sanliturk

2018-06-11 08:03:43 UTC

Post by Eitan Adler
What's the upstream for contrib/tzcode/stdtme?

Post by Eitan Adler
There are a couple of PRs for it and I'd like to either update, fix
upstream, or move to non-contrib & fix.
--
Eitan Adler

In page

https://data.iana.org/time-zones/tz-link.html
( Sources for time zone and daylight saving time data )

the following is written :

"
Changes to the tz database
The tz code and data are by no means authoritative.
If you find errors, please send changes to ***@iana.org, the time zone
mailing list.
You can also subscribe to it and browse the archive of old messages.

"

Mehmet Erol Sanliturk

Philip Paeps

2018-06-15 14:28:50 UTC

Post by Eitan Adler
What's the upstream for contrib/tzcode/stdtme?

https://www.iana.org/time-zones (and https://github.com/eggert/tz).

Post by Eitan Adler
There are a couple of PRs for it and I'd like to either update, fix
upstream, or move to non-contrib & fix.

I've started updating it several times but the rather awkward layout of
the vendor tree makes doing just about anything else more attractive. :)

More than happy to review an update if you actually manage to complete
one!

I wonder if we should just make the "moving around files" in the vendor
tree go away...

Many thanks!
Philip

--
Philip Paeps
Senior Reality Engineer
Ministry of Information

John Baldwin

2018-06-15 16:03:26 UTC

Post by Eitan Adler
What's the upstream for contrib/tzcode/stdtme?

https://www.iana.org/time-zones (and https://github.com/eggert/tz).

Post by Eitan Adler
There are a couple of PRs for it and I'd like to either update, fix
upstream, or move to non-contrib & fix.

I've started updating it several times but the rather awkward layout of
the vendor tree makes doing just about anything else more attractive. :)
More than happy to review an update if you actually manage to complete
one!
I wonder if we should just make the "moving around files" in the vendor
tree go away...

I do think we should perhaps import the vendor tree to contrib/stdtime or
some such and then apply our local patches from libc to there. Alternatively,
we could move the the files from libc into suitable locations in contrib/stdtime
and then do an 'svn merge --record-only' to bootstrap the vendor mergeinfo
for the current version and use that as a basis for updating?

--
John Baldwin

Eitan Adler

2018-06-15 17:46:02 UTC

Post by Eitan Adler
What's the upstream for contrib/tzcode/stdtme?

https://www.iana.org/time-zones (and https://github.com/eggert/tz).

Post by Eitan Adler
There are a couple of PRs for it and I'd like to either update, fix
upstream, or move to non-contrib & fix.

I'll attempt one this weekend. :)

Post by Philip Paeps
I wonder if we should just make the "moving around files" in the vendor
tree go away...

Post by John Baldwin
I do think we should perhaps import the vendor tree to contrib/stdtime or
some such and then apply our local patches from libc to there.

I'd like to do the following:

- identify what local patches we have and divide them into two parts:
(a) issues already fixed upstream (b) issues not already fixed [or
local changes]
- import the latest stdtime into vendor
- copy from vendor into our contrib/stdtime without flattening the structure
- reapply all (b) patches manually

This will make the mergeinfo correct and also make it clear what
patches are local and what are upstream

Post by John Baldwin
Alternatively,
we could move the the files from libc into suitable locations in contrib/stdtime
and then do an 'svn merge --record-only' to bootstrap the vendor mergeinfo
for the current version and use that as a basis for updating?

Exactly.

--
Eitan Adler

John Baldwin

2018-06-15 18:31:46 UTC

Post by Eitan Adler
What's the upstream for contrib/tzcode/stdtme?

https://www.iana.org/time-zones (and https://github.com/eggert/tz).

Post by Eitan Adler
There are a couple of PRs for it and I'd like to either update, fix
upstream, or move to non-contrib & fix.

I'll attempt one this weekend. :)

Post by Philip Paeps
I wonder if we should just make the "moving around files" in the vendor
tree go away...

Post by John Baldwin
I do think we should perhaps import the vendor tree to contrib/stdtime or
some such and then apply our local patches from libc to there.

(a) issues already fixed upstream (b) issues not already fixed [or
local changes]
- import the latest stdtime into vendor
- copy from vendor into our contrib/stdtime without flattening the structure
- reapply all (b) patches manually
This will make the mergeinfo correct and also make it clear what
patches are local and what are upstream

Exactly.

I think this second approach is actually a better plan and is not quite what you
were suggesting.

That is, import the vendor bits corresponding to our existing code first into
the vendor area, then svn cp our existing files from libc, zic, etc. into the
contrib/stdtime tree in the correct layout and do a single 'svn merge --record-only'
merge to initialize the vendor mergeinfo. At that point you should be able to
svn diff contrib/stdtime against the vendor/stdtime/dist to determine what local
changes we have and start to think about them. It also means that svn merge will
consider them during your merge to the newer vendor sources without you have to
manually apply patches in (b) while also preserving the commit history of those
patches.

--
John Baldwin

Eitan Adler

2018-06-15 23:09:16 UTC

Post by John Baldwin
I think this second approach is actually a better plan and is not quite what you
were suggesting.
That is, import the vendor bits corresponding to our existing code first into
the vendor area,
then svn cp our existing files from libc, zic, etc. into the
contrib/stdtime tree in the correct layout and do a single 'svn merge --record-only'
merge to initialize the vendor mergeinfo.
At that point you should be able to
svn diff contrib/stdtime against the vendor/stdtime/dist to determine what local
changes we have and start to think about them.

At this point both vendor and contrib have the same files, same
layout, but possibly different content?

How do we do the update itself then? "svn merge" from the vendor area
into contrib?

--
Eitan Adler

John Baldwin

2018-06-18 17:08:28 UTC

At this point both vendor and contrib have the same files, same
layout, but possibly different content?

Yes. The different content would be our local changes.

Post by Eitan Adler
How do we do the update itself then? "svn merge" from the vendor area
into contrib?

Yes, just like a normal vendor update.

--
John Baldwin

Eitan Adler

2018-08-12 12:23:23 UTC

Coming back to this. In r337683 and r337684 I imported a recent copy
of of TZDB. I was unable to find a copy of the 2010n distribution in
the original layout.

The next step, if I understand, is to do the following:
cd /srv/srv/freebsd/svn/head/contrib
mkdir tzdb
cd tzdb
svn cp ../tzcode/stdtime/* .
svn cp ../tzcode/zic/* .
svn cp ../tzdata/* .
svn ci
(the above ignores duplicated files, but that's just expanding the
wildcards appropriately).

After that:
svn merge --record-only '^/vendor/tzdb' .

At this point we'll be able to diff contrib/tzdb and vendor/tzdb to
show the most current vendor code compared our, modified old code.

Is this correct? Is this the optimal plan?

--
Eitan Adler

Philip Paeps

2018-08-12 12:36:23 UTC

Post by John Baldwin
I think this second approach is actually a better plan and is not quite what you
were suggesting.
That is, import the vendor bits corresponding to our existing code first into
the vendor area,
then svn cp our existing files from libc, zic, etc. into the
contrib/stdtime tree in the correct layout and do a single 'svn merge --record-only'
merge to initialize the vendor mergeinfo.
At that point you should be able to
svn diff contrib/stdtime against the vendor/stdtime/dist to
determine what local
changes we have and start to think about them.

Coming back to this. In r337683 and r337684 I imported a recent copy
of of TZDB. I was unable to find a copy of the 2010n distribution in
the original layout.

That's because we split up the tzcode in our vendor area (for reasons
I've never understood). As far as I know, there is no 1:1 mapping from
any tzcode distribution to what we have in our repository.

Post by Eitan Adler
cd /srv/srv/freebsd/svn/head/contrib
mkdir tzdb
cd tzdb
svn cp ../tzcode/stdtime/* .
svn cp ../tzcode/zic/* .
svn cp ../tzdata/* .
svn ci
(the above ignores duplicated files, but that's just expanding the
wildcards appropriately).
svn merge --record-only '^/vendor/tzdb' .
At this point we'll be able to diff contrib/tzdb and vendor/tzdb to
show the most current vendor code compared our, modified old code.
Is this correct? Is this the optimal plan?

I would really (really!) want to keep tzdata separate from tzcode and
not make it any more difficult to update tzdata (which happens
distressingly often).

tzdata imports/updates are currently not a problem. And it's nice to be
able to simply issue an errata notice only updating the zoneinfo files
without touching the tzcode.

The IANA tzdata project also distributes tzdata and tzcode separately.
If you wanted to update tzcode, you could do it independently of tzdata.

Philip

--
Philip Paeps
Senior Reality Engineer
Ministry of Information

Philip Paeps

2018-08-12 13:00:26 UTC

Post by John Baldwin
I think this second approach is actually a better plan and is not quite what you
were suggesting.
That is, import the vendor bits corresponding to our existing code first into
the vendor area,
then svn cp our existing files from libc, zic, etc. into the
contrib/stdtime tree in the correct layout and do a single 'svn
merge --record-only'
merge to initialize the vendor mergeinfo.
At that point you should be able to
svn diff contrib/stdtime against the vendor/stdtime/dist to determine what local
changes we have and start to think about them.

Coming back to this. In r337683 and r337684 I imported a recent copy
of of TZDB. I was unable to find a copy of the 2010n distribution in
the original layout.

That's because we split up the tzcode in our vendor area (for reasons
I've never understood). As far as I know, there is no 1:1 mapping
from any tzcode distribution to what we have in our repository.

I would really (really!) want to keep tzdata separate from tzcode and
not make it any more difficult to update tzdata (which happens
distressingly often).
tzdata imports/updates are currently not a problem. And it's nice to
be able to simply issue an errata notice only updating the zoneinfo
files without touching the tzcode.
The IANA tzdata project also distributes tzdata and tzcode separately.
If you wanted to update tzcode, you could do it independently of tzdata.

I just noticed you've already committed this import.

In case you're not actively following the tz mailing list, you may have
missed that every release of tzdb comes with lots of minor but annoying
code changes in addition to the data changes we actually want to
distribute as quickly as possible.

Currently, updating the tzdata is a fairly mechanical process of
updating vendor/tzdata and doing a couple of svn copies to contrib. If
we started tracking the combined tzdb rather than tzcode and tzdata
separately, issuing errata notices for timezone updates becomes
practically impossible.

Updating the zoneinfo files in an errata patch is non-controversial,
certainly compared to often seemingly arbitrary changes to the subtle
code that manipulates the representation of time. It makes no sense to
import these together or attempt to maintain them as one vendor tree.
That is why the tzcode and tzdata tarballs exist separately in addition
to the combined tzdb tarball.

As requested on the src-commits list: please import tzcode separately
and leave tzdata alone.

Thank you.

Philip

--
Philip Paeps
Senior Reality Engineer
Ministry of Information

Eitan Adler

2018-08-12 22:09:00 UTC

Post by John Baldwin
I think this second approach is actually a better plan and is not
quite what you
were suggesting.
That is, import the vendor bits corresponding to our existing code first into
the vendor area,
then svn cp our existing files from libc, zic, etc. into the
contrib/stdtime tree in the correct layout and do a single 'svn
merge --record-only'
merge to initialize the vendor mergeinfo.
At that point you should be able to
svn diff contrib/stdtime against the vendor/stdtime/dist to
determine what local
changes we have and start to think about them.

Coming back to this. In r337683 and r337684 I imported a recent copy
of of TZDB. I was unable to find a copy of the 2010n distribution in
the original layout.

That's because we split up the tzcode in our vendor area (for reasons
I've never understood). As far as I know, there is no 1:1 mapping
from any tzcode distribution to what we have in our repository.

...

please import tzcode separately and leave tzdata alone.

Done in r337693 and r337694.

The next step, if I understand, is to do the following:
cd /srv/srv/freebsd/svn/head/contrib
mkdir tzdb
cd tzdb
svn cp ../tzcode/stdtime/* .
svn cp ../tzcode/zic/* .
svn ci
(the above ignores duplicated files, but that's just expanding the
wildcards appropriately).

After that:
svn merge --record-only '^/vendor/tzdb' .

At this point we'll be able to diff contrib/tzdb and vendor/tzdb to
show the most current vendor code compared our, modified old code.

Is this correct? Is this the optimal plan?

--
Eitan Adler

Philip Paeps

2018-08-12 23:32:52 UTC

Post by John Baldwin
I think this second approach is actually a better plan and is not
quite what you
were suggesting.
That is, import the vendor bits corresponding to our existing code first into
the vendor area,
then svn cp our existing files from libc, zic, etc. into the
contrib/stdtime tree in the correct layout and do a single 'svn
merge --record-only'
merge to initialize the vendor mergeinfo.
At that point you should be able to
svn diff contrib/stdtime against the vendor/stdtime/dist to
determine what local
changes we have and start to think about them.

Coming back to this. In r337683 and r337684 I imported a recent copy
of of TZDB. I was unable to find a copy of the 2010n distribution in
the original layout.

That's because we split up the tzcode in our vendor area (for reasons
I've never understood). As far as I know, there is no 1:1 mapping
from any tzcode distribution to what we have in our repository.

...

please import tzcode separately and leave tzdata alone.

Done in r337693 and r337694.

Thank you!

For the next tzdata import, I'll consider moving vendor/tzdata to
vendor/tzdb/tzdata before doing the copies to contrib/.

Post by Eitan Adler
cd /srv/srv/freebsd/svn/head/contrib
mkdir tzdb
cd tzdb
svn cp ../tzcode/stdtime/* .
svn cp ../tzcode/zic/* .
svn ci
(the above ignores duplicated files, but that's just expanding the
wildcards appropriately).
svn merge --record-only '^/vendor/tzdb' .
At this point we'll be able to diff contrib/tzdb and vendor/tzdb to
show the most current vendor code compared our, modified old code.
Is this correct? Is this the optimal plan?

I would keep contrib/tzcode (and contrib/tzdata) rather than moving to
contrib/tzdb. Having them together under vendor/ makes sense because
it's the same vendor, but under contrib, they're definitely separate.

Other than that, this looks like a good plan.

zic should be pretty straightforward to update. Some of the stdtime
changes might want close review.

Thank you for picking this up. I've made several false starts on this
and the merge hell in stdtime always turned me off. :)

Best wishes.
Philip

--
Philip Paeps
Senior Reality Engineer
Ministry of Information

Eitan Adler

2018-08-18 04:43:21 UTC

Post by John Baldwin
I think this second approach is actually a better plan and is not
quite what you
were suggesting.
That is, import the vendor bits corresponding to our existing code
first into
the vendor area,
then svn cp our existing files from libc, zic, etc. into the
contrib/stdtime tree in the correct layout and do a single 'svn
merge --record-only'
merge to initialize the vendor mergeinfo.
At that point you should be able to
svn diff contrib/stdtime against the vendor/stdtime/dist to
determine what local
changes we have and start to think about them.

Coming back to this. In r337683 and r337684 I imported a recent copy
of of TZDB. I was unable to find a copy of the 2010n distribution in
the original layout.

That's because we split up the tzcode in our vendor area (for reasons
I've never understood). As far as I know, there is no 1:1 mapping
from any tzcode distribution to what we have in our repository.

...

please import tzcode separately and leave tzdata alone.

Done in r337693 and r337694.

Thank you!
For the next tzdata import, I'll consider moving vendor/tzdata to
vendor/tzdb/tzdata before doing the copies to contrib/.

I wanted to use a separate directory to make it clearler what's clean
and what's not but lets go with the current directory.

cd /srv/srv/freebsd/svn/head/
cd contrib/tzcode
svn mv stdtime/* .
svn cp zic/* .
cd ../..
$EDITOR usr.sbin/zic/zic/Makefile
$EDITOR usr.sbin/zic/zdump/Makefile
$EDITOR lib/libc/stdtime/Makefile.inc
svn ci
svn merge --record-only '^/vendor/tzcode' .
svn ci

This will leave contrib with the old code and vendor with the new code
and mergeinfo claiming we're fully merged. Is that the state of
affairs that we want? I don't know subversion well enough to
formulate a better plan.

At the end of this, what I'd like to do is see what the diff is, merge
the latest vendor code, and ensure that any local patches we still
require remain applied.

--
Eitan Adler

John Baldwin

2018-08-21 09:13:36 UTC

Post by John Baldwin
I think this second approach is actually a better plan and is not
quite what you
were suggesting.
That is, import the vendor bits corresponding to our existing code
first into
the vendor area,
then svn cp our existing files from libc, zic, etc. into the
contrib/stdtime tree in the correct layout and do a single 'svn
merge --record-only'
merge to initialize the vendor mergeinfo.
At that point you should be able to
svn diff contrib/stdtime against the vendor/stdtime/dist to
determine what local
changes we have and start to think about them.

Coming back to this. In r337683 and r337684 I imported a recent copy
of of TZDB. I was unable to find a copy of the 2010n distribution in
the original layout.

That's because we split up the tzcode in our vendor area (for reasons
I've never understood). As far as I know, there is no 1:1 mapping
from any tzcode distribution to what we have in our repository.

...

please import tzcode separately and leave tzdata alone.

Done in r337693 and r337694.

Thank you!
For the next tzdata import, I'll consider moving vendor/tzdata to
vendor/tzdb/tzdata before doing the copies to contrib/.

I wanted to use a separate directory to make it clearler what's clean
and what's not but lets go with the current directory.
cd /srv/srv/freebsd/svn/head/
cd contrib/tzcode
svn mv stdtime/* .
svn cp zic/* .
cd ../..
$EDITOR usr.sbin/zic/zic/Makefile
$EDITOR usr.sbin/zic/zdump/Makefile
$EDITOR lib/libc/stdtime/Makefile.inc
svn ci
svn merge --record-only '^/vendor/tzcode' .
svn ci
This will leave contrib with the old code and vendor with the new code
and mergeinfo claiming we're fully merged. Is that the state of
affairs that we want? I don't know subversion well enough to
formulate a better plan.

Can't you import the "current" version of tzcode into the vendor area
(what we have now) and then diff against that once it is moved to see
what local diffs we have that we still need vs don't need, then separately
update the vendor area to the latest version and do a "normal" merge?

If we can't determine what version we currently have then you can look
at the manual diff as a fallback, but if you can find the matching
version and import that into vendor first that might be best.

--
John Baldwin

Eitan Adler

2018-08-27 05:18:18 UTC

Post by Eitan Adler
This will leave contrib with the old code and vendor with the new code
and mergeinfo claiming we're fully merged. Is that the state of
affairs that we want? I don't know subversion well enough to
formulate a better plan.

I looked into this before importing the latest version. As far as I
could, the last time that tzcode was MFVed was in

r200832 | edwin | 2009-12-22 11:17:10 +0000 (Tue, 22 Dec 2009) | 10 lines
MFV of tzdata2009t, r200831

There were some changes here:
r214411 | edwin | 2010-10-27 07:14:46 +0000 (Wed, 27 Oct 2010) | 32 lines
Sync code with tzcode2010m

tzcode was tagged at tzcode2010n but I don't see it imported.

When I compared tzcode2010m against what is presently in head I got a
large, although not incomprehensible, diff. That said, its similar in
size to the diff against the most recent version.

Much of the changes arise from the xlocale changes from 2011.

--
Eitan Adler

John Baldwin

2018-08-27 16:17:38 UTC