Discussion:
dlclose() vs. atexit()
Mikhail Teterin
2003-02-04 15:46:13 UTC
Permalink
[Moved to -arch]

On Tuesday 04 February 2003 03:26 am, Alfred Perlstein wrote:
= * Wes Peters <***@softweyr.com> [030203 23:41] wrote:
= > On Mon, 2003-02-03 at 18:58, Mikhail Teterin wrote:
= > > There remains an unresolved issue with mplayer on FreeBSD -- some
= > > of the libraries it dlopens and dlcloses are calling atexit() in
= > > between with their own functions.
= > >
= > > This causes SEGFAULTs in exit(), which tries to call those
= > > functions. The application catches the signals and would not quit
= > > until SIGKILL-ed.
= > >
= > > This does not affect Linux, where, reportedly, calls to atexit()
= > > are treated differently if made from dlopened code. I'm not sure
= > > how best to fix this (Call _exit()? Remove signal handlers before
= > > exit()?), but something needs to be done...

= > I think ideally we'd want dlclose to be able to deinstall any
= > atexit handlers that were installed by library functions. The
= > most straight- forward path to this I can see is an atexit-remove
= > call that can be passed a start and end address and will remove
= > any function references found between the two. dlclose could call
= > this function with the start and end addresses of the library text
= > segment to remove any exit handlers in the library code space.

= > I can probably take a look at this later in the week if this seems
= > like a reasonable approach.

= Please see if you can emulate the glibc behaviour just to ease
= porting. I think that means you must actually call the atexit handler,
= not just deregister it.

Last time I brought this up, it was not clear, what The Right Thing to
do is. Are these details mandatated by some standard out there, or is
everyone on their own, and we should, indeed, do what Linux does, to not
increase the enthropy?

Should, for example, exit handlers, which will still be valid after the
dlclose() still be called at dlclose() or at the actual exit()? How
about atexit() calls made between dlopen() and dlclose(), and not by the
library but by the application, or by another library? Have each library
get its own list of exit-handlers?

Or should it still be a global list, but examined by both dlclose()
and exit() -- if an item on the list is about to become invalid after
dlclose() (how can we know, BTW?), have dlclose() call it -- otherwise,
leave it alone?

That is what Solaris is doing, apparently. From their atexit(3c)

The atexit() function registers the function pointed to by
func to be called without arguments on normal termination of
the program or when the object defining the function is
unloaded.
[...]
On process exit, functions are called in the reverse order
of their registration. On object unloading, any functions
belonging to an unloadable object are called in the reverse
order of their registration.

The Solaris' approach seems the most robust. The Linux' one fails,
when the exit handler belongs to another library. Attached are two
files, which illustrate this -- even if somewhat artificially.

. the main program dlopens two libraries (both built from
the same source);
. it then passes the library number (integer) -- for reporting,
and the address of the handler _found in the other library_
to each one;
. the libraries both register each other's functions as handlers;
. the application unloads both libraris.

On FreeBSD both libraries are quietly unloaded, and the application
SEGFAULTs in exit(). On Linux, the crash happens at the second unload,
which tries to call the handler from the first library. On Solaris
everything runs to completion, which -- unless it violates some standard --
makes it the most appealing implementation (hostnames edited out):

[...] 5.0-CURRENT FreeBSD 5.0-CURRENT #0: Wed Jan 8 19:15:03 EST 2003
[....]
Loading the libraries
./l0.so loaded as 0x28064200
./l1.so loaded as 0x28064300
Library 0 calling atexit(0x2813a6f0)
Library 1 calling atexit(0x281386f0)
Unloading the libraries
0x28064200 unloaded
0x28064300 unloaded
Libraries unloaded. Returning
Segmentation fault

Linux [...] 2.4.18-14 #1 Wed Sep 4 13:35:50 EDT 2002 i686 i686 i386 GNU/Linux
Loading the libraries
./l0.so loaded as 0x8049ad0
./l1.so loaded as 0x8049e00
Library 0 calling atexit(0x40016788)
Library 1 calling atexit(0x40014788)
Unloading the libraries
Exit handler 0x40016788 of library 1 is invoked
0x8049ad0 unloaded
Segmentation fault (core dumped)

SunOS [...] 5.8 Generic_108528-13 sun4u sparc SUNW,Sun-Fire-280R
Loading the libraries
./l0.so loaded as ff3a1458
./l1.so loaded as ff3a17b8
Library 0 calling atexit(ff260380)
Library 1 calling atexit(ff350380)
Unloading the libraries
Exit handler ff350380 of library 0 is invoked
ff3a1458 unloaded
Exit handler ff260380 of library 1 is invoked
ff3a17b8 unloaded
Libraries unloaded. Returning

Yet another plan would be to have the atexit() call simply increase the
ref-count of the library a handler is from, so it will not actually be
unloaded by dlclose(). But that will be yet another implementation...

-mi
Chip Norkus
2003-02-04 16:25:13 UTC
Permalink
On Tue Feb 04, 2003; 10:46AM -0500 Mikhail Teterin propagated the following:

[snip]
Post by Mikhail Teterin
Yet another plan would be to have the atexit() call simply increase the
ref-count of the library a handler is from, so it will not actually be
unloaded by dlclose(). But that will be yet another implementation...
I'm not sure this would be a good idea. I have a program which
uses dynamic libraries to introduce code, and which can unload or
reload them at any time and keep on running. I am especially worried
about the reload case, because what often happens is that an old
version of the .so is unloaded (dlclosed) and then a new version
is loaded (dlopened). If the old version stays around, I'm afraid
there would be some symbol collision. Correct me if I'm wrong,
though.

I think the Solaris approach you mentioned above is by far the best
and most correct sounding approach.
Post by Mikhail Teterin
-mi
-wd
--
chip norkus; unix geek and programmer; ***@arpa.com
"question = (to) ? be : !be;" --Shakespeare http://telekinesis.org/

To Unsubscribe: send mail to ***@FreeBSD.org
with "unsubscribe freebsd-arch" in the body of the message
Jan Grant
2003-02-04 16:42:41 UTC
Permalink
Post by Chip Norkus
[snip]
Post by Mikhail Teterin
Yet another plan would be to have the atexit() call simply increase the
ref-count of the library a handler is from, so it will not actually be
unloaded by dlclose(). But that will be yet another implementation...
I'm not sure this would be a good idea. I have a program which
uses dynamic libraries to introduce code, and which can unload or
reload them at any time and keep on running. I am especially worried
about the reload case, because what often happens is that an old
version of the .so is unloaded (dlclosed) and then a new version
is loaded (dlopened). If the old version stays around, I'm afraid
there would be some symbol collision. Correct me if I'm wrong,
though.
I think the Solaris approach you mentioned above is by far the best
and most correct sounding approach.
I agree; I've done similar work with dlopened libraries which were
"updated" over time, dynamically.

The atexit() / dlclose() behaviour seems robust and "does the right
thing". If you want more complex behaviour (eg, persisting library state
while you update a dynamic library) then you have to roll your own.
That's not a big deal.
--
jan grant, ILRT, University of Bristol. http://www.ilrt.bris.ac.uk/
Tel +44(0)117 9287088 Fax +44 (0)117 9287112 http://ioctl.org/jan/
There's no convincing English-language argument that this sentence is true.


To Unsubscribe: send mail to ***@FreeBSD.org
with "unsubscribe freebsd-arch" in the body of the message
Terry Lambert
2003-02-04 19:12:02 UTC
Permalink
Post by Mikhail Teterin
Last time I brought this up, it was not clear, what The Right Thing to
do is. Are these details mandatated by some standard out there, or is
everyone on their own, and we should, indeed, do what Linux does, to not
increase the enthropy?
Should, for example, exit handlers, which will still be valid after the
dlclose() still be called at dlclose() or at the actual exit()? How
about atexit() calls made between dlopen() and dlclose(), and not by the
library but by the application, or by another library? Have each library
get its own list of exit-handlers?
Or should it still be a global list, but examined by both dlclose()
and exit() -- if an item on the list is about to become invalid after
dlclose() (how can we know, BTW?), have dlclose() call it -- otherwise,
leave it alone?
FWIW, the way Windows handles this is to have various entry point
to manage this:

ProcessAttach Called after a process attaches to
a shared object
ProcessDetach Called before a process detaches a
shared object
ThreadAttach Called after a thread attaches to
a shared object
ThreadDetach Called before a thread detaches a
shared object

The key here is that all processes have at least one thread.

These are really the entry points, not the atexit(), that should
be used.

Right now, FreeBSD can at least do the Process[Attach|Detach] for
dlopen'ed objects, by abusing the .init and .fini sections, and/or
abusing the .ctor and .dtor sections.

Probably, this is the correct approach, overall, but requires some
small mod to crt*.c to make it entirely happy (declaring multiple
.init/.fini sections displaces, rather than aggregating them, as
only the first entry in the linker set is invoked, not the whole
list, and using the .ctor/.dtor approach requires using the C++
crt*.c in the C case).

I've actually posted code to abuse .init/.fini to this list in the
past, a part of creating a static libdlopen.

-- Terry

To Unsubscribe: send mail to ***@FreeBSD.org
with "unsubscribe freebsd-arch" in the body of the message
Max Khon
2003-02-04 19:42:29 UTC
Permalink
hi, there!
Post by Mikhail Teterin
Yet another plan would be to have the atexit() call simply increase the
ref-count of the library a handler is from, so it will not actually be
unloaded by dlclose(). But that will be yet another implementation...
this will break applications which want to reload some
of the dlopen'ed modules

/fjoe


To Unsubscribe: send mail to ***@FreeBSD.org
with "unsubscribe freebsd-arch" in the body of the message
Mikhail Teterin
2003-02-04 20:05:53 UTC
Permalink
On Tuesday 04 February 2003 02:42 pm, Max Khon wrote:
= hi, there!
=
= > Yet another plan would be to have the atexit() call simply increase the
= > ref-count of the library a handler is from, so it will not actually be
= > unloaded by dlclose(). But that will be yet another implementation...
=
= this will break applications which want to reload some
= of the dlopen'ed modules

I guess, I'm especially slow today. How will it break them? That is
what Solaris appears to be doing, BTW -- calling the exit-handler at
dlclose(), but leaving the library in memory:

SunOS [...] 5.8 Generic_108528-13 sun4u sparc SUNW,Sun-Fire-280R
Loading the libraries
./l0.so loaded as ff3a1458
./l1.so loaded as ff3a17b8
Library 0 calling atexit(ff260380)
Library 1 calling atexit(ff350380)
Unloading the libraries
Exit handler ff350380 of library 0 is invoked
ff3a1458 unloaded
Exit handler ff260380 of library 1 is invoked
ff3a17b8 unloaded
Libraries unloaded. Returning

See, the exit handler for library 1 is called and does not crash, even
though it is defined in the already dlclosed library 0...

-mi



To Unsubscribe: send mail to ***@FreeBSD.org
with "unsubscribe freebsd-arch" in the body of the message
Max Khon
2003-02-04 20:44:34 UTC
Permalink
hi, there!
Post by Mikhail Teterin
= > Yet another plan would be to have the atexit() call simply increase the
= > ref-count of the library a handler is from, so it will not actually be
= > unloaded by dlclose(). But that will be yet another implementation...
=
= this will break applications which want to reload some
= of the dlopen'ed modules
I guess, I'm especially slow today. How will it break them? That is
what Solaris appears to be doing, BTW -- calling the exit-handler at
SunOS [...] 5.8 Generic_108528-13 sun4u sparc SUNW,Sun-Fire-280R
Loading the libraries
./l0.so loaded as ff3a1458
./l1.so loaded as ff3a17b8
Library 0 calling atexit(ff260380)
Library 1 calling atexit(ff350380)
Unloading the libraries
Exit handler ff350380 of library 0 is invoked
ff3a1458 unloaded
Exit handler ff260380 of library 1 is invoked
ff3a17b8 unloaded
Libraries unloaded. Returning
See, the exit handler for library 1 is called and does not crash, even
though it is defined in the already dlclosed library 0...
ah. if refcount will be decremented and checked for 0 (with possible dlclose
call) after each call to exit function then there is no problems.

/fjoe


To Unsubscribe: send mail to ***@FreeBSD.org
with "unsubscribe freebsd-arch" in the body of the message
Thomas E. Zander
2003-02-05 13:18:32 UTC
Permalink
Well,

Am Tue, dem 04. Feb 2003, um 10:46 -0500 Uhr schrubte Mikhail Teterin
Post by Mikhail Teterin
Linux [...] 2.4.18-14 #1 Wed Sep 4 13:35:50 EDT 2002 i686 i686 i386 GNU/Linux
Loading the libraries
./l0.so loaded as 0x8049ad0
./l1.so loaded as 0x8049e00
Library 0 calling atexit(0x40016788)
Library 1 calling atexit(0x40014788)
Unloading the libraries
Exit handler 0x40016788 of library 1 is invoked
0x8049ad0 unloaded
Segmentation fault (core dumped)
*this* is really surprising me.
I mean, most of the mplayer developers work under linux.
And nobody of them takes care about this for...yea...years now?!

It seems that this won't be a 5-minute patch to get it to work really
cleanly...

Riggs
--
- Die Welt schläft tief schon lange Zeit | Sent with RiggiSmooth [tm] -
-- Mich nur flieht die Dunkelheit | ------------------------- --
--- Denn per Infrarot seh ich | just to fit your ---
---- Die Nacht ist wirklich widerlich. | primitive screen. ----
Thomas E. Zander
2003-02-05 13:21:42 UTC
Permalink
Oh, sorry...I've made a little mistake..

please do *not* use this
***@trillian.mugiri.au
email address, since it is localnetwork-not-reachable-from-outer-world
stuff.

Thanks
Riggs
--
- Die Welt schläft tief schon lange Zeit | Sent with RiggiSmooth [tm] -
-- Mich nur flieht die Dunkelheit | ------------------------- --
--- Denn per Infrarot seh ich | just to fit your ---
---- Die Nacht ist wirklich widerlich. | primitive screen. ----
Loading...