Casper new architecture.

[Mariusz Zaborski]

Hello,

I have finally a new version of Casper and I wanted to share with you. [1]
I would like to ask for code review and if possible for some people to
run tests on thier machines.

We decided to change a few things in the Casper architecture:

The first one is that Capser isn't a daemon any more.
Now, after calling the cap_init(3) function Casper will fork from
it's original process, using pdfork(2). Thanks to the changes in r286698
the pdforking will not have any affects to the original process.
Forking from a process has a lot of advantages:
1* We have the same cwd as the original process (so the programmer must be
aware that if he changed the original process working directory, Casper
directory will not be changed). But I feel that this is acceptable limitation.
2* The same uid, gid and groups.
3* The same MAC labels.
4* The same descriptor table. This is important for a filesystem service
because with the old Casper for example, process substitution was not
possible. When the filesystem service arrives, I want to add some special
flags that will tell zygote processes to not clear file descriptors. Right
know for the service, all fd are closed.
5* The same routing table. We can change routing table for process using
setfib(2).
6* The same umask.
7* The same cpuset(1).
And I probably missed some things on this list.

I decided to remove the libcapsicum library. In my opinion Capser is
connected with capsicum, but Casper can be used also with different sandbox
techniques. Now I would like to refer to it just as libcasper.

Second change is that Casper will not execute any binary files.
Now every services (cap_{dns,grp,etc.}) will be in form of shared library.
I don't see, right now, any advantages on keeping services as executable
binaries. It's a little bit problematic to manage services when we don't have a
global daemon. Services can be in different locations and hard coding one path
(like before /etc/casperd) didn't feel right. On the other hand, adding additional
arguments to cap_init() also don't convince me, because how can the programmer
guess where the administrator will put the Casper services. So in my opinion using
dynamic libraries right know is the best option. Programs need to know the replacement
API (for example cap_gethostbyname which replace gethostbyname) so it needs to
include some library (before just one global library, libcapsicum), so why not
store services inside that library? Thanks to that we also have an implementation of
service and replaced API in one place, so we don't need to jump between libexec
and lib directories.

I hope that you like new architecture of Casper.

Cheers,
Mariusz

[1] https://people.freebsd.org/~oshogbo/casper.patch

[Mariusz Zaborski]

I cc'ed arch@ one more time. In first email I was using to and Jon replay went
only to the capsicum group.

Post by Mariusz Zaborski
Hello,
I have finally a new version of Casper and I wanted to share with you. [1]
I would like to ask for code review and if possible for some people to
run tests on thier machines.

Hi Mariusz,
This sounds rather exciting, and I will be pleased to do some reading and testing. However, possibly not in great quantities until the term ends on 8 December.
As an initial meta-comment, might you be able to upload the patch to reviews.freebsd.org? Phabricator really does make it easier to read and comment on long, complex patches like this one (I think we’re looking at 22.5 klines?).

Done: https://reviews.freebsd.org/D4277 .
Thanks for advice.

Post by Mariusz Zaborski
The first one is that Capser isn't a daemon any more.
Now, after calling the cap_init(3) function Casper will fork from
it's original process, using pdfork(2). Thanks to the changes in r286698
the pdforking will not have any affects to the original process.
1* We have the same cwd as the original process (so the programmer must be
aware that if he changed the original process working directory, Casper
directory will not be changed). But I feel that this is acceptable limitation.
2* The same uid, gid and groups.
3* The same MAC labels.
4* The same descriptor table. This is important for a filesystem service
because with the old Casper for example, process substitution was not
possible. When the filesystem service arrives, I want to add some special
flags that will tell zygote processes to not clear file descriptors. Right
know for the service, all fd are closed.
5* The same routing table. We can change routing table for process using
setfib(2).
6* The same umask.
7* The same cpuset(1).
And I probably missed some things on this list.

Without reading or running the code yet, I suspect that this is a good tack to take when pursuing application-level compartmentalization. Of course, the new question becomes, can some Casper instance still service multiple applications in, e.g., a login session? Can we interpose on messages sent from an application to its libcasper to the desktop Casper to the system Casper (if there is such a thing any more)? I think this was the “Caspers all the way down” discussion in Cambridge a couple of years ago. :)

There isn't any instance of a global Casper.
I remember that discussion. You suggested to mix both approach have one Casper
in library and second one as global. For now I don't see any advantages of
having them both. Not sure if I understand your example but you still can share
Casper process or one services. You can clone and send the Casper channel to
other program, but for some services you need to remember that some things will
be inhered from original process (list above).

Post by Mariusz Zaborski
I decided to remove the libcapsicum library. In my opinion Capser is
connected with capsicum, but Casper can be used also with different sandbox
techniques. Now I would like to refer to it just as libcasper.

I think this sounds very sensible.

Post by Mariusz Zaborski
Second change is that Casper will not execute any binary files.
Now every services (cap_{dns,grp,etc.}) will be in form of shared library.
I don't see, right now, any advantages on keeping services as executable
binaries. It's a little bit problematic to manage services when we don't have a
global daemon. Services can be in different locations and hard coding one path
(like before /etc/casperd) didn't feel right. On the other hand, adding additional
arguments to cap_init() also don't convince me, because how can the programmer
guess where the administrator will put the Casper services. So in my opinion using
dynamic libraries right know is the best option. Programs need to know the replacement
API (for example cap_gethostbyname which replace gethostbyname) so it needs to
include some library (before just one global library, libcapsicum), so why not
store services inside that library?

Yes, libraries do seem to be the natural place to land when Casper pdforks from the application rather than starting as a system daemon. One question would be whether it’s possible to make the Casper libraries wrap their native counterparts such that we can replace symbols for transparent use (e.g., keep calling gethostbyname rather than cap_gethostbyname).

I'm not sure if this is possible as you presented.
I think we would had a symbols collisions with libc.
So when I was implementing a fileargs library
(//depot/user/oshogbo/capsicum_rights2/lib/libfileargs/...)
I done funny trick. Basically we would always have lib_dns.so
but depending on compilation we would use or not use casper in it:

int
cap_gethostbyname(...)
{

#ifdef HAVE_LIBCAPSICUM
return (casper_gethostbyname(...));
#else
return (gethostbyname(...));
#endif
}

So our application always would use cap_gethostbyname and #ifdef would be moved
to services. It would make application much simpler.
I would love to discuss this approach at some point.

Post by Mariusz Zaborski
Thanks to that we also have an implementation of
service and replaced API in one place, so we don't need to jump between libexec
and lib directories.
I hope that you like new architecture of Casper.

Me too! :)
I wonder if you might be able to provide a little more discussion at the level of how the IPC works, etc.?

So what exactly interest you?

I will tell you in general how it works now.
Application starts.
Service first register it self in libcasper thanks to library constructor
(which is hidden in CREATE_SERVICE macro).
Then we run cap_init() in application. Our process forks.
The zygote is created and libcasper waits for commands.

Then you open some service (service.dns for example). To this you use
cap_service_open() function which will send nvlist whit the name of service to
Casper process. Casper process search for right functions (command function and
limit function) and pass it to zygote process. Zygote transform to be a new
service. Casper send to the original process descriptor to talk with a new
service. All IPC here is done using nvlist.

After that you can create next service or close connection to Casper.
Next you can limit your service using cap_limit_set. Its operates on nvlist.
Its depending on services what you should put to this nvlist.

After limiting you are using cap_*() functions which are sending commands to the
service.

Thanks for interest, Jon.

Cheers,
Mariusz