John Baldwin
2015-08-04 17:56:09 UTC
Many debuggers (recent gdb and lldb) support cross-architecture debugging
just fine. My current WIP port of kgdb to gdb7 supports cross-debugging for
remote targets already, but I wanted it to also support cross-debugging for
vmcores.
The existing libkvm/kgdb code in the tree has some limited support for
cross-debugging. It requires building a custom libkvm (e.g. libkvm-i386.a)
and custom kgdb for each target platform. However, gdb (and lldb) both
support multiple targets in a single binary, so I'd like to have a single
kgdb binary that can cross-debug anything.
I started hacking on libkvm last weekend and have a prototype that I've used
(along with some patches to my kgdb port) to debug an amd64 vmcore on an
i386 machine and vice versa.
To do this I've made some additions to the libkvm API:
1) A new 'kvaddr_t' type represents a kernel virtual address. This is
similar to the psaddr_t type used for MI process addresses in userland
debugging. I almost reused psaddr_t directly, but that would have made
<libkvm.h> depend on <sys/procfs.h>. Instead, I opted for a separate
type. It is currently a uint64_t.
2) A new 'struct kvm_nlist'. This is a stripped-down version of
'struct nlist' that uses kvadd_t for n_value instead of an unsigned
long.
3) kvm_native() returns true if an open kvm descriptor is for a native
kernel and memory image.
4) kvm_nlist2() is like kvm_nlist() but it uses 'struct kvm_nlist'
instead of 'struct nlist'. Internally symbol names are always
resolved to kvaddr_t addresses rather than u_long addresses.
Native kernels still use _fdnlist() from libc to resolve symbols.
Cross kernels use a caller supplied function to resolve symbols
(the older cross code for libkvm required the caller to provide
a global ps_pglobal_lookup symbol typically provided for
<proc_service.h>).
5) kvm_open2() is like kvm_openfiles() except that it drops the unused
'swapfile' argument and adds a new function pointer argument to a
symbol resolving function. The function pointer can be NULL in
which case only native kernels can be opened. Kernels used with
/dev/mem or /dev/kmem must be native.
6) kvm_read2() is like kvm_read() except that it uses kvaddr_t
instead of unsigned long for the kernel virtual address.
Adding new symbols (specifically kvm_nlist2 and kvm_read2) preserves
ABI and API compatibility. Note that most libkvm functions such as
kvm_getprocs(), etc. only work with native kernels. I have not yet
done a full sweep to force them to fail for non-native kernels.
Also, the vnet and dpcpu stuff only works for native kernels currently
though that can be fixed at some point in the future.
For the MD backends, I've added a new kvm_arch switch:
struct kvm_arch {
int (*ka_probe)(kvm_t *);
int (*ka_initvtop)(kvm_t *);
void (*ka_freevtop)(kvm_t *);
int (*ka_kvatop)(kvm_t *, kvaddr_t, off_t *);
int (*ka_uvatop)(kvm_t *, const struct proc *, kvaddr_t, off_t *);
int ka_native;
};
Each backend implements the necessary callbacks (uvatop is optional)
and is added to a global linker set that kvm_open2() walks to find the
appropriate kvm_arch for a given kernel + vmcore. On x86 I've used
separate kvm_arch structures for "plain" vs minidumps.
The backends now have to avoid using native headers. For ELF handling
this means using libelf instead of <machine/elf.h> and raw mmap(). For
the x86 backends it meant defining some duplicate constants for certain
page table fields since <machine/pmap.h> can't be relied on (e.g.
I386_PG_V instead of PG_V). I added static assertions in the "native"
case (e.g. building kvm_i386.c on i386) to ensure the duplicate constants
match the originals.
You can see the current WIP patches here:
https://github.com/freebsd/freebsd/compare/master...bsdjhb:kgdb_enhancements
What I'm mostly after is comments on the API, etc. Once that is settled I
will move forward on converting and/or stubbing the other backends (the
stub route would be to only support other backends on native systems for
now).
Oh, and I do hope to have a 'KGDB' option for the devel/gdb port in the
near future.
just fine. My current WIP port of kgdb to gdb7 supports cross-debugging for
remote targets already, but I wanted it to also support cross-debugging for
vmcores.
The existing libkvm/kgdb code in the tree has some limited support for
cross-debugging. It requires building a custom libkvm (e.g. libkvm-i386.a)
and custom kgdb for each target platform. However, gdb (and lldb) both
support multiple targets in a single binary, so I'd like to have a single
kgdb binary that can cross-debug anything.
I started hacking on libkvm last weekend and have a prototype that I've used
(along with some patches to my kgdb port) to debug an amd64 vmcore on an
i386 machine and vice versa.
To do this I've made some additions to the libkvm API:
1) A new 'kvaddr_t' type represents a kernel virtual address. This is
similar to the psaddr_t type used for MI process addresses in userland
debugging. I almost reused psaddr_t directly, but that would have made
<libkvm.h> depend on <sys/procfs.h>. Instead, I opted for a separate
type. It is currently a uint64_t.
2) A new 'struct kvm_nlist'. This is a stripped-down version of
'struct nlist' that uses kvadd_t for n_value instead of an unsigned
long.
3) kvm_native() returns true if an open kvm descriptor is for a native
kernel and memory image.
4) kvm_nlist2() is like kvm_nlist() but it uses 'struct kvm_nlist'
instead of 'struct nlist'. Internally symbol names are always
resolved to kvaddr_t addresses rather than u_long addresses.
Native kernels still use _fdnlist() from libc to resolve symbols.
Cross kernels use a caller supplied function to resolve symbols
(the older cross code for libkvm required the caller to provide
a global ps_pglobal_lookup symbol typically provided for
<proc_service.h>).
5) kvm_open2() is like kvm_openfiles() except that it drops the unused
'swapfile' argument and adds a new function pointer argument to a
symbol resolving function. The function pointer can be NULL in
which case only native kernels can be opened. Kernels used with
/dev/mem or /dev/kmem must be native.
6) kvm_read2() is like kvm_read() except that it uses kvaddr_t
instead of unsigned long for the kernel virtual address.
Adding new symbols (specifically kvm_nlist2 and kvm_read2) preserves
ABI and API compatibility. Note that most libkvm functions such as
kvm_getprocs(), etc. only work with native kernels. I have not yet
done a full sweep to force them to fail for non-native kernels.
Also, the vnet and dpcpu stuff only works for native kernels currently
though that can be fixed at some point in the future.
For the MD backends, I've added a new kvm_arch switch:
struct kvm_arch {
int (*ka_probe)(kvm_t *);
int (*ka_initvtop)(kvm_t *);
void (*ka_freevtop)(kvm_t *);
int (*ka_kvatop)(kvm_t *, kvaddr_t, off_t *);
int (*ka_uvatop)(kvm_t *, const struct proc *, kvaddr_t, off_t *);
int ka_native;
};
Each backend implements the necessary callbacks (uvatop is optional)
and is added to a global linker set that kvm_open2() walks to find the
appropriate kvm_arch for a given kernel + vmcore. On x86 I've used
separate kvm_arch structures for "plain" vs minidumps.
The backends now have to avoid using native headers. For ELF handling
this means using libelf instead of <machine/elf.h> and raw mmap(). For
the x86 backends it meant defining some duplicate constants for certain
page table fields since <machine/pmap.h> can't be relied on (e.g.
I386_PG_V instead of PG_V). I added static assertions in the "native"
case (e.g. building kvm_i386.c on i386) to ensure the duplicate constants
match the originals.
You can see the current WIP patches here:
https://github.com/freebsd/freebsd/compare/master...bsdjhb:kgdb_enhancements
What I'm mostly after is comments on the API, etc. Once that is settled I
will move forward on converting and/or stubbing the other backends (the
stub route would be to only support other backends on native systems for
now).
Oh, and I do hope to have a 'KGDB' option for the devel/gdb port in the
near future.
--
John Baldwin
John Baldwin