Speeding Up Frequently Accessed System Calls With VDSO


While perusing the latest articles on hacker news I came across an amazing debugging article by Hector Martin. He recently tracked down a nasty segmentation violation and explained step-by-step how he did it. The article dicusses the VDSO (Virtual ELF Dynamic Shared Object) which is an efficient way to access frequently used system calls. Here is a nice description of this feature from the vdso(7) manual page:

Why does the vDSO exist at all? There are some system calls the kernel provides that user-space code ends up using frequently, to the point that such calls can dominate overall performance. This is due both to the frequency of the call as well as the context-switch overhead that results from exiting user space and entering the kernel.

If you run ldd on a program you can see that linux-vdso.so.1 is linked to the executable:

$ ldd /bin/zsh

	linux-vdso.so.1 (0x00007ffc509d6000)
	libgdbm.so.4 => /lib64/libgdbm.so.4 (0x00007fa6c5cab000)
	libpcre.so.1 => /lib64/libpcre.so.1 (0x00007fa6c5a38000)
	libdl.so.2 => /lib64/libdl.so.2 (0x00007fa6c5834000)
	libncursesw.so.6 => /lib64/libncursesw.so.6 (0x00007fa6c55fc000)
	libtinfo.so.6 => /lib64/libtinfo.so.6 (0x00007fa6c53d0000)
	librt.so.1 => /lib64/librt.so.1 (0x00007fa6c51c8000)
	libm.so.6 => /lib64/libm.so.6 (0x00007fa6c4eb2000)
	libc.so.6 => /lib64/libc.so.6 (0x00007fa6c4add000)
	libpthread.so.0 => /lib64/libpthread.so.0 (0x00007fa6c48be000)
	/lib64/ld-linux-x86-64.so.2 (0x00007fa6c6192000)

The VDSO is resident in the kernel and mapped to the process at runtime. If you check the memory map for a process you can see where VDSO is mapped into the address space:

$ grep vdso /proc/$$/maps

7ffc616ec000-7ffc616ee000 r-xp 00000000 00:00 0                          [vdso]

VDSO is extremely small and takes a measly 8K of memory on x86/X64 architectures:

$ echo $((0x7ffc616ee000-0x7ffc616ec000))

8192

$ grep -A 1 vdso /proc/$$/smaps

7ffc616ec000-7ffc616ee000 r-xp 00000000 00:00 0                          [vdso]
Size:                  8 kB

The vdso manual page lists 4 system calls that are implemented on x86/X64 architectures. We can verify this with nm (or by reading ARCHITECTURE-SPECIFIC NOTES in vdso(7)):

$ strings vdso64.so | grep '__vdso'

__vdso_clock_gettime
__vdso_gettimeofday
__vdso_time
__vdso_getcpu

Huge thanks to Hector Martin for the amazing article. I learned a lot by reading his article and digging through the VDSO documentation. Adding this here for reference.

UPDATED 12/06/2017

Hector responded to a twitter thanks and mentioned that the VDSO is resident in the kernel and mapped into the process address space when the process is initialized. The object code doesn’t come from disk like I initially thought.

This article was posted by Matty on 2017-12-05 18:34:13 -0500 -0500