This past weekend I spent a good deal of time playing with riemann. Riemann is a powerful stream processor and I’m hoping to use it to correlated and analyze metrics from disparate data sources. After downloading and installing it I received the following error:
$ ./riemann
ERROR [2016-10-10 12:21:36,614] main - riemann.bin - Couldn't start
java.util.concurrent.ExecutionException:
java.lang.UnsatisfiedLinkError:
/tmp/libnetty-transport-native-epoll3719688563306389605.so:
/tmp/libnetty-transport-native-epoll3719688563306389605.so: failed to
map segment from shared object
Not exactly what I was exepecting on our first date but I guess riemann plays hard to get. :) To make a little more sense out of this error I fired up strace to retrieve the ERRNO value and to see which system call was failing:
$ strace -f ./riemann
.....
[pid 9826]
open("/tmp/libnetty-transport-native-epoll27354233456383270.so",
O_RDONLY|O_CLOEXEC) = 45
[pid 9826] read(45,
"ELF
832) = 832
[pid 9826] fstat(45, {st_mode=S_IFREG|0644, st_size=63749, ...}) =
0
[pid 9826] mmap(NULL, 2146168, PROT_READ|PROT_EXEC,
MAP_PRIVATE|MAP_DENYWRITE, 45, 0) = -1 EPERM (Operation not
permitted)
.....
Now that hits the spot! mmap() is getting an EPERM which the mmap(2) manual page describes as:
“EPERM The prot argument asks for PROT_EXEC but the mapped area belongs to a file on a filesystem that was mounted no-exec.”
One problem solved! The tmp file system is mounted with the noexec flag which prohibits the mmap() from running with PROT_EXEC set. Here’s a picture of tmp:
$ mount | grep "/tmp"
tmpfs on /tmp type tmpfs (rw,noexec)
I’m not one to disable security measures so I went looking for a workaround. After reading through the netty source code for 20 minutes I came across this nugget:
NativeLibraryLoader.java
f = toDirectory(SystemPropertyUtil.get("java.io.tmpdir"));
if (f != null) {
logger.debug("-Dio.netty.tmpdir: " + f + " (java.io.tmpdir)");
return f;
}
Netty uses the java.io.tmpdir property to craft the temporary file location. Further digging showed that riemann passes properties to the Java runtime through the EXTRA_JAVA_OPTS variable:
exec java $EXTRA_JAVA_OPTS $OPTS -cp "$JAR" riemann.bin "$COMMAND" "$CONFIG"
Furthermore, you can pass properties to the Java runtime via the “-D” option. So to make riemann happy I set the EXTRA_JAVA_OPTS environment variable to “-Djava.io.tmpdir=/path/to/new/tmpdir” and fired off riemann:
$ EXTRA_JAVA_OPTS=-Djava.io.tmpdir=/sfw/riemann/latest/tmp cd
$ RIEMANN_HOME && ./riemann
Bingo! Riemann started and I was able to start working with it. Clojure takes me back to my Lisp programming days in college. Glad vim has () matching built in! :)