Why mutrace

Recently, I want to profile the linux user lock(pthread mutex/rwlock) performance of a project, and there are few options for this purpose:

• valgrind(drd)
• systemtap(futexes.stp)
• lttng
• mutrace

Finally, I selected the mutrace due to:

1. valgrind(drd): It’s really slow, cannot provide credible information.
2. systemtap(futexes.stp): It’s really great in profiling kernel, but the user layer profile need to additional efforts to setup the environment(some debuginfo pkg).
3. lttng: There is only rpm pkg available on RHEL7, have to compile by myself. And there are more than one dependency lib, hard to fix the dependency issue.
4. mutrace: There is no rpm pkg available on RHEL6, have to compile by myself, and fortunately it’s not very hard to pass the compilation.

Installation Steps

To get it done, need a few steps:

1. Get the source code

   git clone git://git.0pointer.net/mutrace.git
   

2. Modify the autoconf required version from 2.68 to 2.63

   diff --git a/configure.ac b/configure.ac
   index fcb1397..0d36e41 100644
   --- a/configure.ac
   +++ b/configure.ac
   @@ -18,7 +18,7 @@
   # You should have received a copy of the GNU Lesser General Public
   # License along with mutrace. If not, see <http://www.gnu.org/licenses/>.
   
   -AC_PREREQ([2.68])
   +AC_PREREQ([2.63])
   
   AC_INIT([mutrace], [0.2], [mzzhgenpr (at) 0pointer (dot) net])
   AC_CONFIG_SRCDIR([mutrace.c])
   

3. Upgrade gcc version to at least gcc 4.6

4. Do some code changes for backtrace-symbols.c to pass the compilation

   diff --git a/backtrace-symbols.c b/backtrace-symbols.c
   index 0a0d751..6f84c56 100644
   --- a/backtrace-symbols.c
   +++ b/backtrace-symbols.c
   @@ -34,6 +34,8 @@
     along with this program; if not, write to the Free Software
     Foundation, 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.  */
   
   +#include "config.h"
   +
   #define fatal(a, b) exit(1)
   #define bfd_fatal(a) exit(1)
   #define bfd_nonfatal(a) exit(1)
   @@ -44,13 +46,13 @@
   #define true 1
   #define false 0
   
   -#define _GNU_SOURCE
   +//#define _GNU_SOURCE
   #include <string.h>
   #include <stdio.h>
   #include <stdlib.h>
   #include <execinfo.h>
   #include <bfd.h>
   -#include <libiberty.h>
   +//#include <libiberty.h>
   #include <dlfcn.h>
   #include <link.h>
   #if 0
   

5. Run ./bootstrap.sh to generate Makefile

6. Change ‘-O0’ to ‘-O2’ from CFLAGS in the Makefile
7. Run make and make install

In the end

Now, mutrace, matrace and the other related libraries have been generated, profile your program by:

mutrace -r $test_app ...

And big thanks to the original author @Lennart Poettering

References

http://0pointer.de/blog/projects/mutrace.html
http://0pointer.net/blog/projects/mutrace2.html

have a fun :)