Approach to "Passive Fingerprint" tracers and debuggers.


There had been well known ways to know if a program is being debugged, most of then using signals and ptrace (see references). The following methods try not only to identify the presence of a debugger, but also identify it in a more passive way..

This paper was first published in august 2001. The crackme released for a (supossed white-hack)-con: ncn at Mallorca, spain, includes one of the methods explained in the paper. I've translated it to english just skipping historical notes and so.

Hope at the end still could be readable. The software has evolved since then, so I don't know if they are still working.

Basically those are the methods explained:
  • 1.- File descriptors: detecting the present debugger by the fingerprints revealed just after the fork().
  • 2.- Environment, variables and stack: shell conditions.
  • 3.- Process tree: session leaders, parents and so: system process conditions.



1. File descriptions, or passive fingerprint of debuggers by fork()

If you analyze how exec* and fork functions works, you'll see how a new process is created in memory. Basically, every tool/debugger/Tracer needs to launch the target file (when it's not attaching) using any of exec* functions providing by the system.

Note from the man page for fork syscall:
    ...fork  creates  a  child process that differs from the parent 
    process only in its PID and PPID, and in the fact that resource
    utilizations are set  to  0...
 
 or: http://www.unidata.ucar.edu/cgi-bin/man-cgi?fork+2

     The fork() and fork1() functions create a new process.   The
     new  process (child process) is an exact copy of the calling
     process (parent process).  The child  process  inherits  the
     following attributes from the parent process:

          o real user ID,  real  group  ID,  effective  user  ID,
            effective group ID
          o environment
          o open file descriptors
          o close-on-exec flags (see exec(2))
          o signal handling settings (that is, SIG_DFL,  SIG_IGN,
            SIG_HOLD, function address)
          o supplementary group IDs
          o set-user-ID mode bit
          o set-group-ID mode bit
          o profiling on/off status
          o nice value (see nice(2))
          o scheduler class (see priocntl(2))
          o all attached shared memory segments (see shmop(2))
          o process group ID -- memory mappings (see mmap(2))
          o session ID (see exit(2))
          o current working directory
          o root directory
          o file mode creation mask (see umask(2))
          o resource limits (see getrlimit(2))
          o controlling terminal
          o saved user ID and group ID


From this we can get some real fun topic to detect a parent process such a debugger. In the first example we'll see how open file descriptors may help us detecting that. Let's trace a very simple application and see the results:
/* basic_1.c */
void main (void){
	printf ("hello shell..\n");
}
First trace it with strace and see what happens with file descriptors (open and close functions)..:
ilo@rebeldy:~antidbg# strace -eopen ./basic_1
open("/etc/ld.so.preload", O_RDONLY)    = -1
open("/etc/ld.so.cache", O_RDONLY)      = 3
open("/lib/libc.so.6", O_RDONLY)        = 3

File descriptor returned by open function never reaches 4, as only 3 handlers are being used (stdin, stdout and stderr are descriptors too with values 0,1,2).

Launch it now with gdb:
rebeldy:/home/antidbg# strace -eopen /usr/bin/gdb -q ./basic_1
..
open("/lib/libthread_db.so.1", O_RDONLY) = 3
open("/etc/terminfo/x/xterm", O_RDONLY) = 5
open("/etc/inputrc", O_RDONLY)          = 5
open("/root/.gdbinit", O_RDONLY)        = 5
open("./basic_1", O_RDONLY)             = 5
open("./basic_1", O_RDONLY)             = 6
open("/etc/mtab", O_RDONLY)             = 7
..
As you can see, gdb before prompting it's interface open several files, so last call to open returns a higher value.

You can use lsof to see opened files by program/pid. The output from lsof now is:
debian:~/antidbg# lsof ./basic_1
COMMAND   PID USER   FD   TYPE DEVICE SIZE    NODE NAME
gdb     10438 root    5r   REG    3,5 4805 6714136 ./basic_1
gdb     10438 root    6r   REG    3,5 4805 6714136 ./basic_1
Next available file descriptor will be 7 as 5 and 6 remains openned by gdb (mapping and reading information).

So well.. we have a method here that allows us to detect gdb and maybe other debuggers. We can write a very simple application to get a new fd from the system and make a value-based decission.
/* get_fd.c */
 #include 
 #include 

 int main (int argc, char*argv[]){
 int fd;
        fd = open ("detect_fd.c", "r+b");
        printf ("fd : %d\n",fd);fflush(stdout);
        close (fd);

        return 0;
 } /* EOF */
//(i know, no error checks in example)
and now execute it and compare the results:
rebeldy:/home/antidbg# ./detect_fd
fd : 3
using gdb:
rebeldy:/home/antidbg# gdb ./detect_fd -q
(no debugging symbols found)...(gdb) run
Starting program: /home/antidbg/./detect_fd
fd : 5
(no debugging symbols found)...(no debugging symbols found)...
Program exited normally.

Not only we can execute an open call and read the value to see if gdb is tracing the program. We can also try to close a closed (in usual execution) descriptor to see if there are any errors. For example, try to close fd 3 in your program and you'll get an error if not being traced by gdb, but you'll get a great OK returned from close function if gdb is pressent.
        if (close(3)==-1){
                printf ("Hello shell..\n");
        } else {
                printf ("being traced..\n");
        }
By the way, there are many calls that allow us to detect gdb using this way like read, write, fchmod, fstat, fcntl, ioctl.. etc... almost all the uses fd.

Improve it, make it passive:

The most passive way to use this method is to keep a file handler used in memory. When you use a handler to access a file, and even when you close that file, the value is still in the memory allocated for the variable untill it's erased by another operation. Making a fd global for example, you could check at any moment of the execution if gdb is present making it hard to detect.



2º.- environment, arguments and variables the shell method:

From all the amount of fingerprints leaved by fork, let's chosse another one, for example, environment and arguments..

When reading the manual page of bash:
       _      At  shell startup, set to the absolute file name of
              the shell or shell script being executed as  passed
              in the argument list.  Subsequently, expands to the
              last argument to the previous command, after expan­
              sion.   Also set to the full file name of each com­
              mand  executed  and  placed  in   the   environment
              exported to that command.  When checking mail, ...


let's program a 2nd basic application for testing somethings..
/* test file */
int main (int argc, char*argv[]){
        printf ("argv[0] %s\n", argv[0]);fflush(stdout);
        printf ("'_'     %s\n", getenv("_"));fflush(stdout);
		return (0);
} 

This one will show the first argument in command line (program name) and the '_' environment variable, and run it with some tracers..

This is the output table with the results:
           argv[0]                getenv("_")
shell      ./test                 ./test
strace     ./test                 /usr/bin/strace
ltrace     ./test                 /usr/bin/ltrace
fenris     ./test                 /usr/bin/fenris
gdb        /home/antidbg/./test   (NULL) 


Improve it, make it passive:
We can use getenv() to read that value, we can again detect tracers or debuggers. But then, getenv will be an alarm for tracers, like ptrace is, so let's do it harder to detect reading the environment variables withouth getenv() function:
/* test-env.c */
#include 
#include 

int main (int argc, char *argv[], char *env[]){
int c=0;
  while (env[c]>0){
    if ((env[c][0]=='_') && (env[c][1]=='=')){
      if ((argv[0][0]==env[c][2])&&argv[0][3]==env[c][5])){
        printf ("Hello shell..\n"); 
        fflush(stdout);
        return(0);
      }
      printf ("being traced?\n"); fflush(stdout);
      return (0);
    }
    c++;
  }
  printf ("maybe in gdb?..\n"); fflush(stdout);
  return (-1);
}
Try to execute the program again againts the debuggers and you'll see the results. With diferent shells you can get diferent results.. nothing more to say here. Execution environment is also something to check when detecting a debugger.



3 .- Process tree: session leaders, parents and so. The system method

This is an output from pstree:
init-+-atd
     |-cron
     `-sshd---bash---bash-+-fenris---test
                          |-strace---test
                          |-gdb---test
                          |-ltrace---test
                          `-test
As you can see, i've started a program with all the tracers i have installed for this game.

There are a lot of pids involved in this simple tree. A pid for each process, that can be also ppid (parent pids) of others, an also sids (session ids), leader pids of the session. You can read more about session ids in the manual of your shell.
  ...The getsid() function obtains the process group  ID  of  the
  process  that is the session leader of the process specified 
  by pid... 
Check how ppid and sid (parent pid and session id) differs with diferent environments. Using a simple program get a list of pids readables at user space involved in the execution:
int main (int argc, char*argv[]){
        /* when calling with 0 it's our own pid*/
        printf ("sid : 0x%x\n", getsid(0));
        printf ("ppid: 0x%x\n", getppid());
        printf ("pgid: 0x%x\n", getpgid(0)); 
        printf ("pgrp: 0x%x\n", getpgrp());
        return (0);
}
And launch this against all the tracers to see this results
             shell       gdb     strace    ltrace    fenris
getsid      0x1968    0x1968     0x1968    0x1968    0x1968
getppid     0x1968    0x3a6f     0x3a71    0x3a73    0x3a75
getpgid     0x3a6e    0x3a70     0x3a71    0x3a73    0x3a75
getpgrp     0x3a6e    0x3a70     0x3a71    0x3a73    0x3a75
When the program is being executed from the shell getsid == getppid. Also, again , gdb differs from the others as getppid != getpgid.

A way to improve (resume) this could based in what sid is and the results table:
 if (setsid()!=-1){ debugger_present= TRUE }
This will happend with strace, ltrace and fenris (gdb is different).

anyway, setsid is a function trappable by debuggers, so try to find a better aproach for a more passive way.

Conclusion

Still a lot of terrain to explore. Fingerprinting tracers or debuggers is not so dificult as shown here.

Biblio

General documentation

http://www.gnu.org/manual/
http://www.gnu.org/manual/bash-2.05a/bashref.html

Old stuff:
http://vx.netlux.org/lib/vsc04.html#5
http://www.einride.org/source/antitrace.c

Used programs..
http://www.gnu.org/directory/gdb.html
http://sourceforge.net/projects/strace/
http://utopia.knoware.nl/users/driehuis/ (ltrace)
http://lcamtuf.coredump.cx/fenris/devel.shtml
ftp://vic.cc.purdue.edu/pub/tools/unix/lsof/