[Linux] Unloading From the Cheat Itself

So, I stumbled across this problem while writing my cheat. The unload/panic key. On windows I did it rather easily, create a thread which unhooks everything, waits a bit and calls FreeLibraryAndExitThread. On Linux, you would call dlclose on the cheat’s handle until the return value is non-zero, but here the root of the problem lays – once the library is actually being unloaded, glibc munmaps the region from the address space, and once dlclose returns, an instruction in already unmapped memory range will be tried to be executed. Of course, you will crash.

So here is where my hacky method comes in to play. The idea is simple: create a thread which will make sure dlclose will return not to the cheat and after the cheat code returns.

First of all, we need to get the handle of our library. You can use the following code for it:

Dl_info info;
if (dladdr(ptr, &info) && info.dli_fname)
return dlopen(info.dli_fname);

Where ptr is a pointer to some variable/function that is known to be inside the cheat address space itself, not in the heap or anywhere else.

Afterwards, you need to get an idea on how dlclose works. GlibC keeps track of how many times each library gets referenced (called dlopen on), and each time you call dlclose, the reference count is subtracted by 1, and once it reaches zero, the library gets unloaded. Once it gets unloaded, before freeing the memory and calling munmap, a destructor function gets called, if there is any. So in short, you may have to call dlclose multiple times to make it be unloaded. By default, it should be 2, since we call dlopen on the lib once we want to get it’s handle, but we can’t be sure. So we fabricate a loop similar to this:

while (!dlclose(handle));

This will unload any library that could be referenced any amount of times, but won’t unload itself. To solve this, we need to create a thread of dlclose, since the thread will not run inside the cheat’s address range:

pthread_t thread;
pthread_create(&thread, NULL, (void *(*)(void *))dlclose, handle);

This is very hacky, since posix threads expect the function to return a void pointer, while dlclose returns an int. That’s why we need to cast the function typedef over to something compatible. I also made the thread detached just in case:

pthread_attr_t tAttr;
pthread_attr_init(&tAttr);
pthread_attr_setdetachstate(&tAttr, PTHREAD_CREATE_DETACHED);

And pass the pointer to the attribute over as a second argument of pthread_create.

But now we lose the ability to check for the return value. And even if we did, we would only be able to check for it once we are unloaded AKA once we can’t check for it. Seems like a cat and mouse game huh? Well, we can place in more hacks. We can have a thread lock, which will be locked during the dlclose calling, then, inside the destructor, we set a bool stating we have are unloading, and try to lock the same lock, the while loop will check for the bool, and if it’s set, exit the loop and unlock the lock, voila! We will use atomic_flag for the lock and atomic_bool for the bool value since we don’t want to cause raise conditions just for that haha.

#ifdef __linux__
std::atomic_flag dlCloseLock = ATOMIC_FLAG_INIT;
std::atomic_bool isClosing(false);

__attribute__((destructor))
void DLClose()
{
isClosing = true;
LOCK(dlCloseLock);
UNLOCK(dlCloseLock);
usleep(1000*1000);
}

We need to sleep afterwards, because we are not sure how long the main thread will be inside the cheat (let’s be real, 1 second might be too long though but still, this is a separate thread and is unnoticeable). Here is lock and unlock:

#define LOCK(lck) while (lck.test_and_set(std::memory_order_acquire));
#define UNLOCK(lck) lck.clear(std::memory_order_release);

And this is how the unload function looks like:

static void ULoadThread(MHandle handle)
{
#ifdef _WIN32
Sleep(1000);
FreeLibraryAndExitThread(handle, 1);
#else
dlclose(handle);
LOCK(dlCloseLock);
//The hackiest shit ever
while (!isClosing.load())
{
pthread_attr_t tAttr;
pthread_t thread;
pthread_attr_init(&tAttr);
pthread_attr_setdetachstate(&tAttr, PTHREAD_CREATE_DETACHED);
pthread_create(&thread, &tAttr, (void *(*)(void *))dlclose, handle);
usleep(500*1000);
}
UNLOCK(dlCloseLock);
#endif
}

We are sleeping so much in the loop because again, we have no idea how long will it take for the destructor to be called, and we don’t want to call dlclose twice at the same time. But this one is in the main thread (unlike the name of the function implies, it’s separate thread only on windows) thus is noticeable, so I might try with lower sleep values since it shouldn’t really take half a second.

This is it, hope it helps others with Linux internals.