Summary: A simple reverse engineering challenge requiring basic knowledge in assembly lanuage and python scripting as well as experience in debugging.
We start off with an ELF file that has no debugging symbols, meaning we can't analyse it using GDB.
We can however debug and analyse the binary using radare2. From the screenshot you can see we have just the main fucntion to analyse and no extra custom functions.
Using visual mode to better see where the jumps of the program lead to, we can see that the first jump is right after a comparison of the value in register RAX to the hex value 0x20, which equals 32 in decimal. This comparison comes right after a call to the strlen function which will fetch the integer value of the specified strings length and save the result in RAX. It's safe to assume the program is running our input through strlen but, let's run it first and see what happens.
As you can see it tells us that our input is incorrect but, even if we specify a string which has 32 characters we get the same message.
Luckily we can perform dynamic analysis on the program as it runs through radare2. The memory addresses look much different when we run the program in debug mode.
Here we set a breakpoint just after the jump that would be taken if the previous comparison instruction checks out as true. If we hit the breakpoint this will confirm that we do indeed need to feed the program 32 bytes of input.
And as you can see, we hit our breakpoint just fine. This means we are able to successfully perform dynamic analysis!
Now we want to analyse the next comparison instruction. The value of register EAX is being compared to the hexadecimal value of 0x1f, which is 31 in decimal. The jump instruction here 'jbe' stands for 'jump if below or equal to' and operates in a similar fashion to jle the only difference being, an unsigned comparison is performed. As we can see, if this check fails then the program will tell us we have the correct flag! And if it passes as true it appears to initialise a loop.
This loop is the meat of the program which we want to reverse engineer. The comparison being made is between the registers DL and AL, these are 8 bit segments of EDX and EAX respecitively meaning they will hold one byte of data each. As we can see there is a strange hardcoded string being loaded into RDX. It also appears our input is loaded into RCX. A byte is taken from each of these values, with the byte from our input being xor'd with the hexadecimal value of 0x19 which is 25 in decimal. From this we can assume that the loop is comparing our input after it has been xor'd to the hardcoded string byte by byte.
To confirm this we will run the program in debug mode again via radare2, set a breakpoint in the loop, and print out the values of the registers as we step through the program. As you can see we confirm that our input is loaded into RCX, we can also see the value of our byte in EAX before and after it's xor'd and finally, we can see the values of both AL and DL. Now we have all the information we need to reverse the encrypted flag!
By using a simple python script we are able to recreate the loop partially and check our output against the encrypted string. When we find matching characters we add them to our flag variable and once the loop has finished we print the flag! ◟ʕ´∀`ʔ◞