CCSC 2021 Writeups — Neuralmate
Description:
Benny is showing off again… intelligence is a trait of his, but so is laziness. This time he came up with a neural network for recognizing the position of chess pieces on the board for him without even looking.He gave you 3 boards and challenged you to beat him in a single move. He sounds confident… but maybe you can mess with his brain.Beat him in all three boards to get the flag.
Neuralmate was the only challenge with the AI tag on this CTF. We are told in the description that there is a neural network which recognises the board position. We are given three boards and we need to checkmate in a single move.
After looking at the boards, we understand that beating in a single move is impossible without any modifications to the board.
When connecting to the TCP socket, we are requested for the image of the board. The neural network is then used to identify the position of the board, and we are requested for a move. From this, we can understand that we can alter the image given to us to our advantage, so that the neural network identifies a different board position, in which we can actually checkmate in a single move.
Looking at the service code given in the asset, I noticed the lines
if not board.is_valid() or not is_visually_similar(
groundtruth_board_img, board_input, 0.06
):
raise ValueError("What is this? Are you serious?")The service checks if the original image (ie one of the three boards given as part of the asset) and the input are similar. Looking at the code in the is_visually_similar function, we can see that the difference is calculated by
difference = (img_a - img_b).reshape(-1)
return max(difference.max(), abs(difference.min()))We need the absolute difference between each pixel in our input and the corresponding pixel in the original image to be less than or equal to 0.06.
Therefore, we can’t just outright give the neural network a completely different image, it needs to be as similar as possible, but different enough for the neural network to misidentify it.
Having all the necessary information, we can start working. The first thing that I did was having a closer look at the boards, and trying to find a way to modify it with the least changes so that we can win with a checkmate, while the board remains valid.
In order to be in a checkmate position, the changes I chose were to add a knight at g6 and a knight at e7. However, since we will need to make a move before the checkmate, I moved the knight from e7 to d5. The final image I used to win the board was this one, were you can see that there are some changes in those cells
In order to produce the image above, we need to understand how neural networks work, but not in a traditional sense. We need to see how each pixel in the input affects the final decision of the piece present in a cell. This is called the gradient of the output with respect to the input of the neural network.
Since the neural network uses tensorflow, we can use GradientTape, which can calculate the gradient we need. After having that gradient, we know in which direction we need to change the input (ie increase or decrease the value of a pixel) so that the output goes to where we want it. Since we want to add white knights to the board, we need to maximise the probability that the piece with the largest output is going to be the one for the knight, which we can see from the code is the 10th (0-indexed) output for each cell.
Since we want to maximise that, we change the input board in the direction denoted by the gradients. The code used for this is here:
(Remember that we need to make sure that the changes are within the range we are allowed, so we clip the input to the range, and also to the range [0,1], so that it can be converted to an image without any losses at the end)
After running the above code (here is the full original code, where I also tried minimizing the output of another piece at the same time, and also experimented with dynamic step sizes) in a loop for some time, we generate an image that looks like this, and is identified as (almost) all cells having a knight
Now we just split this into cells, choose the cells we need and replace the original image with it.
We repeat the same process for the other 2 boards given, and we then upload it to the TCP service to get the flag. The images that I used for the other 2 boards are:
After generating those images in the exact same way as above, we upload it to the service using pwntools
Flag: CCSC{y0u_4r3_r3ally_g4tt1ng_0n_my_n3rv3s_n0w!!}
