The LIBOR square is a (rare) derivative contract which pays, as mentioned in the question, the square of the interest rate move between two dates. If LIBOR at inception is y, and upon settlement is x, the contract pays (x - y)^2 for x > y; and -(x - y)^2 for x < y.
For any question that involves calculating delta or gamma, and the payoff is described in terms of variables as is the case here, remember that delta is always the first derivative and gamma is the second derivative. For this question, let us calculate the second derivative and see what the gamma is:
If x > y, then the payoff is (x - y)^2
The first derivative wrt x is 2(x - y)
The second derivative wrt x is 2.
ie, the gamma is 2
If x < y, then the payoff is -(x - y)^2
The first derivative wrt x is -2(x - y)
The second derivative wrt x is -2.
ie, the gamma is -2
If x = y, then the payoff is 0. Both the first and the second derivatives are zero. ie the gamma is 0.
Based on the above, we see that the contract can have a gamma of either 0, +2 or -2. 1 is not a possible value for gamma, and therefore Choice 'b' is the correct answer.
