You can’t fail to answer a riddle and then declare it invalid. Gollum messed up by accepting a cry of incredulity as a legitimate riddle.
Gollum does try to declare it as invalid before attempting to answer:
“What have I got in my pocket?” he said aloud. He was talking to himself, but Gollum thought it was a riddle, and he was frightfully upset.
“Not fair! not fair!” he hissed. “It isn’t fair, my precious, is it, to ask us what it’s got in its nassty little pocketses?”
Bilbo seeing what had happened and having nothing better to ask stuck to his question, “What have I got in my pocket?” he said louder.
“S-s-s-s-s,” hissed Gollum. “It must give us three guesseses, my preciouss, three guesseses.”Ah, fair. I thought he gave an answer and then said “no fair”. Uh, get bullied, nerd. I suppose part of riddling is knowing when to simply fight your competition.
He didn’t intend for it to be his riddle, G dog jumped to conclusions
Even three guesses was not really a fair handicap for it either



