When Link first reaches the Gerudo Mesa, he finds the broken portion of the Bridge of Eldin jammed in the outcropping. Before he can reach it however, a Twilight Portal opens and three Shadow Beasts ambush the hero. Once Link dispatches the enemies, he can get close to the bridge part, and with Midna's aid, warp it back to where it belongs in Eldin Province.
If Link returns to the Gerudo Mesa after warping the bridge fragment, he will find that the obstruction was covering the entrance to the Cave of Ordeals. With the bridge gone, Link can successfully enter the mini-dungeon, although he cannot complete it until he has obtained the items from a majority of the dungeons in the game.