The Graveyard Key is found in Oracle of Ages, in a secret cave in the southern part of Yoll Graveyard. It is needed to open a gate in the northern part of the graveyard, in order to then access the eastern part and reach the Spirit's Grave.
The key also appears in The Minish Cap, where Dampé the gravekeeper gives it to Link. As Link steps out of Dampé's house a crow steals it. By killing the bird in this same area, he will get it back and then be able to enter the graveyard.