To the South of the Susannah Place museum are lots of stairs, going down. Walk down these stairs until you get to the New South Wales state records. Notice the fenced off original stairs, as in a picture below (with new stairs for people use on the side). Cross Harrington st, and go down some more stairs, slightly to the right, until you arrive at the NSW state records.
The NSW state records don’t have much to do with the rocks, but we thought you’d be interested in it anyway. It has lots of information (like family trees), photos and historical records of NSW. The bottom right photo shows some of its interior. Have a look around, then go back out the front door.