The observatory was founded and named after John Radcliffe by the Radcliffe Trustees. It was built on the suggestion of the astronomer Thomas Hornsby, who was occupying the Savilian Chair of Astronomy, following his observation of the notable transit of Venus across the sun's disc in 1769 from a room in the nearby Radcliffe Infirmary. The observatory building commenced to designs by Henry Keene in 1772, and was completed in 1794 to the designs of James Wyatt, based on the Tower of the Winds in Athens. Because of the viewing conditions, weather, and urban development at Oxford, the observatory was moved to South Africa in 1939. Eventually that site, in Pretoria, also became untenable and the facility was combined with others into the South African Astronomical Observatory (SAAO) in the 1970s.
The building is now used by Green Templeton College off the Woodstock Road and forms a centrepiece for the college. The original instruments are located in the Museum of the History of Science in central Oxford, with the exception of the Radcliffe 18/24-inch Twin Refractor telescope, which was transferred to the University of London Observatory.