The virtual reconstruction of heritage sites and artefacts is a complicated task that requires researchers to gather and assess many different types of historical evidence which can vary widely in accuracy, authority, completeness, interpretation and opinion. It is now acknowledged that elements of speculation, interpretation and subjectivity form part of 3D reconstruction using primary research sources. Ensuring transparency in the reconstruction process and therefore the ability to evaluate the purpose, accuracy and methodology of the visualization is of great importance. Indeed, given the prevalence of 3D reconstruction in recent heritage research, methods of managing and displaying reconstructions alongside their associated metadata and sources has become an emerging area of research. In this paper, we describe the development of techniques that allow research sources to be added as multimedia annotations to a 3D reconstruction of the British Empire Exhibition of 1938. By connecting a series of wireless touchpad PCs with an embedded webserver we provide users with a unique collaborative interface for semantic description and placement of objects within a 3D scene. Our interface allows groups of users to simultaneously create annotations, whilst also allowing them to move freely within a large display visualization environment. The development of a unique, life-size, stereo visualization of this lost architecture with spatialised semantic annotations enhances not only the engagement with and understanding of this significant event in history, but the accountability of the research process itself.