Satoshi Tanaka

College of Information Science and Engineering, Ritsumeikan University

The Borobudur Temple in Indonesia is a UNESCO World Heritage Site and the largest Buddhist Temple ruin in the world (see Fig.1). The Digital Archiving and Visualization of the Borobudur Temple project is working to digitally document and archive the entire temple site, including areas not currently visible to the naked eye, including sections of the structure buried underground or obstructed behind stone walls. We are also generating high-quality 3D digital images based on archived data. The project’s progress to date is detailed in our recent paper published in the ISPRS Journal of Photogrammetry and Remote Sensing [1].

In the course of our work we create 3D models for several aspects of the structure: 1) the temple building, 2) the underground foundation, and 3) the ‘hidden reliefs’ that were obscured for centuries behind a supporting wall. We then integrate the three models to construct a unified point-based 3D model that depicts the entire temple site. Finally, we apply high-definition see-through visualization to the unified model to review previously invisible features. Below, we explain the steps involved in the modeling and visualization one by one.

The creation of the overall 3D model of the visible portions of the temple building was produced with digital cameras shooting for photogrammetry. This included two sets of photogrammatic data: i) rough wide-area data measured from an unmanned aerial vehicle (UAV) flying over the temple building, and ii) high-definition close-range data acquired from onsite measurements conducted along the narrow corridors of the temple. Both sets of photo data acquired are registered into 3D point clouds using the SFM-MVS (structure from motion-multi-view stereo) method.

The underground foundation of the temple building cannot be directly measured by photogrammetry. However, a cross-section drawing that depicts the four-layer foundation is available for documentation (see Fig.2). The drawing is based on a boring survey executed by UNESCO in 1968. Using 2D CAD data from that drawing, which have been preserved by the Borobudur Conservation office, we extrapolated to produce a 3D reconstruction of the entire underground foundation. Since one cross-section drawing is insufficient to construct a full 3D model, we first mathematically extrapolate the 2D data of the boundary of each layer based on asymmetric rotation to create 3D surfaces.

Next, we produced a 3D reconstruction of the hidden relief panels using 2D archival photos taken with a monocular. For this, we use deep learning-based methods to estimate depth values from the 2D photos. An example of the 3D reconstruction (including the depth map and the 3D model) is shown in Fig.3.

By integrating the above 3D models of (1) to (3), we were able to obtain a comprehensive 3D model of the entire Borobudur Temple site. The integration is achieved by merging the 3D point cloud data of each of the three models. In the case of the model of the temple itself, 3D points are available directly from the 3D scanned data. As for the remaining models of (2) and (3), we convert them to 3D points by executing their resampling. Finally, we create a high-definition 3D see-through visualization of the unified 3D model (see Fig.4).

We have also developed a virtual reality (VR) system through which users can experience virtual tours of the Borobudur Temple (see Fig.5). The system is constructed based on the Unity game engine. The unified point-based 3D model of the entire temple site does not use polygonization, but rather the raw, unprocessed 3D scanned data. This approach is suitable to digitally archive and to develop VR of digitally preserved cultural heritage sites.

This work was part of “Research on Digital Archives of Indonesian Cultural Heritages”, a collaborative project between Ritsumeikan University in Japan and the National Research and Innovation Agency (BRIN) in Indonesia. This collaboration is based on a Memorandum of Understanding (MOU) concluded in May 2017 between BRIN and the Art Research Center (ARC) of Ritsumeikan University. The project is also supported by the Nara National Research Institute for Cultural Properties (Nabunken), Japan, based on an agreement between Nabunken and ARC.

References:

[1] Jiao Pan, Liang Li, Hiroshi Yamaguchi, Kyoko Hasegawa, Fadjar I. Thufail, Brahmantara, and Satoshi Tanaka, “3D reconstruction of Borobudur reliefs from 2D monocular photographs based on soft-edge enhanced deep learning,” ISPRS Journal of Photogrammetry and Remote Sensing, 183 (January 2022): 439-450, DOI: 10.1016/j.isprsjprs.2021.11.007