Projection-based exhibition methods have been used in a museum to create digital contents of cultural objects. They can enrich the exhibition of a cultural heritage object with physically co-located digital contents, and multiple users can enjoy the projected contents without the aid of additional devices. But the quality of projection is often restricted by surrounding environment such as the influence of the ambient light and occlusions by obstacles. The degree of freedom of projected contents also is limited due to the usage of a static projection surface. In this paper, we propose a novel projection-based exhibition system that resolves these shortcomings. We introduce a new design by combining the multi projection mapping with an optical see-through display. It provides high-quality projection contents robust to the influence of the ambient light and to the problem of obstacles. We also introduce a mechanically moving projection surface to provide a dynamic projection content by changing its shape and appearance. Our prototype system demonstrates applications that show a realistic three-dimensional effect and photo-realistic appearance of a cultural object.
The assessment of the structural behavior of historic masonry structures like gothic cathedrals is an important engineering and architectural issue, because of the economic and cultural relevance of such buildings. In this paper, we present a complete numerical methodology for point clouds processing, geometrical and parametric 3D modeling, and finite element structural analysis of the spire of the cathedral of Senlis, France. Our work highlights the particular difficulties linked with digitization and geometrical modeling of highly complex Gothic structures, as well as the need to find compromises between quality and accuracy of extracted data used for geometrical modeling and structural analysis. The methodology enables the semi-automatic transformation of a three-dimensional points cloud, surveyed through terrestrial laser scanner, into a three-dimensional geometrical BIM-oriented model, and its use to propose a consistent 3D finite element mesh suitable for advanced structural analysis. A full software chain was integrated in the proposed numerical process, so as to use the most important data contained in the real geometry and accurately transposed in the point clouds. After a successful data processing step with 3DReshaper software that proved to be necessary for enhancement of point clouds, a semi-automated geometrical BIM-oriented modeling step with Rhinoceros5 software and VisualARQ plugin has allowed the construction of a hybrid model by reverse engineering from the point clouds. This 3D model, containing both geometrical and parametric data of the structure, has been exported to the Hyperworks suite for finite element structural analysis under self-weight. Our computations focused on the estimation of the structure deformation and on the distribution of compression and traction stresses in all components of the complex structure. It is found that the spire is safe. Based on reliable and properly detailed results, our study provides significant information for understanding the behavior of the structure and potential damage monitoring.
Technologies such as the internet of the things and social computing offer promising opportunities to envision augmented experiences in a wide variety of cultural spaces. Such interactive technologies act as a hook to attract visitors attention, but if they do not provide opportunities to engage in a more personal and meaningful way they can be soon discarded. Augmented experiences need to be intrinsically motivating, connecting with values and expectations of visitors and institutions. In a world that is becoming more a more connected and moving towards a participatory culture, the need to participate actively to coproduce knowledge and meaning can be exploited as a powerful intrinsic motivator. In this paper we describe how the Social Display Environment (SDE) promotes participation in cultural spaces whilst keeping the physical connection with the real objects exhibited. The SDE was exhibited in a cultural center over a weekend to investigate different dynamics in terms of peoples interaction styles, behaviors, and perceptions. The prototype not also was assessed as easy to learn and useful, but also promoted a number of social interactions around the objects involved in the exhibition.
The study of cultural heritage involves many different activities, including visualizing digital data, analyzing information and sharing results. Current technologies focus on providing better tools for data representation and processing, neglecting the importance of analysis and sharing. In this paper, we present a software system, CHER-Ob, which offers powerful tools for evaluating and publishing the results of cultural heritage research. CHER-Ob provides the capability to visualize and add various types of annotations to data in a wide variety of formats. These annotations assist in the analysis phase, and are used for sharing the results of a study. A written report can be generated and automatically illustrated using the annotations. In addition, an ``animation scheme" is associated with each type of annotation. The schemes make it possible to generate an introductory video overview of an analysis by selecting preferences and annotations. A series of animated sequences of 2D and 3D objects will appear in a user-specified order in a video subtitled by annotations. The system is useful for integrating cultural and digital resources as well as for providing a method to author materials for disseminating cultural heritage findings to the public.
We consider the problem of localizing visitors in a cultural site from egocentric (first person) images. Localization information can be useful both to assist the user during his visit (e.g., by suggesting where to go and what to see next) and to provide behavioral information to the manager of the cultural site (e.g., how much time has been spent by visitors at a given location? What has been liked most?). To tackle the problem, we collected a large dataset of egocentric videos using two cameras: a head-mounted HoloLens device and a chest-mounted GoPro. Each frame has been labeled according to the location of the visitor and to what he was looking at. The dataset is freely available in order to encourage research in this domain. The dataset is complemented with baseline experiments performed considering a state-of-the-art method for location-based temporal segmentation of egocentric videos. Experiments show that compelling results can be achieved to extract useful information for both the visitor and the site-manager.
This paper describes the first analysis of the differences between what the Digital Humanities community who build online digital editions of texts provides, and what the user-base wants. It therefore provides useful guidance for those working on these projects and will also be of interest to those who commission, fund, and support digital editions of textual material, including publishers, universities, libraries and archives.
Munsell Soil Charts are a very common tool used by archaeologists for the color specification task. Charts are usually employed directly on cultural heritage sites to identify color of soils and collected artifacts. However, charts are designed to be used specifying the color through subjective perception of users, by visual mean, in a time consuming and error-prone procedure. It is likely that two users may estimate different Munsell notations for the same specimen, as colors are not perceived uniformly by different people. Hence, estimation process should be repeated several times and by more than a single expert user in order to be considered reliable. In this work, we employ our framework ARCA: Automatic Recognition of Color for Archaeology, specifically designed to provide a method for objective, deterministic, fast, and automatic Munsell estimation. ARCA is a valuable asset for archaeologists as it provides the definition of a smooth pipeline for an affordable Munsell notation estimation: image acquisition of specimens with general purpose digital cameras in an uncontrolled environment, manual sampling of specimen images in the ARCA desktop application, automatic Munsell color specification, and report generation. We further assess our method with improved color tolerance validations and evaluations, introducing a comparison between E00, E76, L*, a*, and b* differences. One of the main contribution of this paper is the extension of our former dataset ARCA108. We gathered two additional sets of images obtaining a new dataset consisting of pictures of Munsell Soil Charts Editions 2000 and 2009 plus images from a real test-case with 16 pottery shards. The new dataset counts 56,160 samples and 328 images, so it has been called ARCA328. Experimental results are reported to investigate which could be the best configuration to be used in the acquisition phase.
With this paper we present the ongoing research project Tango Danceability of Music in European Perspective and the transdisciplinary research design it is built upon. Three main aspects of tango argentino are in focusthe music, the dance, and the people in order to understand what is considered danceable in tango music. The study of all three parts involves computer-aided analysis approaches, and the results are examined within ethnochoreological and ethnomusicological frameworks. Two approaches are illustrated in detail to show initial results of the research model. Network analysis based on the collection of online tango event data and quantitative evaluation of data gathered by an online survey showed significant results, corroborating the hypothesis of gatekeeping effects in the shaping of musical preferences. The experiment design includes incorporation of motion capture technology into dance research. We demonstrate certain advantages of transdisciplinary approaches in the study of Intangible Cultural Heritage, in contrast to conventional studies based on methods from just one academic discipline.
Thanks to the digital preservation of cultural heritage material, multimedia tools, e.g. based on automatic visual processing, enable to considerably ease the work of scholars in the humanities and help them to perform quantitative analysis of their data. In this context, this paper assesses three different Convolutional Neural Network (CNN) architectures along with three learning approaches to train them for hieroglyph classification, which is a very challenging task due to the limited availability of segmented ancient Maya glyphs. More precisely, the first approach, the baseline, relies on pretrained networks as feature extractor. The second one investigates a transfer learning method by fine-tuning a pretrained network for our glyph classification task. The third approach considers directly training networks from scratch with our glyph data. The merits of three different network architectures are compared: a generic sequential model (i.e. LeNet), a sketch-specific sequential network (i.e. Sketch-a-Net), and the recent Residual Networks. The sketch-specific model trained from scratch outperforms other models and training strategies. Even for a challenging 150-class classification task, this model achieves 70.3% average accuracy and proves itself promising in case of small amount of cultural heritage shape data. Furthermore, we visualize the discriminative parts of glyphs with the recent Grad-CAM method, and demonstrate that the discriminative parts learned by the model agrees in general with the expert annotation of the glyph specificity (diagnostic features). Finally, as a step towards systematic evaluation of these visualizations, we conduct a perceptual crowdsourcing study. Specifically, we analyze the interpretability of the representations from Sketch-a-Net and ResNet-50. Overall, our paper takes two important steps towards providing tools to scholars in the digital humanities: increased performance for automation, and improved interpretability of algorithms.
Inpainting of Dunhuang Murals by Sparsely Modelling the Texture Similarity and Structure Continuity
In this paper, we present a new method for the analysis of Visible/Infrared multispectral sets producing chromatically faithful false-color images, which maintain a good readability of the information contained in the non-visible Infrared band. Examples of the application of this technique are given on the multispectral images acquired on the Pietà of Santa Croce of Agnolo Bronzino (1569, Florence) and on the analysis and visualization of the multispectral data obtained on Etruscan mural paintings (Tomb of the Monkey, Siena, Italy, V century B.C.). The fidelity of the chromatic appearance of the resulting images, coupled to the effective visualization of the information contained in the Infrared band, opens interesting perspectives for the use of the method for visualization and presentation of the results of multispectral analysis in Cultural Heritage diffusion, research and diagnostics.
In this work we explore the use of RGB-D cameras for digital preservation of cultural heritage. Three-dimensional (3D) digital preservation is usually performed with the use of three dimensional scanners, as the 3D points generated by these equipments are in average less than ten micrometers away from the real position. The downside of 3D scanners, in addition to the high cost, is the infrastructure requirements, for instance, own source of energy, large workspace with tripods, special training to calibrate and operate the equipment and the acquisition time, potentially taking several minutes for capturing a single image. An alternative is the use of low cost depth cameras that are easy to operate and only require connection to a laptop. There are several recent studies showing the potential of RGB-D sensors, however they often exhibit errors when applied to a full 360 degrees 3D reconstruction setup, known as the loop closure problem. This kind of error accumulation is intensified by the lower accuracy and large volume of data generated by RGB-D cameras. We propose a complete methodology for 3D reconstruction based on RGB-D sensors. In order to mitigate the loop closure effect, we developed a pairwise alignment method. Our approach expands the connectivity graph connections in a pairwise alignment system by automatically discovering new pairs of meshes with overlapping regions. Then we distribute the alignment error more evenly over the aligned pairs, avoiding the loop closure problem of full 3D reconstructions. The experiments were performed on a collection of 30 artworks made by the Baroque artist Antonio Francisco Lisboa, known as Aleijadinho, as part of the Aleijadinho Digital project conducted in partnership with IPHAN (Brazilian National Institute for Cultural and Artistic Heritage) and United Nations Educational, Scientific and Cultural Organization (UNESCO). We present results showing 3D models that are favorably compared to state-of-the-art methods available in the literature using RGD-D sensors. Our main contributions are: A new method for 3D alignment dedicated to solve the RGB-D camera loop closure problem; the development of practical solutions and 3D digital models of an important and challenging collection of Brazilian cultural heritage.
The last years, Virtual Archaeology has started to introduce more experiential elements in virtual reconstructions, therefore going beyond the traditional visualization of 3D architectural models. In the case of dissemination, these experiences are equated with a trip in time, thanks to which users witness what the past was like and learn about it. However, due to a lack of explicit theoretical frameworks and/or systematic evaluation focusing on such experiential elements, it is uncertain whether the intended goals are achieved and why. Based on a novel theoretical framework arising from the concept of Cultural Presence, this paper will investigate if and how current virtual environments achieve the feeling traveling to the past. To that end, six different virtual reconstructions of the Neolithic site of Çatalhöyük (Turkey) were built and evaluated in a between-subjects experiment. The results support the role of content meaningfulness, responsive characters, enhanced interaction, and multi-sensory realism in the achievement of successful VR-mediated experiences.
Archaeologists spend considerable time orientating and drawing ceramic fragments by hand for documentation, to infer their manufacture, the nature of the discovery site and its chronology, and to develop hypotheses about commercial and cultural exchanges, social organisation, resource exploitation and taphonomic processes. This study presents an R/Shiny application to automate this time-consuming orientation and drawing of pottery fragments. Orientation is based on the 3D geometry of pottery models, acquired in minutes with low-cost 3D scanners. Several methods (using normal vectors, or circle fittings, or profile fittings) are applied to determine the optimal position of the rotation axis. The profile and contours of the fragment, as well as any possible decoration, can then be depicted in various ways: photorealistic rendering or dotted patterns, calculated by ambient occlusion, combined or not with artificial light. The general workflow, evaluated using both synthetic and real-world fragments, appears to be rapid, accurate, and reproducible. It drastically reduces the amount of routine work required to document ceramic artefacts. The information produced, together with the 3D representation of the fragments, can easily be archived and/or exchanged within the archaeological community for further research. The source code built for the R environment (together with an installation notice and examples) is freely downloadable. To the best of our knowledge, it is the first operational tool available for such a purpose.