Projection-based exhibition methods have been used in a museum to create digital contents of cultural objects. They can enrich the exhibition of a cultural heritage object with physically co-located digital contents, and multiple users can enjoy the projected contents without the aid of additional devices. But the quality of projection is often restricted by surrounding environment such as the influence of the ambient light and occlusions by obstacles. The degree of freedom of projected contents also is limited due to the usage of a static projection surface. In this paper, we propose a novel projection-based exhibition system that resolves these shortcomings. We introduce a new design by combining the multi projection mapping with an optical see-through display. It provides high-quality projection contents robust to the influence of the ambient light and to the problem of obstacles. We also introduce a mechanically moving projection surface to provide a dynamic projection content by changing its shape and appearance. Our prototype system demonstrates applications that show a realistic three-dimensional effect and photo-realistic appearance of a cultural object.
The assessment of the structural behavior of historic masonry structures like gothic cathedrals is an important engineering and architectural issue, because of the economic and cultural relevance of such buildings. In this paper, we present a complete numerical methodology for point clouds processing, geometrical and parametric 3D modeling, and finite element structural analysis of the spire of the cathedral of Senlis, France. Our work highlights the particular difficulties linked with digitization and geometrical modeling of highly complex Gothic structures, as well as the need to find compromises between quality and accuracy of extracted data used for geometrical modeling and structural analysis. The methodology enables the semi-automatic transformation of a three-dimensional points cloud, surveyed through terrestrial laser scanner, into a three-dimensional geometrical BIM-oriented model, and its use to propose a consistent 3D finite element mesh suitable for advanced structural analysis. A full software chain was integrated in the proposed numerical process, so as to use the most important data contained in the real geometry and accurately transposed in the point clouds. After a successful data processing step with 3DReshaper software that proved to be necessary for enhancement of point clouds, a semi-automated geometrical BIM-oriented modeling step with Rhinoceros5 software and VisualARQ plugin has allowed the construction of a hybrid model by reverse engineering from the point clouds. This 3D model, containing both geometrical and parametric data of the structure, has been exported to the Hyperworks suite for finite element structural analysis under self-weight. Our computations focused on the estimation of the structure deformation and on the distribution of compression and traction stresses in all components of the complex structure. It is found that the spire is safe. Based on reliable and properly detailed results, our study provides significant information for understanding the behavior of the structure and potential damage monitoring.
We consider the problem of localizing visitors in a cultural site from egocentric (first person) images. Localization information can be useful both to assist the user during his visit (e.g., by suggesting where to go and what to see next) and to provide behavioral information to the manager of the cultural site (e.g., how much time has been spent by visitors at a given location? What has been liked most?). To tackle the problem, we collected a large dataset of egocentric videos using two cameras: a head-mounted HoloLens device and a chest-mounted GoPro. Each frame has been labeled according to the location of the visitor and to what he was looking at. The dataset is freely available in order to encourage research in this domain. The dataset is complemented with baseline experiments performed considering a state-of-the-art method for location-based temporal segmentation of egocentric videos. Experiments show that compelling results can be achieved to extract useful information for both the visitor and the site-manager.
The manual shape archaeological projectile point classification is an extensive and complex process because involves a large number of typological categories. The present work is focused on the development of an automatic classifier algorithm of projectile points, based on its digital image. Additionally, the algorithm supports different conditions such as scale and quality image. The algorithm requires a uniform background and an approximate north south projectile point orientation. The principal computer methods that compose the classifier are the CSS-map (curvature scale space map), the gradient contour application on the projectile point, and the SVM (Support Vector Machines) algorithm. The classifier was trained and tested on a dataset of approximately 800 projectile points images. The results have shown a better performance than other shape descriptors such as PHOG, HOOSC (both used in a ?Bag of Words? context), and geometric moment invariants (Hu moments).
The most successful approach for hieroglyph representation for retrieval starts thinning the hieroglyph contour lines. Then, a set of interest points from the thinned hieroglyph is randomly selected, and a local descriptor from each selected interest point is computed. These local descriptors are used under the Bag of Visual Words (BoVW) model for performing hieroglyph retrieval. This approach has as drawback that a random selection of a subset of interest points does not guarantee suitably preserving the most useful information of a hieroglyph. Additionally, during the thinning process, contour shape distortions could lead to unwanted branches, which do not represent important information and could affect the quality of the local descriptors. Therefore, in this paper, we propose improving the hieroglyph representation quality by pruning unwanted branches over the thinned contour of a hieroglyph and introducing an improved interest point selection process. Our experiments show that our proposal allows significantly improving the image retrieval results previously reported in the literature.
The technological advances brought about by the Internet of Things enable new opportunities for a more direct interaction between users, objects and places. This is an extremely valuable innovation for the Cultural Heritage sector, as it allows a more transparent use of technology in the digital augmentation of museums and cultural heritage sites. The possibility to augment physical objects with sensors detecting when they are moved and manipulated enables scenarios where descriptive information about objects is presented to users at the very exact time they are looking at them, stimulating engagement. This paper describes a collaborative research effort between cultural heritage professionals, human-computer interaction experts and developers which was aimed at investigating the goals and constraints curators consider for a physical encounter between visitors and historic relics. In a case study, we co-designed an interactive plinth centred on tangible interaction and evaluated the impact on the user experience of combining digital information with a hands-on experience of relics of World War I. Our findings show that visitors value this type of tangible interaction with collection objects positively, as it allows the discovery of details and the learning of aspects that normally go unnoticed. The synergy between physical and digital aspects stimulates empathy with the original users of the object and fosters social interaction.
Acquiring images of archaeological artifacts is an essential step for the study and preservation of cultural heritage. In constrained environments, traditional acquisition techniques may fail or be too invasive. We present an optical device including a camera and a wedge waveguide that is optimized for imaging within confined spaces in archeology. The major idea is to redirect light by total internal reflection to circumvent the lack of room, and to compute the final image from the raw data. We tested various applications onsite in autumn 2017 during an archaeological mission in Medamoud (Egypt). Our device was able to successfully record images of the underground from slim trenches of about 15 cm wide, including underwater trenches, and between rocks composing a wall temple. Experts agreed that the acquired images were good enough to get useful information that cannot be obtained as easily with traditional techniques.
In the cultural heritage domain, games have been used to engage users into an active state of learning through immersive and playful interactions that include visually enriched interaction contexts. There is evidence that individual differences in the inherent way people search, process, analyze, comprehend, store, and retrieve visual information in their surrounding environment are reflected on their performance, experience, effectiveness and efficiency in such environments. Despite that cultural heritage game designers favor learning experiences in such contexts, current design practices of cultural heritage games barely consider the gamers individual differences in visual information processing. This can be accredited in that there is a deficiency in understanding and modelling the effects among users visual behavior, gameplay behavior, and cognitive styles in cultural heritage games towards knowledge acquisition; resulting in insufficient methods of creating cognition-centered user models and considering such human cognitive factors practically, within current state-of-the-art design approaches. We selected three known cultural heritage games, adopted a credible cognitive style theory and performed, over a six-month period, three separate user studies (N=127) following a between-subject, eye-tracking based, experimental design. The study results revealed that game designers decisions, related to visual exploration and search, unintentionally favored users belonging to specific cognitive style groups by influencing their visual and consequently gameplay behavior, resulting in differences in knowledge acquisition. Findings also revealed correlation effects, during game play, among individual differences in visual information processing, users visual behavior strategies and gameplay behavior, which provide useful insights for practitioners and researchers about the technical feasibility to develop real-time cognitive-centered user models based on users eye-gaze patterns, aiming to scaffold the development of personalized solutions adjusted to users individual cognitive characteristics. Practical implications related to an eye-gaze driven cognitive-centered personalization framework for cultural-heritage games are also discussed in this paper.
With this paper we present the ongoing research project Tango Danceability of Music in European Perspective and the transdisciplinary research design it is built upon. Three main aspects of tango argentino are in focusthe music, the dance, and the people in order to understand what is considered danceable in tango music. The study of all three parts involves computer-aided analysis approaches, and the results are examined within ethnochoreological and ethnomusicological frameworks. Two approaches are illustrated in detail to show initial results of the research model. Network analysis based on the collection of online tango event data and quantitative evaluation of data gathered by an online survey showed significant results, corroborating the hypothesis of gatekeeping effects in the shaping of musical preferences. The experiment design includes incorporation of motion capture technology into dance research. We demonstrate certain advantages of transdisciplinary approaches in the study of Intangible Cultural Heritage, in contrast to conventional studies based on methods from just one academic discipline.
Inpainting of Dunhuang Murals by Sparsely Modelling the Texture Similarity and Structure Continuity
In this paper, we present a new method for the analysis of Visible/Infrared multispectral sets producing chromatically faithful false-color images, which maintain a good readability of the information contained in the non-visible Infrared band. Examples of the application of this technique are given on the multispectral images acquired on the Pietà of Santa Croce of Agnolo Bronzino (1569, Florence) and on the analysis and visualization of the multispectral data obtained on Etruscan mural paintings (Tomb of the Monkey, Siena, Italy, V century B.C.). The fidelity of the chromatic appearance of the resulting images, coupled to the effective visualization of the information contained in the Infrared band, opens interesting perspectives for the use of the method for visualization and presentation of the results of multispectral analysis in Cultural Heritage diffusion, research and diagnostics.
Digital heritage comprises a broad variety of approaches and topics and involves researchers from multiple disciplines. Against this background, this paper presents a four-stage investigation on standards, publications, disciplinary cultures as well as scholars in the field of digital heritage and particularly tangible objects as monuments and sites, carried out in 2016 and 2017. It includes results of (1) the inquiry of nearly 4000 publications from major conferences, (2) a workshop-based survey involving 44 researchers, (3) 15 qualitative interviews as well as (4) two online surveys with 1000 and 700 participants respectively. As an overall finding, the community is driven by researchers from European countries, especially Italy, with a background in humanities. Cross-national co-authorships are promoted by cultural and spatial closeness and?probably due to funding policy?EU membership. A discourse is primarily driven by technologies and the most common keywords refer to the technologies used. Most prominent research areas are data acquisition and management, visualization or analysis. Recent topics are for instance unmanned airborne vehicle (UAV)-based 3D surveying technologies, augmented and virtual reality visualization, metadata and paradata standards for documentation or virtual museums. Since a lack of money is named as biggest obstacle nowadays, competency and human resources are most frequently named as demand. An epistemic culture in the scholarly field of digital heritage is closer to engineering then to humanities. Moreover, conference series are most relevant for a scientific discourse, and especially EU projects set pace as most important research endeavors.
This article presents a new algorithm for the automated reconstruction and visualization of damaged ancient inscriptions. After reviewing current methods for enhancing incisions, a hybrid approach is adopted that combines advantages of 2D and 3D analytical techniques. A photogrammetric point cloud of an inscription is projected orthographically from an ideal vantage point, generating a 2.5D raster including channels describing depth and surface derivatives. Next, the obstacles to legibility posed by breaks in an ancient text are considered, leading to the creation of a new segmentation algorithm based on SLIC superpixels and region-merging that operates on the geometry channels of the inscribed surface, rather than color or intensity values. With high accuracy, the algorithm classifies surface points by their likelihood of belonging to the uninscribed original plane, deliberate strokes, or breaks. Conventions for static visualization are developed for epigraphical analysis and publication. Three case studies demonstrate the power and flexibility of this method, which has resulted in substantial changes to IG XIV 1, an early Greek text whose reading has been debated for more than 150 years.
Terrestrial laser scanning campaigns provide an important means to document the 3D structure of historical sites. Unfortunately, the process of converting the 3D point clouds acquired by the laser scanner into a coherent and accurate 3D model has many stages and is not generally automated. In particular, the initial cleaning stage of the pipeline in which undesired scene points are deleted remains largely manual and is usually labour intensive. In this paper we introduce a semi-automated cleaning approach which incrementally trains a random forest (RF) classifier on an initial keep/discard point labelling generated by the user when cleaning the first scan(s). The classifier is then used to predict the labelling of the next scan in the sequence. Before this classification is presented to the user, a denoising post-process, based on the 2D range map representation of the laser scan, is applied. This significantly reduces small isolated point clusters, which the user would otherwise have to fix. The user then selects the remaining incorrectly labelled points, and these are weighted, based on a confidence estimate, and fed back into the classifier to retrain it for the next scan. Our experiments, across 4 scanning campaigns, show that when the scan campaign is coherent i.e. it does not contain widely disparate or contradictory data, the classifier yields a keep/discard labelling which typically ranges between 95-99%. This is somewhat surprising, given that the data in each class can represent many object types, such as tree, person, wall etc, and that no further effort beyond the point labeling of keep/discard is required of the user. An informal timing experiment over a 15 scan campaign, to compare the cleaning times produced by our software against those of an experienced user, showed that we were able to produce a result at 98% (average) accuracy 20 minutes sooner than the cumulative expert cleaning time, even with a non optimized code.
Numerous image inpainting algorithms are guided by a basic assumption that the known region in the original image itself can provide sufficient prior information for the guess recovery of the unknown part, which is not often the case in actual art image inpainting. Sometimes, the art image need to be inpainted is so badly damaged that there is little priors as a good model to infer the unknown fragment. Focusing on the lookup strategy for optimal patches, a novel semi-automatic exemplar-based inpainting framework based on a sample dataset is proposed in this paper to solve such a problem with 3 steps: 1) reference images selection from the dataset using deep convolutional network; 2) sample image creation based on reference images with melding algorithm; 3) exemplar-based inpainting according to the created sample image. Several comparative experiments over Dazu Rock Carvings with the state-of-the-art image completion approaches demonstrate the effectiveness of our contributions. Firstly, the search space for candidate patches is extended from the known region to a sample image. It performs effectively for the inpainting case of little prior information existing in the original image itself. Furthermore, sample image creation is added to reduce the complexity of inpainting via multiple images and avoid the taboo of complete duplication in art restoration. Moreover, Poisson blending is used for post-procedure to improve the visual harmony between the reconstructed fragment and the known region in both color and illumination. Last but not least, our method is successfully applied in the virtual inpainting of Dazu Buddhist face images. The inpainted proposals can be as a reference for the final actual artificial inpainting as well as a base for VR show.
This paper discusses the results of a user study on a mobile cultural heritage game, designed to stimulate reflection on a citys historical content and topics. Aided by geolocated technology, the game fosters serendipitous discovery of Points-Of-Interest, historical images and stories, whilst users traverse the city. The process of serendipitous exploration, which differs from the typical pre-calculated path recommendations used by other city apps, results in an immersive situated learning experience. It triggers reflection on the citys historic past in view of its present that is as unique as the visitors. This is one of the first studies to attempt an understanding of the effects of serendipitous urban discovery and historic reflection-triggering technologies on user experience. Through a detailed qualitative analysis, which considers both the perceived value of technology and the patterns and form of individual user responses, we identify four prevalent user types and their respective technological attitudes. The Enthusiast and the Interested users, felt that the technological elements of social recommendation and personalization, provided by the game, facilitated their movement and led them to experience feelings of empowerment because of the sense of accomplishment they acquired through situational learning. These user types also found that the interface interactions, designed to stimulate reflection, contributed to a feeling of connectedness and created a sense of social and public consciousness rooted in their user experience. We also observed that some of these users felt frustration because of the apps lack of instructional navigational directions. In contrast, the Cynical and the Sceptical type of responders were less tolerant of the perceived technological issues of the game, required more perfection in its functionality and overall design, and were less likely to adopt such applications at an early stage. The difference between the sceptic and the cynic was that the former saw potential in the app whereas the latter felt it had little or no value. The results of this study can contribute towards providing a grounded understanding of user experience and help progress the development of cultural heritage applications that aim to incorporate elements of reflection and/or place-based exploration into their functionalities.
During the last years, different types of digital museum interactives are being introduced in museums and other informal learning environments alongside more traditional interpretive media (guided tours, audio tours, hands-on workshops etc). An exponential number of studies has proved the potential of digitally mediated learning. However, there is still a lot of controversy as to the advantages and disadvantages of adopting and introducing digital museum interpretive media, primarily related with considerable risks and investment in terms of time, human and financial resources required for digital communication, museum education and learning. This work introduces a comprehensive evaluation framework, the D-COMP model, developed within the EU-meSch (material encounters with digital cultural heritage) project. One of the main ideas and unique features of the Evaluation Framework is that it examines, categorizes and scrutinizes different evaluation questions, issues and key-points related with the introduction and use of museum interactives from three different perspectives: the perspective of the CH professional, the perspective of the Cultural Heritage Institution and the perspective of the Museum Visitor. The framework benefited from an extensive review of the current state of the art and from inputs of cultural heritage professionals, designers and engineers. The D-COMP model can be used as a tool for reflection, before, during and after adopting and introducing novel digital media resources. The model covers the life-circle of digital interpretive media as diverse as mobile museum guides and applications, Augmented and Virtual Reality applications, edutainment applications, hands-on museum interactives, digitally mediated tangible and embodied experiences or web, on-line approaches used for museum education and learning.