ACM Journal on

Computing and Cultural Heritage (JOCCH)

Latest Articles

A Probabilistic, Ontological Framework for Safeguarding the Intangible Cultural Heritage

In this article, we propose Multi-Entity Bayesian Networks (MEBNs) as the probabilistic ontological framework for the analysis of the Tsamiko and... (more)

Generating Islamic Quasi-Periodic Patterns: A New Method

The discovery of quasi-crystals has led to a great debate about their unusual structure. The big surprise is that these structures were found in Islamic art several centuries ago. This latest discovery drew the attention of scientists to propose several approaches for the comprehension of these structures by analyzing several quasi-periodic... (more)

Saphir: Optical Playback of Damaged and Delaminated Analogue Audio Disc Records

The goal of optical playback of analogue audio discs records has been pursued since at least 1929. Several different approaches have been demonstrated to work. But in most cases the playback quality is worse than using mechanical playback. The Saphir process uses a specifically designed colour illuminator that exploits the reflective properties of... (more)

Finding Parallel Passages in Cultural Heritage Archives

It is of great interest to researchers and scholars in many disciplines (particularly those working on cultural heritage projects) to study parallel... (more)

Mapping History: Orienting Museum Visitors across Time and Space

At historic open-air museums, many of the “objects” under investigation are buildings and landscapes that could tell multiple, overlapping narratives: i.e., they were built/manipulated over the course of years by different peoples and groups who used them for varying purposes. In this article, we address this challenge by... (more)


Franco Niccolucci has been appoitned as the new Editor-in-Chief for JOCCH. For more information see here.

Good news from Scopus - CiteScore 2017 figure has been released
The CiteScore 2017, powered by Scopus, has just been released and the JOCCH 2017 CiteScores is 1.62! Considering that this indicator was 0.85 in 2016, JOCCH has really improved!

Thanks to all JOCCH authors and reviewers who contributed to this success.

Text of older news is available on the Announcements page.


Forthcoming Articles

A New Projection-based Exhibition System for a Museum

Projection-based exhibition methods have been used in a museum to create digital contents of cultural objects. They can enrich the exhibition of a cultural heritage object with physically co-located digital contents, and multiple users can enjoy the projected contents without the aid of additional devices. But the quality of projection is often restricted by surrounding environment such as the influence of the ambient light and occlusions by obstacles. The degree of freedom of projected contents also is limited due to the usage of a static projection surface. In this paper, we propose a novel projection-based exhibition system that resolves these shortcomings. We introduce a new design by combining the multi projection mapping with an optical see-through display. It provides high-quality projection contents robust to the influence of the ambient light and to the problem of obstacles. We also introduce a mechanically moving projection surface to provide a dynamic projection content by changing its shape and appearance. Our prototype system demonstrates applications that show a realistic three-dimensional effect and photo-realistic appearance of a cultural object.

From point cloud data to structural analysis through a geometrical hBIM-oriented model.

The assessment of the structural behavior of historic masonry structures like gothic cathedrals is an important engineering and architectural issue, because of the economic and cultural relevance of such buildings. In this paper, we present a complete numerical methodology for point clouds processing, geometrical and parametric 3D modeling, and finite element structural analysis of the spire of the cathedral of Senlis, France. Our work highlights the particular difficulties linked with digitization and geometrical modeling of highly complex Gothic structures, as well as the need to find compromises between quality and accuracy of extracted data used for geometrical modeling and structural analysis. The methodology enables the semi-automatic transformation of a three-dimensional points cloud, surveyed through terrestrial laser scanner, into a three-dimensional geometrical BIM-oriented model, and its use to propose a consistent 3D finite element mesh suitable for advanced structural analysis. A full software chain was integrated in the proposed numerical process, so as to use the most important data contained in the real geometry and accurately transposed in the point clouds. After a successful data processing step with 3DReshaper software that proved to be necessary for enhancement of point clouds, a semi-automated geometrical BIM-oriented modeling step with Rhinoceros5 software and VisualARQ plugin has allowed the construction of a hybrid model by reverse engineering from the point clouds. This 3D model, containing both geometrical and parametric data of the structure, has been exported to the Hyperworks suite for finite element structural analysis under self-weight. Our computations focused on the estimation of the structure deformation and on the distribution of compression and traction stresses in all components of the complex structure. It is found that the spire is safe. Based on reliable and properly detailed results, our study provides significant information for understanding the behavior of the structure and potential damage monitoring.

Augmented experiences in cultural spaces through social participation

Technologies such as the internet of the things and social computing offer promising opportunities to envision augmented experiences in a wide variety of cultural spaces. Such interactive technologies act as a hook to attract visitors attention, but if they do not provide opportunities to engage in a more personal and meaningful way they can be soon discarded. Augmented experiences need to be intrinsically motivating, connecting with values and expectations of visitors and institutions. In a world that is becoming more a more connected and moving towards a participatory culture, the need to participate actively to coproduce knowledge and meaning can be exploited as a powerful intrinsic motivator. In this paper we describe how the Social Display Environment (SDE) promotes participation in cultural spaces whilst keeping the physical connection with the real objects exhibited. The SDE was exhibited in a cultural center over a weekend to investigate different dynamics in terms of peoples interaction styles, behaviors, and perceptions. The prototype not also was assessed as easy to learn and useful, but also promoted a number of social interactions around the objects involved in the exhibition.

CHER-Ob: A Tool for Shared Analysis and Video Dissemination

The study of cultural heritage involves many different activities, including visualizing digital data, analyzing information and sharing results. Current technologies focus on providing better tools for data representation and processing, neglecting the importance of analysis and sharing. In this paper, we present a software system, CHER-Ob, which offers powerful tools for evaluating and publishing the results of cultural heritage research. CHER-Ob provides the capability to visualize and add various types of annotations to data in a wide variety of formats. These annotations assist in the analysis phase, and are used for sharing the results of a study. A written report can be generated and automatically illustrated using the annotations. In addition, an ``animation scheme" is associated with each type of annotation. The schemes make it possible to generate an introductory video overview of an analysis by selecting preferences and annotations. A series of animated sequences of 2D and 3D objects will appear in a user-specified order in a video subtitled by annotations. The system is useful for integrating cultural and digital resources as well as for providing a method to author materials for disseminating cultural heritage findings to the public.

Egocentric Visitors Localization in Cultural Sites

We consider the problem of localizing visitors in a cultural site from egocentric (first person) images. Localization information can be useful both to assist the user during his visit (e.g., by suggesting where to go and what to see next) and to provide behavioral information to the manager of the cultural site (e.g., how much time has been spent by visitors at a given location? What has been liked most?). To tackle the problem, we collected a large dataset of egocentric videos using two cameras: a head-mounted HoloLens device and a chest-mounted GoPro. Each frame has been labeled according to the location of the visitor and to what he was looking at. The dataset is freely available in order to encourage research in this domain. The dataset is complemented with baseline experiments performed considering a state-of-the-art method for location-based temporal segmentation of egocentric videos. Experiments show that compelling results can be achieved to extract useful information for both the visitor and the site-manager.

Digital Editions of Text: Surveying User Requirements in the Digital Humanities

This paper describes the first analysis of the differences between what the Digital Humanities community who build online digital editions of texts provides, and what the user-base wants. It therefore provides useful guidance for those working on these projects and will also be of interest to those who commission, fund, and support digital editions of textual material, including publishers, universities, libraries and archives.

Improved Hieroglyph Representation for Image Retrieval

The most successful approach for hieroglyph representation for retrieval starts thinning the hieroglyph contour lines. Then, a set of interest points from the thinned hieroglyph is randomly selected, and a local descriptor from each selected interest point is computed. These local descriptors are used under the Bag of Visual Words (BoVW) model for performing hieroglyph retrieval. This approach has as drawback that a random selection of a subset of interest points does not guarantee suitably preserving the most useful information of a hieroglyph. Additionally, during the thinning process, contour shape distortions could lead to unwanted branches, which do not represent important information and could affect the quality of the local descriptors. Therefore, in this paper, we propose improving the hieroglyph representation quality by pruning unwanted branches over the thinned contour of a hieroglyph and introducing an improved interest point selection process. Our experiments show that our proposal allows significantly improving the image retrieval results previously reported in the literature.

Wedge cameras for minimally invasive archaeology

Acquiring images of archaeological artifacts is an essential step for the study and preservation of cultural heritage. In constrained environments, traditional acquisition techniques may fail or be too invasive. We present an optical device including a camera and a wedge waveguide that is optimized for imaging within confined spaces in archeology. The major idea is to redirect light by total internal reflection to circumvent the lack of room, and to compute the final image from the raw data. We tested various applications onsite in autumn 2017 during an archaeological mission in Medamoud (Egypt). Our device was able to successfully record images of the underground from slim trenches of about 15 cm wide, including underwater trenches, and between rocks composing a wall temple. Experts agreed that the acquired images were good enough to get useful information that cannot be obtained as easily with traditional techniques.

Do Game Designers' Decisions Related to Visual Activities Affect Knowledge Acquisition in Cultural Heritage Games? An Evaluation from a Human Cognitive Processing Perspective

In the cultural heritage domain, games have been used to engage users into an active state of learning through immersive and playful interactions that include visually enriched interaction contexts. There is evidence that individual differences in the inherent way people search, process, analyze, comprehend, store, and retrieve visual information in their surrounding environment are reflected on their performance, experience, effectiveness and efficiency in such environments. Despite that cultural heritage game designers favor learning experiences in such contexts, current design practices of cultural heritage games barely consider the gamers individual differences in visual information processing. This can be accredited in that there is a deficiency in understanding and modelling the effects among users visual behavior, gameplay behavior, and cognitive styles in cultural heritage games towards knowledge acquisition; resulting in insufficient methods of creating cognition-centered user models and considering such human cognitive factors practically, within current state-of-the-art design approaches. We selected three known cultural heritage games, adopted a credible cognitive style theory and performed, over a six-month period, three separate user studies (N=127) following a between-subject, eye-tracking based, experimental design. The study results revealed that game designers decisions, related to visual exploration and search, unintentionally favored users belonging to specific cognitive style groups by influencing their visual and consequently gameplay behavior, resulting in differences in knowledge acquisition. Findings also revealed correlation effects, during game play, among individual differences in visual information processing, users visual behavior strategies and gameplay behavior, which provide useful insights for practitioners and researchers about the technical feasibility to develop real-time cognitive-centered user models based on users eye-gaze patterns, aiming to scaffold the development of personalized solutions adjusted to users individual cognitive characteristics. Practical implications related to an eye-gaze driven cognitive-centered personalization framework for cultural-heritage games are also discussed in this paper.

Digital Methods in Intangible Cultural Heritage Research: A Case Study in Tango Argentino

With this paper we present the ongoing research project Tango Danceability of Music in European Perspective and the transdisciplinary research design it is built upon. Three main aspects of tango argentino are in focusthe music, the dance, and the people in order to understand what is considered danceable in tango music. The study of all three parts involves computer-aided analysis approaches, and the results are examined within ethnochoreological and ethnomusicological frameworks. Two approaches are illustrated in detail to show initial results of the research model. Network analysis based on the collection of online tango event data and quantitative evaluation of data gathered by an online survey showed significant results, corroborating the hypothesis of gatekeeping effects in the shaping of musical preferences. The experiment design includes incorporation of motion capture technology into dance research. We demonstrate certain advantages of transdisciplinary approaches in the study of Intangible Cultural Heritage, in contrast to conventional studies based on methods from just one academic discipline.

How to Tell Ancient Signs Apart? Recognizing and Visualizing Maya Glyphs with CNNs

Thanks to the digital preservation of cultural heritage material, multimedia tools, e.g. based on automatic visual processing, enable to considerably ease the work of scholars in the humanities and help them to perform quantitative analysis of their data. In this context, this paper assesses three different Convolutional Neural Network (CNN) architectures along with three learning approaches to train them for hieroglyph classification, which is a very challenging task due to the limited availability of segmented ancient Maya glyphs. More precisely, the first approach, the baseline, relies on pretrained networks as feature extractor. The second one investigates a transfer learning method by fine-tuning a pretrained network for our glyph classification task. The third approach considers directly training networks from scratch with our glyph data. The merits of three different network architectures are compared: a generic sequential model (i.e. LeNet), a sketch-specific sequential network (i.e. Sketch-a-Net), and the recent Residual Networks. The sketch-specific model trained from scratch outperforms other models and training strategies. Even for a challenging 150-class classification task, this model achieves 70.3% average accuracy and proves itself promising in case of small amount of cultural heritage shape data. Furthermore, we visualize the discriminative parts of glyphs with the recent Grad-CAM method, and demonstrate that the discriminative parts learned by the model agrees in general with the expert annotation of the glyph specificity (diagnostic features). Finally, as a step towards systematic evaluation of these visualizations, we conduct a perceptual crowdsourcing study. Specifically, we analyze the interpretability of the representations from Sketch-a-Net and ResNet-50. Overall, our paper takes two important steps towards providing tools to scholars in the digital humanities: increased performance for automation, and improved interpretability of algorithms.

Inpainting of Dunhuang Murals by Sparsely Modelling the Texture Similarity and Structure Continuity

A new Infrared True-Color approach for visible-infrared multispectral image analysis

In this paper, we present a new method for the analysis of Visible/Infrared multispectral sets producing chromatically faithful false-color images, which maintain a good readability of the information contained in the non-visible Infrared band. Examples of the application of this technique are given on the multispectral images acquired on the Pietà of Santa Croce of Agnolo Bronzino (1569, Florence) and on the analysis and visualization of the multispectral data obtained on Etruscan mural paintings (Tomb of the Monkey, Siena, Italy, V century B.C.). The fidelity of the chromatic appearance of the resulting images, coupled to the effective visualization of the information contained in the Infrared band, opens interesting perspectives for the use of the method for visualization and presentation of the results of multispectral analysis in Cultural Heritage diffusion, research and diagnostics.

Segmentation, Reconstruction, and Visualization of Ancient Inscriptions in 2.5D

This article presents a new algorithm for the automated reconstruction and visualization of damaged ancient inscriptions. After reviewing current methods for enhancing incisions, a hybrid approach is adopted that combines advantages of 2D and 3D analytical techniques. A photogrammetric point cloud of an inscription is projected orthographically from an ideal vantage point, generating a 2.5D raster including channels describing depth and surface derivatives. Next, the obstacles to legibility posed by breaks in an ancient text are considered, leading to the creation of a new segmentation algorithm based on SLIC superpixels and region-merging that operates on the geometry channels of the inscribed surface, rather than color or intensity values. With high accuracy, the algorithm classifies surface points by their likelihood of belonging to the uninscribed original plane, deliberate strokes, or breaks. Conventions for static visualization are developed for epigraphical analysis and publication. Three case studies demonstrate the power and flexibility of this method, which has resulted in substantial changes to IG XIV 1, an early Greek text whose reading has been debated for more than 150 years.

Semi-automated cleaning of laser scanning campaigns with machine learning

Terrestrial laser scanning campaigns provide an important means to document the 3D structure of historical sites. Unfortunately, the process of converting the 3D point clouds acquired by the laser scanner into a coherent and accurate 3D model has many stages and is not generally automated. In particular, the initial cleaning stage of the pipeline  in which undesired scene points are deleted  remains largely manual and is usually labour intensive. In this paper we introduce a semi-automated cleaning approach which incrementally trains a random forest (RF) classifier on an initial keep/discard point labelling generated by the user when cleaning the first scan(s). The classifier is then used to predict the labelling of the next scan in the sequence. Before this classification is presented to the user, a denoising post-process, based on the 2D range map representation of the laser scan, is applied. This significantly reduces small isolated point clusters, which the user would otherwise have to fix. The user then selects the remaining incorrectly labelled points, and these are weighted, based on a confidence estimate, and fed back into the classifier to retrain it for the next scan. Our experiments, across 4 scanning campaigns, show that when the scan campaign is coherent i.e. it does not contain widely disparate or contradictory data, the classifier yields a keep/discard labelling which typically ranges between 95-99%. This is somewhat surprising, given that the data in each class can represent many object types, such as tree, person, wall etc, and that no further effort beyond the point labeling of keep/discard is required of the user. An informal timing experiment over a 15 scan campaign, to compare the cleaning times produced by our software against those of an experienced user, showed that we were able to produce a result at 98% (average) accuracy 20 minutes sooner than the cumulative expert cleaning time, even with a non optimized code.

Communicating Museum Collections Information Online: Analysis of the Philosophy of Communication extending the Constructivist Approach

Cultural heritage institutions are spending considerable effort and resources in order to provide online access to their collection catalogues and collection management systems, usually through their institutional websites. This improves accessibility and supports research and engagement by diverse user groups, as well as meeting the increasing expectation by audiences that this type of information will be freely and easily available online. However, cultural organisations have not responded to these needs in the same way and have been employing different web tools and features to present their collections online. This technological implementation as well as the type of content provided reflects also the philosophy of communication of the institution itself. The paper describes the research carried out studying the provision of collections information online by different types of museums. The research, whose first phase started initially in 2007, involved studying a large number of museum collection catalogues on the web and comparing them with those in 2017 in order to establish categories of presentation types and analyse their corresponding philosophy of communication, as well as compare the changes over time.

Exploring RGB-D cameras for 3D reconstruction of cultural heritage: a new approach applied to Brazilian baroque sculptures

In this work we explore the use of RGB-D cameras for digital preservation of cultural heritage. Three-dimensional (3D) digital preservation is usually performed with the use of three dimensional scanners, as the 3D points generated by these equipments are in average less than ten micrometers away from the real position. The downside of 3D scanners, in addition to the high cost, is the infrastructure requirements, for instance, own source of energy, large workspace with tripods, special training to calibrate and operate the equipment and the acquisition time, potentially taking several minutes for capturing a single image. An alternative is the use of low cost depth cameras that are easy to operate and only require connection to a laptop. There are several recent studies showing the potential of RGB-D sensors, however they often exhibit errors when applied to a full 360 degrees 3D reconstruction setup, known as the loop closure problem. This kind of error accumulation is intensified by the lower accuracy and large volume of data generated by RGB-D cameras. We propose a complete methodology for 3D reconstruction based on RGB-D sensors. In order to mitigate the loop closure effect, we developed a pairwise alignment method. Our approach expands the connectivity graph connections in a pairwise alignment system by automatically discovering new pairs of meshes with overlapping regions. Then we distribute the alignment error more evenly over the aligned pairs, avoiding the loop closure problem of full 3D reconstructions. The experiments were performed on a collection of 30 artworks made by the Baroque artist Antonio Francisco Lisboa, known as Aleijadinho, as part of the Aleijadinho Digital project conducted in partnership with IPHAN (Brazilian National Institute for Cultural and Artistic Heritage) and United Nations Educational, Scientific and Cultural Organization (UNESCO). We present results showing 3D models that are favorably compared to state-of-the-art methods available in the literature using RGD-D sensors. Our main contributions are: A new method for 3D alignment dedicated to solve the RGB-D camera loop closure problem; the development of practical solutions and 3D digital models of an important and challenging collection of Brazilian cultural heritage.

Did we just travel to the past? Building and evaluating with Cultural Presence different modes of VR-mediated experiences in Virtual Archaeology

The last years, Virtual Archaeology has started to introduce more experiential elements in virtual reconstructions, therefore going beyond the traditional visualization of 3D architectural models. In the case of dissemination, these experiences are equated with a trip in time, thanks to which users witness what the past was like and learn about it. However, due to a lack of explicit theoretical frameworks and/or systematic evaluation focusing on such experiential elements, it is uncertain whether the intended goals are achieved and why. Based on a novel theoretical framework arising from the concept of Cultural Presence, this paper will investigate if and how current virtual environments achieve the feeling traveling to the past. To that end, six different virtual reconstructions of the Neolithic site of Çatalhöyük (Turkey) were built and evaluated in a between-subjects experiment. The results support the role of content meaningfulness, responsive characters, enhanced interaction, and multi-sensory realism in the achievement of successful VR-mediated experiences.

Computer-assisted Orientation and Drawing of Archaeological Pottery

Archaeologists spend considerable time orientating and drawing ceramic fragments by hand for documentation, to infer their manufacture, the nature of the discovery site and its chronology, and to develop hypotheses about commercial and cultural exchanges, social organisation, resource exploitation and taphonomic processes. This study presents an R/Shiny application to automate this time-consuming orientation and drawing of pottery fragments. Orientation is based on the 3D geometry of pottery models, acquired in minutes with low-cost 3D scanners. Several methods (using normal vectors, or circle fittings, or profile fittings) are applied to determine the optimal position of the rotation axis. The profile and contours of the fragment, as well as any possible decoration, can then be depicted in various ways: photorealistic rendering or dotted patterns, calculated by ambient occlusion, combined or not with artificial light. The general workflow, evaluated using both synthetic and real-world fragments, appears to be rapid, accurate, and reproducible. It drastically reduces the amount of routine work required to document ceramic artefacts. The information produced, together with the 3D representation of the fragments, can easily be archived and/or exchanged within the archaeological community for further research. The source code built for the R environment (together with an installation notice and examples) is freely downloadable. To the best of our knowledge, it is the first operational tool available for such a purpose.

Evaluating digital cultural heritage 'in the wild': the case for reflexivity

Digital heritage interpretation is oftentimes untethered from traditional museological techniques and environments. As museums and heritage sites explore the potentials of locative technologies and ever more sophisticated content triggering mechanisms for use outdoors, the kinds of questions digital heritage researchers are able to explore have complexified. Researchers now find themselves in the realm of the immersive, the experiential, the performative and the ungovernable. Working closely with their research participants, they navigate ambiguous terrain including the often unpredictable affective resonances that are the direct consequences of interaction. This article creates a dialogue between two case studies which taken together help to unpack some key methodological and ethical questions emerging from these developments. Firstly, we introduce With New Eyes I See, an itinerant and immersive digital heritage encounter which collapsed boundaries between physical/digital, fact/fiction and past/present. Secondly, we detail Rock Art on Mobile Phones, a set of dialogic web apps that aimed to explore the potential of mobile devices in delivering heritage interpretation in the rural outdoors. Looking outward from these case studies we demonstrate how traditional evaluative methodologies are being stretched and strained given the kinds of questions digital heritage researchers are now exploring. Drawing on vignettes from experience-oriented qualitative studies with participants we articulate specific common evaluative challenges related to the embodied, multimodal and transmedial nature of the digital heritage experiences under investigation. In looking at the larger picture, we further highlight the need for nuanced, agile and reflexive approaches in this field going forward, which acknowledge the generative role of evaluation in processes of heritage meaning making.


This is the editorial to Volume 11 Issue 4

All ACM Journals | See Full Journal Index

Search JOCCH
enter search term and/or author name