dance

Exploring Automatic Recognition of Labanotation Dance Scores

~ Victor ~ Leave a comment

[This post describes the research of Michelle de Böck and is based on her MSc Information Sciences thesis.]

Digitization of cultural heritage content allows for the digital archiving, analysis and other processing of that content. The practice of scanning and transcribing books, newspapers and images, 3d-scanning artworks or digitizing music has opened up this heritage for example for digital humanities research or even for creative computing. However, with respect to the performing arts, including theater and more specifically dance, digitization is a serious research challenge. Several dance notation schemes exist, with the most established one being Labanotation, developed in 1920 by Rudolf von Laban. Labanotation uses a vertical staff notation to record human movement in time with various symbols for limbs, head movement, types and directions of movements.

Generated variations of movements used for training the recognizers

Where for musical scores, good translations to digital formats exist (e.g. MIDI), for Lanabotation, these are lacking. While there are structured formats (LabanXML, MovementXML), the majority of content still only exists either in non-digitized form (on paper) or in scanned images. The research challenge of Michelle de Böck’s thesis therefore was to identify design features for a system capable of recognizing Labanotation from scanned images.

Examples of Labanotation files used in the evaluation of the system.

Michelle designed such a system and implemented this in MATLAB, focusing on a few movement symbols. Several approaches were developed and compared, including approaches using pre-trained neural networks for image recognition (AlexNet). This approach outperformed others, resulting in a classification accuracy of 78.4%. While we are still far from developing a full-fledged OCR system for Labanotation, this exploration has provided valuable insights into the feasibility and requirements of such a tool.

Share This:

Dancing and Semantics

~ Victor ~ Leave a comment

This post describes the MSc theses of Ana-Liza Tjon-a-Pauw and Josien Jansen. 

As a semantic web researcher, it is hard to sometimes not see ontologies and triples in aspects of my private life. In this case, through my contacts with dancers and choreographers, I have since a long time been interested in exploring knowledge representation for dance. After a few failed attempts to get a research project funded, I decided to let enthusiastic MSc. students have a go to continue with this exploration. This year, two Information Sciences students, Josien Jansen and Ana-Liza Tjon-a-Pauw, were willing to take up this challenge, with great success. With their background as dancers they did not only have the necessary background knowledge at but also access to dancers who could act as study and test subjects.

The questions of the two projects was therefore: 1) How can we model and represent dance in a sensible manner so that computers can make sense of choreographs and 2) How can we communicate those choreographies to the dancers?

Screenshot of the mobile choreography assistant prototype

Josien’s thesis addressed this first question. Investigating to what extent choreographers can be supported by semi-automatic analysis of choreographies through the generation of new creative choreography elements. She conducted an online questionnaire among 54 choreographers. The results show that a significant subgroup is willing to use an automatic choreography assistant in their creative process. She further identified requirements for such an assistant, including the semantic levels at which should operate and communicate with the end-users. The requirements are used for a design of a choreography assistant “Dancepiration”, which we implemented as a mobile application. The tool allows choreographers to enter (parts of) a choreography and uses multiple strategies for generating creative variations in three dance styles. Josien  evaluated the tool in a user study where we test a) random variations and b) variations based on semantic distance in a dance ontology. The results show that this latter variant is better received by participants. We furthermore identify many differences between the varying dance styles to what extent the assistant supports creativity.

Four participants during the 2nd user experiment. From left to right this shows variations presented through textual, 2D animation, 3D animation, and auditory instructions.

In her thesis, Ana-Liza dove deeper into the human-computer interaction side of the story. Where Josien had classical ballet and modern dance as background and focus, Ana-Liza looked at Dancehall and Hip-Hop dance styles. For her project, Ana-Liza developed four prototypes that could communicate pieces of computer-generated choreography to dancers through Textual Descriptions, 2-D Animations, 3-D Animations, and Audio Descriptions. Each of these presentation methods has its own advantages and disadvantages, so Ana-Liza made an extensive user survey with seven domain experts (dancers). Despite the relatively small group of users, there was a clear preference for the 3-D animations. Based on the results, Ana-Liza also designed an interactive choreography assistant (IDCAT).

The combined theses formed the basis of a scientific article on dance representation and communication that was accepted for publication in the renowned ACE entertainment conference, co-authored by us and co-supervisor Frank Nack.

You can find more information here:

Share This:

[Reading club] Dance in the World of Data and Objects

~ Victor ~ Leave a comment

This is a first post in a new series on VU Semantic Web reading club. During this weekly reading club we discuss a research paper related to Semantic Web, Human Computation or Computer Science in general. Every week, one group member selects and prepares a paper to discuss. This week it was my time and I chose a paper from 2013: “Dance in the World of Data and Objects” by Katerina El Raheb and Yannis Ioannidis (full citation and abstract below). The paper presents the need for (OWL) ontologies for dance representation. A quite nice slide deck supporting the paper is found here.

‘Dance’. CC-By (Teresa Alexander-Arab) 

Computer-interpretable knowledge representations for dance is something I have been thinking about for a while now. I am mostly interested in representations that actually match the conceptual level at which dancers and choreoraphers communicate and how these are related to low-level representations such as Labanotation. I am currently supervising two Msc students on this topic.

The paper by El Raheb and Ioannidis and our discussion afterwards outlined the potential use of such a formal representations for:

  1. Archiving dance and for retrieval. This is a more ‘traditional’ use of such representations in ICT for Cultural Heritage. An interesting effect of having this represented using standard semantic web languages is that we can connect deep representations of choreographers to highly heterogeneous knowledge about for example dance or musical styles, locations, recordings, emotions etc. An interesting direct connection could be to Albert Merono’s RDF midi representations.
  2. For dance analysis. By having large amounts of data in this representation, we can support Digital Humanities research. Both in more distant reading, but potentially also more close analysis of dance. Machine learning techniques could be of use herer.
  3. For creative support. Potentially very interesting is to investigate to what extent representations of dance can be used to support the creative process of dancers and choreographers. We can think of pattern-based adaptations of choreographies.

 

Citation: El Raheb K., Ioannidis Y. (2013) Dance in the World of Data and Objects. In: Nesi P., Santucci R. (eds) Information Technologies for Performing Arts, Media Access, and Entertainment. Lecture Notes in Computer Science, vol 7990. Springer, Berlin, Heidelberg

Abstract: In this paper, we discuss the challenges that we have faced and the solutions we have identified so far in our currently on-going effort to design and develop a Dance Information System for archiving traditional dance, one of the most significant realms of intangible cultural heritage. Our approach is based on Description Logics and aims at representing dance moves in a way that is both machine readable and human understandable to support semantic search and movement analysis. For this purpose, we are inspired by similar efforts on other cultural heritage artifacts and propose to use an ontology on dance moves (DanceOWL) that is based on the Labanotation concepts. We are thus able to represent dance movement as a synthesis of structures and sequences at different levels of conceptual abstraction, which serve the needs of different potential users, e.g., dance analysts, cultural anthropologists. We explain the rationale of this methodology, taking into account the state of the art and comparing it with similar efforts that are also in progress, outlining the similarities and differences in our respective objectives and perspectives. Finally, we describe the status of our effort and discuss the steps we intend to take next as we proceed towards the original goal.

Share This: