Hearing (Knowledge) Graphs

[This post is based on Enya Nieland‘s Msc Thesis “Generating Earcons from Knowledge Graphs” ]

Three earcons with varying pitch, rythm and both pitch and rythm

Knowledge Graphs are becoming enormously popular, which means that users interacting with such complex networks are diversifying. This requires new and innovative ways of interacting. Several methods for visualizing, summarizing or exploring knowledge have been proposed and developed. In this student project we investigated the potential for interacting with knowledge graphs through a different modality: sound.

The research focused on the question how to generate meaningful sound or music from (knowledge) graphs. The generated sounds should provide users some insights into the properties of the network. Enya framed this challenge with the idea of “earcons” the auditory version of an icon.

Enya eventually developed a method that automatically produces these types of earcon for random knowledge graphs. Each earcon consist of three notes that differ in pitch and duration. As example, listen to the three earcons which are shown in the figure on the left.

Earcon where pitch varies
Earcon where note duration varies
Earcon where both pitch and rythm vary

The earcon parameters are derived from network metrics such as minimum, maximum and average indegree or outdegree. A tool with user interface allowed users to design the earcons based on these metrics.

The pipeline for creating earcons
The GUI

The different variants were evaluated in an extensive user test of 30 respondents to find out which variants were the most informative. The results show that indeed, the individual elements of earcons can provide insights into these metrics, but that combining them is confusing to the listener. In this case, simpler is better.

Using this tool could be an addition to a tool such as LOD Laundromat to provide an instant insight into the complexity of KGs. It could additionally benefit people who are visually impaired and want to get an insight into the complexity of Knowledge Graphs

Share This:

A look back at UCDS at ICT.Open2018

Two weeks ago, ICT.Open2018 was held in Amersfoort. This event brings together Computer Science researchers from all over the Netherlands and our research group was present with many posters and presentations.

We even won a prize! (Well, a 2nd place prize, but awesome nonetheless). Xander Wilcke presented work on using Knowledge Graphs for Machine Learning. He was awarded the runner-up prize for best poster presentation at ICTOpen2018. Congrats!

 

Ronald Siebes presented work in the ArchiMediaL project on reconstructing 4D street views from historical images.

Oana Inel presented her work on Named Entity Recognition and Gold Standard critiquing. She also demonstrated the Clariah MediaSuite.

Anca Dumitrache talked about using crowdsourcing as part of the Machine Learning life cycle.

Tobias Kuhn talked about Reliable Granular References to Changing Linked Data, which was previously published at ISWC2017.

Cristina Bucur introduced  Linkflows: enabling a web of linked semantic publishing workflows

I talked myself a bit about current work in the ABC-Kb Network Institute project

All in all, this was quite a nice edition of the yearly event for our group. See you next year in Amersfoort!

Share This: