Building Smarter: Designing Dashboards for Energy Management

[This post is based on Givannah Noor‘s Master Information Science Thesis]
Dashboards are essential for data-driven decisions, but effective design, especially for smart building and energy management, is often overlooked. Givannah Noor’s thesis, “Brick by Brick: A Human-Centered Approach to Effective Dashboard Design for Smart Building and Energy Management,” tackles this gap head-on.

At Arnhems Buiten, an IoT pilot site and part of the HEDGE-IoT project, a specialized dashboard was developed to monitor various devices. Givannah took a human-centered approach, conducting stakeholder interviews to uncover pain points and key needs. These insights informed the development of a high-fidelity prototype. To that end, she conducted several interviews with stakeholders, developed personas, scenarios and prototypes.

The overview page in the hi-fi prototype
The insights page in the hi-fi prototype

The prototype was then evaluated with stakeholders using a think-aloud technique. The findings revealed that effective dashboard design goes beyond typical UI/UX principles, emphasizing the crucial role of storytelling and designing for predictive analytics. This research offers valuable insights for creating dashboards that truly meet the complex needs of smart building and energy management, and improve how we visualize future trends.

Share This:

UI for Polyvocal Provenance Reporting

[This post is based on Bella Abelardo‘s Master Information Science thesis, “Designing a User Interface for Provenance Reporting of Objects with Colonial Heritage”]

Bella’s thesis addresses a critical challenge in cultural institutions: representing multiple perspectives for colonial heritage items. Current systems often create a “singular truth” in provenance reports, and unstructured data hinders discoverability.

Bella’s goal was to create a user interface to help provenance researchers holistically document the “polyvocal knowledge” often present in colonial heritage objects. Her research intended to explore improvements to the popular TMS content management system. To this end, she conducted interviews with various domain experts to gather design requirements and built a prototype, CultureSource.

two figures showing the lo-fi design of the improved user interface (imgs: B. Abelardo)

The evaluation showed CultureSource’s potential to help researchers document multiple perspectives. Bella’s research provides key requirements—standardization, multiple perspectives, usability, and data management—for future user interfaces aimed at documenting complex, multi-layered histories.

Share This:

Unlocking Smarter Customs: using Linked Data in Container Tracking

[This post is based on Auke Hofman’s Master Information Science thesis].
The Dutch Customs Administration handles an immense volume of data daily, primarily for risk assessment and critical safety, health, economy, and environment (VGEM) tasks. However, as Auke Hofman highlights in his Master Information Science thesis, “Opportunities and Challenges for Linked Data at Customs Administration of The Netherlands,” the current focus on declaration data, rather than real-time container events, creates a significant bottleneck, limiting transparency and effectiveness.

Auke’s research dives deep into how Customs can dramatically improve its risk assessment by shifting its attention to these crucial events. His main objective was to explore the opportunities and challenges of using Linked Data to enhance local container tracking. By integrating diverse data sources through Linked Data principles, he aimed to provide a more holistic view.

His methodology employed the Design Science Research Methodology (DSRM), iteratively developing and evaluating a Container Tracking System. He prioritized key requirements using the MoSCoW method, ensuring that the most pressing needs were addressed first. The evaluation itself was framed around user stories, offering practical use cases and demonstrating the system’s potential. Auke built a prototype featuring two knowledge graphs with visualizations, data analysis capabilities, and a notification system. One graph was manually created, while the other leveraged the FEDeRATED prototype, a system designed for real-time data exchange between stakeholders. The evaluation successfully demonstrated the prototype’s ability to retrieve data from the FEDeRATED knowledge graph and apply complex business rules. While some user interface features were deprioritized, the focus shifted to incorporating machine learning algorithms and providing architectural views, illustrating how this innovative prototype could be seamlessly integrated into Customs’ existing infrastructure.

Visualisation of the hand-constructed ontology

In conclusion, Auke Hofman’s thesis showcases, in a test environment, that Customs can significantly enrich container data by integrating it with other datasets using Linked Data principles. This not only allows for the application of sophisticated business rules but also paves the way for AI/ML-powered risk assessment capabilities such as anomaly detection and pattern extraction. His work emphasizes the transformative potential of Linked Data, while also acknowledging the essential need for manual effort in semantic data alignment before fully leveraging industry standards like FEDeRATED. This research marks a significant step towards a more intelligent and efficient Customs operation.

His thesis can be found below.

Share This:

Information Extraction and Knowledge Graph Creation from Handwritten Historical Documents

[This post is based on the Bachelor Project AI of Annriya Binoy]

In her bachelor thesis “Evaluating Methodologies for Information Extraction and Knowledge Graph Creation from Handwritten Historical Documents”, Annriya Binoy provides a systematic evaluation of various methodologies for extracting and structuring information from historical handwritten documents, with the goal of identifying the most effective strategies.

As a case study, the research investigates several methods on scanned pages from the National Archive of the Netherlands, specifically the service records and pension registers of the late 18th century and early 19th century of the Koninklijk Nederlands Indisch Leger (KNIL), see the example below. The task was defined as that of extracting birth events.


Four approaches are analyzed:

  1. Handwritten Text Recognition (HTR) using the Transkribus tool
  2. a combination of Large Language Models (LLM) and Regular Expressions (Regex),
  3. Regex alone
  4. Fuzzy Search

HTR and the LLM-Regex combination show strong performance and adaptability with F1 measure values of 0.88. While Regex alone delivers high accuracy, it lacks comprehensiveness. Fuzzy Search proves effective in handling transcription errors common in historical documents, offering a balance between accuracy and robustness. This research offers initial but practical solutions for the digitization and semantic enrichment of historical archives, and it also addresses the challenges of preserving contextual integrity when constructing knowledge graphs from extracted data.

More details can be found in Annriya’s thesis below.

Share This:

Exploring Culinary Links with NLP and Knowledge Graphs

[This post is based on Nour al Assali‘s bachelor AI thesis]

Nour’s research explores the use of Natural Language Processing (NLP) and Knowledge Graphs to investigate the historical connections and cultural exchanges within global cuisines. The thesis “Flavours of History: Exploring Historical and Cultural Connections Through Ingredient Analysis Using NLP and Knowledge Graphs” describes a method for analyzing ingredient usage patterns across various cuisines by processing a dataset of recipes. Its goal is to trace the diffusion and integration of ingredients into different culinary traditions. The primary aim is to establish a digital framework for addressing questions related to culinary history and cultural interactions.

The methodology involves applying NLP to preprocess recipe data, focusing on extracting and normalizing ingredient names. The pipeline contains steps for stop word removal, token- and lemmatization, character replacements etc.

With the results, a Knowledge Graph is constructed to map relationships between ingredients, recipes, and cuisines. The approach also includes visualizing these connections, with an interactive map and other tools designed to provide insights into the data and answer key research questions. The figure below shows a visualisation of top ingredients per cuisine.

Case studies on ingredients such as pistachios, tomatoes, basil, olives, and cardamom illustrate distinct usage patterns and origins. The findings reveal that certain ingredients—like pistachios, basil, and tomatoes—associated with specific regions have gained widespread international popularity, while others, such as olives and cardamom, maintain strong ties to their places of origin. This research underscores the influence of historical trade routes and cultural exchanges on contemporary culinary practices and offers a digital foundation for future investigations into culinary history and food culture.

The code and dataset used in this research are available on GitHub: https://github.com/Nour-alasali/BPAI. The complete thesis can be found below.

Share This:

Generating Synthetic Time-Series Data For Smart-Building Knowledge Graphs Using Generative Adversarial Networks

[This blog post is based on Jesse van Haaster‘s bachelor thesis Artificial Intelligence at VU]

Knowledge Graphs represent data as triples, connecting related data points. This form of representation is widely used for various applications, such as querying information and drawing inferences from data. For fine-tuning such applications, actual KGs are needed. However, in certain domains like medical records or smart home devices, creating large-scale public knowledge graphs is challenging due to privacy concerns. To address this, generating synthetic knowledge graph data that mimics the original while preserving privacy is highly beneficial.

Jesse’s thesis explored the feasibility of generating meaningful synthetic time series data for knowledge graphs. He specifically does this in the smart building / IoT domain, building on our previous work on IoT knowledge graphs, including OfficeGraph.

To this end, two existing generative adversarial networks (GANs), CTGAN and TimeGAN, are evaluated for their ability to produce synthetic data that retains key characteristics of the original OfficeGraph dataset. Jesse compared among other things the differences in distributions of values for key features, such as humidity, temperature and co2 levels, seen below.

Key value distributions for CTGAN-generated data vs original data
Key value distributions for TimeGAN-generated data vs original data

The experiment results indicate that while both models capture some important features, neither is able to replicate all of the original data’s properties. Further research is needed to develop a solution that fully meets the requirements for generating meaningful synthetic knowledge graph data.

More details can be found in Jesse’s thesis (found below) and his Github repository https://github.com/JaManJesse/SyntheticKnowledgeGraphGeneration

Share This:

Simulating creativity in GANs with IoT

[This blog post is based on the Artificial Intelligence MSc thesis project from Fay Beening, supervised by myself and Joost de Boo, more information can be found on Fay’s website]

Recently, generative art has been one of the fields where AI, especially deep learning has caught the public eye. Algorithms and online tools such as Dall-E are able to produce astounding results based on large artistic datasets. One class of algorithms that has been at the root of this success is the Generative Adversarial Network (GAN), frequently used in online art-generating tools because of their ability to produce realistic artefacts.

but, is this “””real””” art? is this “””real””” creativity?

To address this, Fay investigated current theories on art and art education and found that these imply that true human creativity can be split into three types: 1) combinational, 2) explorative and 3) transformative creativity but that it also requires real-world experiences and interactions with people and the environment. Therefore, Fay in her thesis proposes to combine the GAN with an Internet of Things (IoT) setup to make it behave more creative.

Arduin-based prototype (image from Fay’s thesis)

She then designed a system that extends the original GAN with an interactive IoT system (implemented in an Arduino-based prototype) to simulate a more creative process. The prototype of the design showed a successful implementation of creative behaviour that can react to the environment and gradually change the direction of the generated images.

Images shown to the participant during the level of creativity task. Images 2 and 6 are creative GAN generated images. Images 1 and 5 are human-made art. Images 3 and 4 are online GAN generated art.

The generated art was evaluated based on their creativity by doing task-based interviews with domain experts. The results show that the the level to which the generated images are considered to be creative depends heavily on the participant’s view of creativity.

Share This:

Thesis writing guidelines

As supervisor of many MSc and BSc theses, I find myself giving writing tips and guidelines quite often. Inspired by Jan van Gemert’s guidelines, I compiled my own document with tips and guidelines for writing an CS/AI/IS bachelor or master thesis. These are things that I personally care about and other lecturers might have different ideas. Also, this is by no means a complete list and I will use it as a living document. You can find it here: https://tinyurl.com/victorthesiswriting

Share This:

Student-supported project in the news

It was great to see that one of this year’s Digital Humanities in Practice projects lead to a conversation between the students in that project Helene Ayar and Edith Brooks, their external supervisors Willemien Sanders (UU) and Mari Wigham (NISV) and an advisor for another project André Krouwel (VU). That conversation resulted in original research and CLARIAH MediaSuite data story “‘Who’s speaking?’- Politicians and parties in the media during the Dutch election campaign 2021” where the content of news programmes was analysed for politicians’ names, their gender and party affiliation.

The results are very interesting and subsequently appeared on Dutch news site NOS.nl, showing that right-wing politicians are more represented on radio and tv: “Onderzoek: Rechts domineert de verkiezingscampagne op radio en tv“. Well done and congratulations!

Share This:

Digital Humanities in Practice 2020-2021

This year’s edition of the VU Digital Humanities in Practice course was of course a virtual one. In this course, students of the Minor Digital Humanities and Social Analytics put everything that they have learned in that minor in practice, tackling a real-world DH or Social Analytics challenge. As in previous years, this year we had wonderful projects provided and supervised by colleagues from various institutes. We had projects related to the Odissei and Clariah research infrastructures, projects supervised by KNAW-HUC, Stadsarchief Amsterdam, projects from Utrecht University, UvA, Leiden University and our own Vrije Universiteit. We had a project related to Kieskompas and even a project supervised by researchers from Bologna University. A wide variety of challenges, datasets and domains! We would like to thank all the supervisors and the students on making this course a success.

The compilation video below shows all the projects’ results. It combines 2-minute videos produced by each of the 10 student groups.

After a very nice virtual poster session, everybody got to vote on the Best Poster Award. The winners are group 3, whose video you can also see in the video above. Below we list all the projects and the external supervisors.

1Extracting named entities from Social Science data.ODISSEI project / VU CS – Ronald Siebes
2Gender bias data story in the Media SuiteCLARIAH project / UU / NISV –  Mari Wigham Willemien Sanders
3Food & SustainabilityKNAW-HUC –  Marieke van Erp
4Visualizing Political Opinion (kieskompas)Kieskompas – Andre Krouwel
5Kickstarting the HTR revolutionUU – Auke Rijpma
6Reconstructing the international crew and ships of the Dutch West India CompanyStadsarchief Amsterdam – Pauline van den Heuvel
7Enriching audiovisual encyclopediasNISV – Jesse de Vos
8Using Social Media to Uncover How Patients CopeLIACS Leiden – Anne Dirkson
9Covid-19 CommunitiesUvA – Julia Noordegraaf, Tobias Blanke, Leon van Wissen
10Visualizing named graphsUni Bologna – Marilena Daquino

Share This: