mugshotI am an assistant professor (UD) at the Web & Media group at the Computer Science department of the Vrije Universiteit Amsterdam (VU). I am also a senior research fellow at Netherlands Institute for Sound and Vision. In my research, I combine (Semantic) Web technologies with Human-Computer Interaction, Knowledge Representation and Information Extraction to tackle research challenges in various domains. These include Cultural Heritage, Digital Humanities and ICT for Development (ICT4D). More information on these projects can be found on this site or through my CV .

Big Data Europe Youtube channel

For those curious about the Big Data Europe technology stack and who rather view videos than read descriptions and documentation, we have started a youtube video channel where BDE researchers explain the how, why and what of the BDE stack. Embedded below is a short clip of Hajira Jabeen explaining how BDE enables someone to get started with Big Data. More clips are available on the channel.

Share This:

ICT4D 2017 promo video

As a teaser for our upcoming ICT4D students. Have a look at this nice video that André Baart made

Share This:

Speech technology and colorization for audiovisual archives

[This post describes and is based on Rudy Marsman‘s MSc thesis and is partly based on a Dutch blog post by him]

The Netherlands Institute for Sound and Vision (NISV) archives Dutch broadcast TV and makes it available to researchers, professionals and the general public. One subset are the Polygoonjournaals (Public News broadcasts) that are published under open licenses as part of the OpenImages platform. NISV is also interested in exploring new ways and technologies to make interaction with the material easier and to increase exposure to their archives. In this context, Rudy explored two options.

Two stills from the film ‘Steegjes‘, with the right frame colorized. Source: Polygoon-Profilti (producent) / Nederlands Instituut voor Beeld en Geluid  / colorized by Rudy Marsman, CC BY-SA

One part of the research was the autonomous colorization of old black-and-white video footage using Neural Networks. Rudy used a pre-trained NN (Zhang et al 2016) that is able to colorize black and white images. Rudy developed a program to split videos into frames, colorize the individual frames using the NN and then ‘stitch’ them back together into colorized videos. The stunning results were very well received by NISV employees. Examples are shown below.


Tour de France 1954 (colorized by Rudy Marsman in 2016), Polygoon-Profilti (producent) / Nederlands Instituut voor Beeld en Geluid (beheerder), CC-BY SA

Results from the comparison of the different variants of the method on different corpora
Results from the comparison of the different variants of the method on different corpora

In the other part of his research, Rudy investigated to what extent the existing news broadcast corpus, with a voice-overs from the famous Philip Bloemendal  can be used to develop a modern text-to-speech engine with his voice. To do so he have mainly focused on natural language processing and the determination to what extent the language used by Bloemendal in the 1970s is still comparable enough to contemporary Dutch.

Rudy used precompiled automatic speech recognition (ASR) results to match words to sounds and developed a slot-and-filler text-to-speech system based on this. To increase the limited vocabulary, he implemented a number of strategies, including term-expansion through the use of Open Dutch Wordnet and smart decompounding (this mostly works for Dutch, mapping ‘sinterklaasoptocht’ to ‘sinterklaas’ and ‘optocht’. The different strategies were compared to a baseline. Rudy found that a combination of the two resulted in the best performance (see figure). For more information:

Share This:

ArchiMediaL proposal granted by Volkswagen Stiftung

Volkswagen stiftung letterI received a good news letter from Volkswagen Stiftung who decided to award us a research grant for a 3-year Digital Humanities project named “ArchiMediaL” around architectural history. This project will be a collaboration between  architecture historians from TU Delft,  computer scientists from TU Delft and VU-Web and Media. A number of German scholars will also be involved as domain experts. The project will combine image analysis software with crowdsourcing and semantic linking to create networks of visual resources which will foster understanding of understudied areas in architectural history.
From the proposal:In the mind of the expert or everyday user, the project detaches the digital images from its existence as a single artifact and includes it into a global network of visual sources, without disconnecting it from its provenance. The project that expands the framework of hermeneutic analysis through a quantitative reference system, in which discipline-specific canons and limitations are questions. For the dialogue between the history of architecture and urban form this means a careful balancing of qualitative and quantitative information and of negotiating new methodological approaches for future investigation.

Share This:

A Look Back at the 2nd BDE Workshop on Big Data in Health, Demographic Change and Wellbeing

[reblogged from Big-Data-Europe.eu]

On 9 December 2016, the second workshop for the Big Data Europe Health, Demographic Change and Wellbeing societal challenge was held in Brussels. The aim of this workshop was to highlight progress from the BigDataEurope project in building the foundations of a generically applicable big data platform which can be applied across all Horizon 2020 societal challenges. This workshop specifically focused on health, and showcased our first pilot’s application to early bioscience research data.

The workshop in full effect

The workshop had 15 participants, from within the health domain and outside it, including many participants from the European Commission. Together we discussed different perspectives on how we may use appropriate H2020 instruments and work programmes to better integrate the ecosystem of linked data repositories, data management services and virtual collaboration environments to increase the pace of knowledge sharing in health.

The workshop featured presentations from BDE’s Simon Scerri and Aad Versteden on the general goals and progress of the BigDataEurope project and the BDE infrastructure respectively. After lunch, Ronald Siebes (BDE / VU Amsterdam) presented the first pilot in this specific domain. More information on that pilot can be found here. An extensive round-table discussion followed, in which possible options for new applications and connections were considered.

Snapshot of the SC1 pilot interface, as presented by Ronald Siebes

One question raised was whether the generic BDE infrastructure can be used by European SMEs. The fact that the BDE infrastructure is completely Open Source, very easy to install and features intuitive interface components makes re-use relatively simple even for smaller institutions and companies.

A significant part of the discussion focussed on possible new use cases for expanding the scope of the pilot. One suggestion was to look at post-hoc integration of clinical data, which represents a typical problem of data ‘variance’. This would require integrating information from different versions of medical questionnaires, which may be recorded or stored in different ways. Data provenance is also a key concern, as keeping a trail of what has happened to clinical data is crucial to tracking patients’ histories. Once integrated, this data could then be mined to identify biases or data patterns.

Finally, the workshop participants discussed potential connections to other European projects. Here many projects were mentioned including the MIDAS project, the Big-O project on childhood obesity, the PULSE projects and IMI / IMI2 projects including EMIF. We will be seeking collaborations with these projects and will continue to develop new and interesting Big Data use cases in this domain in the coming year.

More images can be found below: BDE Health Workshop SC 1.2

Share This:

Web of Voices and W4RA video at the Webscience@10 TV Channel

For its 10th anniversary, the Web Science Trust organized an event Webscience@10. For this event, a Webscience@10 TV channel was launched to showcase different research and education initatives around the world. On behalf of the VU Network Institute and W4RA, we submitted our Web of Voices video as well as a short introduction to the W4RA team.

You can watch the ~10 hours of video content at  http://www.webscience.org/webscience10/tv-channel-webscience10/. You can find us (listed under Netwerk Institute Amsterdam) at 2h31mins:

Share This:

Niels’ paper awarded first Bob Wielinga award at EKAW

Niels Ockeloen’s paper on Data2Documents was awarded the first Bob Wielinga memorial award for best research paper at the 20th International Conference on Knowledge Engineering and Knowledge Management (EKAW2016). “Data 2 Documents: Modular and distributive content management in RDF” was authored by Niels Ockeloen, Victor de Boer, Tobias Kuhn and Guus Schreiber from the Web and Media group.. The paper describes Niels’ PhD. work on a method for creating human readable web documents out of machine readable Linked Data, focussing on modularity and re-use. You can view the slides for Niels’ presentation slides here

Niels wins Best Paper Award

The award is named after Prof. Bob Wielinga, one of the most prominent European scientists in the area of knowledge-based systems, best known for his work on the KADS methodology, who has been one of the key influences on the development of the area in the past three decades. Bob was both my own and Guus Schreiber’s promotor so this makes it extra-special for us. In 2009 he was also appointed at our department, where he continued supervising PhD students until he passed away earlier this year. It is especially nice that the award, which was named after Bob Wielinga goes to work that is not only authored by people from Amsterdam but also work that Bob at some point discussed with Niels in the Basket, before his passing.

Share This:

W4RA research displayed in Museon

In modern day research, dissemination is key and it is therefore always nice to see research results being shared with the public in new and unforseen ways. Our work within the Web for Regreening in Africa (W4RA) is now part of a exhibition in the Museon museum in the Hague. The exhibition focuses on the United Nations’ 17 Sustainable Development Goals (SDGs) for 2030. As content partner of the Museon, the W4RA programme of VU has contributed ideas, visuals and texts for the exposition related to SDG No. 15, entitled “Plant in het Zand” (Plant in the sand).museon exhibition images

From the press release: Land degradation and desertification are increasing due to both natural and human causes, including climate change and population pressures. Areas can no longer meet the needs of their populations, with famine and poverty as a result. There are various solutions, but regreening – the natural (re)generation and protection of trees by local farmers themselves – is a highly successful one. Belts of trees act as windbreaks, helping to stop soil blowing away, keeping it moist for longer, and providing a micro-climate that is better for people, animals and plants. Trees also provide food and many other economically useful products.

Within the W4RA programme, we integrate local ICT web and mobile app innovations to support local knowledge sharing around regreening efforts.

 

Share This:

VU looking further in Mali

[This post by Anna Bon is cross-posted from W4RA.org. See also the VU press release: VU looking further in Mali]

On 13 October 2016, the W4RA team organized and co-chaired, a Green Climate Funds workshop together with Malian farmer organization AOPP (l’Association des Organisations professionnelles paysannes). The objective of the meeting was to form a consortium and prepare a project plan, which will be submitted in the framework of this United Nations program.

The workshop was attended by representatives from the Dutch Embassy, the Swedish and Norwegian embassies, and by development (donor) agencies from the EU, Germany, the United Nations Capital Development Fund, the Global Environment Facility (GEF) and a range of Malian and Dutch development organizations.

The workshop in full effect (photo Anna Bon/W4RA.org)
The workshop in full effect (photo Anna Bon/W4RA.org)

 

Mali is one of the poorest countries in the world, plagued by the effects of climate change and a civil war in the northern regions. The effects of land degradation and desertification are a serious threat to the food security of millions of people, especially those living in rural regions.

Recently, the United Nations prioritized its support to Mali in the framework of the Green Climate Funds, a new programme to fight the effects of climate change on global scale. In response to a call for proposals, organizations in Mali are forming consortia, to prepare project proposals for funding by the Green Climate Funds.

Through ongoing interdisciplinary research collaboration, W4RA has obtained extensive experience in socio-technical field-based action research in West Africa. Building on partnerships with local partners (AOPP, Sahel Eco and Radio Rurale – Mali, Réseau MARP -Burkina Faso, University for Development Studies – Ghana) VU’s research programme W4RA wants to contribute to regreening, local knowledge sharing, local innovation and emerging rural agro-forestry value chains.

Meanwhile the W4RA is training students, through community service education, in rural Africa. This is done through the ICT4D master course (artificial intelligence, information science, computer science,) and various master research projects (Network Institute Academy assistants, various master research projects).

 

Share This:

The Role of Narratives in DIVE

[This post is based on Maartje Kruijt‘s Media Studies Bachelor thesis: “Supporting exploratory search with features, visualizations, and interface design: a theoretical framework“.]

In today’s network society there is a growing need to share, integrate and search in collections of various libraries, archives and museums. For researchers interpreting these interconnected media collections, tools need to be developed.  In the exploratory phase of research the media researcher has no clear focus and is uncertain what to look for in an integrated collection. Data Visualization technology can be used to support strategies and tactics of interest in doing exploratory research

Dive screenshotThe DIVE tool is an event-based linked media browser that allows researchers to explore interconnected events, media objects, people, places and concepts (see screenshot). Maartje Kruijt’s research project involved investigating to what extent and in what way the construction of narratives can be made possible in DIVE, in such a way that it contributes to the interpretation process of researchers. Such narratives can be either automatically generated on the basis of existing event-event relationships, or be constructed  manually by researchers.

The research proposes an extension of the DIVE tool where selections made during the exploratory phase can be presented in narrative form. This allows researchers to publish the narrative, but also share narratives or reuse other people’s narratives. The interactive presentation of a narrative is complementary to the presentation in a text, but it can serve as a starting point for further exploration of other researchers who make use of the DIVE browser.

Within DIVE and Clariah, we are currently extending the user interface based on the recommendations made in the context of this thesis. You can read more about it in Maartje Kruijt’s thesis (Dutch). The user stories that describe the needs of media researchers are descibed in English and found in Appendix I.

Share This: