PhD Position in AI and Visual Analytics for Multi-Modal Summarization

Are you interested in performing cutting-edge research in Artificial Intelligence (AI) and visual analytics for social good? Visual analytics is the science of making sense of data using visualization and modeling techniques. The Informatics Institute at the University of Amsterdam is looking for an ambitious PhD student to integrate AI and visual analytics in summarizing multi-modal data in the public health domain. Your research is part of the Multimedia Analytics (MultiX) lab with a strong focus on dealing with multi-modal data.

Nowadays, there is a lot of multi-modal data related to public health in local regions, such as citizen reports/complaints, social media streams, camera monitoring footage, and environmental sensor readings. There is a strong need to build AI-driven visual analytics tools that can summarize these data to help local stakeholders understand the patterns and events that are currently happening, such as air pollution, disease outbreaks, and flooding. Besides, summarization can be used to encourage citizens to monitor their surroundings, which can lead to a sense of community, autonomy, and empowerment. For example, when citizens receive an event notification via apps/emails, they can click on the notification to go to an AI-driven visual analytics tool that shows the summary with different ways for users to provide feedback interactively.

This research topic poses at least the following challenges:

• Summarizing evolving multi-modal data is difficult. Public health data are often multi-modal. Traditional techniques typically use different pipelines for different types of data, making it difficult to connect findings. Moreover, public health data is dynamic and evolving, which means the patterns and events can shift quickly over time (i.e., concept drift). Most techniques deal with a fixed dataset, which may not be suitable for evolving data.
• Improving summarization based on stakeholder feedback is hard. Public health data depends on context. Thus, the summarization model must consider knowledge and feedback from stakeholders. However, stakeholders can provide different feedback interactively (e.g., labels, comparisons, scalars, rankings, corrections, natural language). It is hard to let the system learn from those and have the right balance between their richness and efficiency.
• Representing diverse stakeholder values is challenging. Public health data can be biased due to limitations in sampling strategies or stakeholders’ prior beliefs. Moreover, stakeholders can have misaligned and even conflicting values when providing feedback. There is a need to provide bias and fairness measurements for the patterns and events in the summarization for laypeople who have limited technical backgrounds.

What are you going to do?

You will conduct research, experiments, and empirical studies to address the challenges that are mentioned above (or other related challenges).

Your tasks and responsibilities:

• Conduct research and experiments in integrating AI and visual analytics to summarize evolving multi-modal data;
• Create and deploy a web-based visual analytics tool that can show the summarized insights to stakeholders and enable them to provide different forms of feedback interactively;
• Conduct research and experiments in investigating human-in-the-loop AI techniques to improve the summarization model using continuous stakeholder feedback;
• Inspect how AI and visual analytics can be used to help stakeholders identify bias and fairness issues in the summarization;
• Conduct empirical studies using a mix-method approach (i.e., both qualitative and quantitative) to evaluate the visual analytics tool with stakeholders;
• Publish and present research in international peer-reviewed conferences (e.g., ACM WWW, AMC MM, IEEE VIS, ACM CHI, ACM KDD, ACM IUI, AAAI) and/or journals (e.g., ACM TIIS, IEEE TVCG);
• Pursue and complete a PhD thesis within the appointed duration of four years;
• Assist in teaching activities, such as teaching labs/tutorials in courses and supervising bachelor/master students;
• Carry out administrative tasks in the research group, such as scheduling and planning activities for group meetings.

What do you have to offer?

Your experience and profile:

• A relevant master’s degree to the PhD topic of interest;
• Research experiences in artificial intelligence, visual analytics, or related topics;
• Solid programming skills with experience using Python and machine learning frameworks;
• The willingness to work collaboratively with other researchers and external stakeholders;
• Professional command of English (both verbal and written).

Our ideal candidate has an artificial intelligence and/or visual analytics background. It is a preference if you additionally speak professional Dutch, have experience in developing/deploying web-based applications, or have co-designed tools with stakeholders.

Our offer

A temporary contract for 38 hours per week for the duration of 4 years (the initial contract will be for a period of 18 months and after satisfactory evaluation it will be extended for a total duration of 4 years). The preferred starting date is as soon as possible. This should lead to a dissertation (PhD thesis). We will draft an educational plan that includes attendance of courses and (international) meetings. We also expect you to assist in teaching undergraduates and master students.

The gross monthly salary, based on 38 hours per week, ranges between € 2,770 in the first year to € 3,539 in the last year (scale P). UvA additionally offers an extensive package of secondary benefits, including 8% holiday allowance and a year-end bonus of 8.3%. The UFO profile PhD Candidate is applicable. A favourable tax agreement, the ‘30% ruling’, may apply to non-Dutch applicants. The Collective Labour Agreement of Universities of the Netherlands is applicable.

Besides the salary and a vibrant and challenging environment at Science Park we offer you multiple fringe benefits:

• 232 holiday hours per year (based on fulltime) and extra holidays between Christmas and 1 January;
• multiple courses to follow from our Teaching and Learning Centre;
• a complete educational program for PhD students;
• multiple courses on topics such as leadership for academic staff;
• multiple courses on topics such as time management, handling stress and an online learning platform with 100+ different courses;
• 7 weeks birth leave (partner leave) with 100% salary;
• partly paid parental leave;
• the possibility to set up a workplace at home;
• a pension at ABP for which UvA pays two third part of the contribution;
• the possibility to follow courses to learn Dutch;
• help with housing for a studio or small apartment when you’re moving from abroad.

Are you curious to read more about our extensive package of secondary employment benefits, take a look here.

About us

The University of Amsterdam (UvA) is the Netherlands' largest university, offering the widest range of academic programmes. At the UvA, 42,000 students, 6,000 staff members and 3,000 PhD candidates study and work in a diverse range of fields, connected by a culture of curiosity.

The Faculty of Science (FNWI) has a student body of around 8,000, as well as 1,800 members of staff working in education, research or support services. Researchers and students at the Faculty of Science are fascinated by every aspect of how the world works, be it elementary particles, the birth of the universe or the functioning of the brain.

The mission of the Informatics Institute (IvI) is to perform curiosity-driven and use-inspired fundamental research in Computer Science. The main research themes are Artificial Intelligence, Computational Science and Systems and Network Engineering. Our research involves complex information systems at large, with a focus on collaborative, data driven, computational and intelligent systems, all with a strong interactive component.

The Multimedia Analytics Lab Amsterdam (MultiX) performs research on multimedia analytics by developing AI techniques for getting the richest information possible from the data, visualizations, and interactions surpassing human and machine intelligence. We blend multi-modal data in effective interfaces for applications and social impact in public health, forensics and law enforcement, cultural heritage, and data-driven business.

Want to know more about our organisation? Read more about working at the University of Amsterdam.

Any questions?

Do you have any questions or do you require additional information? Please contact:

• E: Yen-Chia Hsu, Assistant Professor
• E: Marcel Worring, Full Professor