Smart speakers, such as Google Home or Amazon Echo, are entering our homes and enriching the Internet of Things (IoT) ecosystem already present in them. The Intelligent Personal Assistants (IPAs) they include allow users to ask for different information (e.g., the weather or a recipe), set up reminders and lists, and to directly control other IoT devices (e.g., lamps), among the various options. Such assistants, through a companion app installed on their owner’ smartphone, provide advanced features like the possibility to set up some personalization rules in the form of trigger-action: if something happens, then do something else. IPAs can be part of these rules either as triggers (i.e., when the user says a specific sentence) or actions (i.e., to reproduce a specific sentence), transforming them from intelligent agents to simple sensors and actuators. End-user personalization capabilities, in other terms, are present in the system but segregated in a mobile app and take no advantages from the NLP and vocal capabilities of such devices.

This thesis aims at exploring novel approaches for creating personalization rules through conversation between the user and an IPA. To this end, novel IPAs might proactively suggest new rules to be activated, e.g., by proposing to automatize some detected user’s behavior or by extracting rules starting from a generic user’s preference/need, or they might employ novel paradigms to allow the direct specification of trigger-action rules in natural language. The following steps are planned:

  1. Identification of a set of strategies that could be implemented by an IPA to enable the creation of personalization rules, in the trigger-action format, via conversation. Such a step will be performed by a subset of the following: a) reviewing recent works in the end-user development research area, b) analyzing the capability and the functionality offered by contemporary IPAs, and c) interviewing IPAs’ users.
  2. Design and implementation of the identified strategies in one or more IPA prototypes. An intermediate validation of the strategies might be needed, especially if one of them includes some new paradigms in natural language. The implemented prototypes, in addition, may use existing NLU platforms like DialogFlow or Rasa, or they may be implemented on top of existing IPAs, e.g., through Amazon Alexa Skills.
  3. Evaluation with users through an in-the-lab experiment. The evaluation will assess (and possibly compare) the usability and efficacy of the developed prototypes.


Carlo Borsarelli

Thesis Details

Luigi De Russis, Alberto Monge Roffarello
Master Degree in Computer Engineering