Emotion - which is integral to service delivery - is studied predominantly using traditional tools of surveys, interviews and experiments. I will describe a new research program that relies on Digital Traces and sentiment analysis data to study emotions of customers and of service agents. The new research offers a more objective picture of customer emotion, its dynamics within service conversations, and its effects on customer satisfaction and on agent behavior. We find, for example, far less negative customer emotion than agents seem to recall, and that prevailing research presumes. We also find emotion displays of service agents to be different from what research on emotional labor suggested. And we show influences of customer emotion on agent response time and agent availability to customers that were not identified in previous research. This new research program opens up questions of why agents recall and report much more customer negative emotion than objectively occurs and call for research on how this can be addressed by further research and by service management.

In a digitalized, remote, and, potentially, hybrid world it becomes more and more complex for individuals to understand and regulate their emotions. For organizations, this has a crucial impact. For example, video communication via Zoom and MS Teams aggravates the connection to peers or Slack shortens and accelerates the exchange of text messages. In the future, the desired scenario of mixed reality collaboration will prevail with uncertain and unknown emotional experiences for the users. Therefore, it is important to develop solutions to support emotion regulation in situ or develop emotion regulation capabilities in organizations. This talk will present two studies in which emotional regulation is studied and developed through the help of AI-enabled artifacts. The talk focuses on two areas of organizational communication, a tool for emotional competence development in video meeting collaboration and a dataset development of emotional regulation dynamics with biosignals in learning.

The convergence of sensing, communication and computing technologies — most dramatically witnessed in the global proliferation of smartphones, and IoT deployments — offers tremendous opportunities for continuous acquisition, analysis and sharing of diverse, information-rich yet unobtrusive time series data that provide a multimodal, spatiotemporal characterization of an individual’s behavior and state, and of the environment within which they operate. This has in turn enabled hitherto unimagined possibilities for understanding and supporting various aspects of human functioning in realms ranging from health and well-being to job performance.

This talk will discuss the challenges and opportunities in the domain of human-centered signal processing and machine learning using example use cases drawn from interdisciplinary research in the broad area of behavioral machine intelligence, including conducted in complex workplaces like hospitals.

Loris offers an AI assistant for customer service agents in text-based conversations (e.g., chat, text and email). A central component of our agent-assist software is our real-time, message-level customer sentiment score predictor. Every customer message is assigned a score from -2 to +2, which corresponds to the range from very dissatisfied to very satisfied. Each score is mapped to an emoji to help the agent identify the customer’s current level of satisfaction. Based on the incoming score, Loris guides the agent through suggested de-escalation techniques. Each message’s score is plotted to show the agent a trajectory of the customer’s sentiment throughout the conversation. The development and deployment of this real-time, five-tier sentiment analysis across multiple business domains provides a unique perspective to examine text-based conversations.

Flow refers to the holistic sensation that people feel when they act with total involvement. Promoting flow in the context of work is desirable, because it leads to increased workers' performance and well-being. However, with the continously growing number of interruptions at work, it is becoming increasingly difficult to achieve the desirable flow state. The talk will present results of a series of lab and field experiments investigating flow-adaptive systems for the digital workplace. In order to recognize flow states, we leverage supervised machine learning on a dataset combining recorded electrical biosignals (ECG) with self-reported flow states collected through an experience sampling method (ESM) procedure. Using this flow classifier, we design and experimentally evaluate several workplace applications, e.g. a mobile flow awareness app and a flow-adaptive notification system embedded into the collaboration tool Slack.

Previous research into online hate - racist, religious, anti-immigrant, misogynistic, and similar comments - assumes that it is intended to harm its targets. A new perspective suggests that online hate is primarily motivated by social approval in online relationships. This perspective offers several hypotheses: First, hate messages increase as individuals garner signals of admiration and friendship through social media’s affordances. Second, online hate is socially organized in certain sequences. While anecdotal and limited empirical research support these assertions, new methods and research ideas are sought to investigate these processes.

We will discuss how Behavioral Signals' emotion recognition engine, OliverAPI, can be used to track group interaction patterns and emotions longitudinally in virtual communication environments. As a case study, we will present the corresponding analysis of the daily scrum meetings of an agile software development team over the course of multiple consecutive sprints. OliverAPI processes the intonation and other vocal qualities of all participants in a conversation and employs deep models to provide a robust assessment of the speakers' emotional state. The proposed, fully automated processing pipeline offers emotion researchers a unique opportunity to gain a better understanding and derive insights related to the complex emotional dynamics occurring in virtual multi-party communication setups in real-life conditions.