People with Asperger’s and NLD often have difficulty differentiating nonverbal signals of anger, teasing, sarcasm and other emotional tones. MIT News featured a wearable AI watch that can tell the wearer the tone of a conversation he’s in. Designed by MIT graduate student Tuka Alhani and PhD candidate Mohammad Ghassemi, the watch would capture emotional information in real time. It’s still in the development phase, but it could be useful in differentiating a conversation that’s hostile from one that’s friendly.
Teasing and sarcasm are communicated nonverbally; friendly teasing and bullying can be hard to differentiate. My AS and NLD clients (and people in general) respond to their interpretation of an interaction, not to its intent. Accurate interpretation is important. At this point, the MIT device is only for 1:1 conversation, but the developers anticipate expanding its use to multiple person conversations.
The development of AI for social emotional recognition is interesting big players – Apple has bought Emotient and Nielsen bought Innerscape. According to Hubspot, an online marketing company, both startups will be marketed to assess consumer emotional response to media and marketing.
AI to support social emotional intelligence in real time is in its infancy. A device might help understanding the emotional tone communicated, but this would not necessarily aid in knowing what to do next. Current AI programs may decode social/ emotional pragmatics like tone, minutiae of facial expression and emotional reaction, but there are other considerations to take into account if the need is to come up with an appropriate response. These considerations are what I call big picture thinking.
Big picture thinking takes in the entire social context – the setting, antecedent events, the nature of relationships among the actors (friends, coworkers, authorities, adversaries, family, ex-boyfriends), the social history of those involved, and the potential goals and outcomes involved. Maybe the tone suggests that the speaker is negative and irritated and the situation involves one’s boss or teacher; that’s different than if it were a friend. With a boss, we might listen to see if anything is directed to us personally. If it’s not, it’s best to respect the boundaries of the roles and not respond. With a friend, we might ask what’s wrong, and offer a supportive comment.
For any social situation, there’s a number of potential responses. Many people pick up all of these elements instinctively and unconsciously, and are able to respond flexibly. For those with AS, NLD and other social challenges wanting to navigate these situations, it’s necessary to think through the big picture step by step. Deciding what to do requires an accurate understanding of the big picture and one’s role in it, and also developing a comfort level with different ways of responding.
Algorithms certainly will be developed for social prompting or coaching, but they will have to be capable of flexibly handling this rapidly shifting and complex social environment. There could be a collaborative interaction between AI and the user. Using software including facial recognition, a user could input other people’s identity (name and relationship, via prompted questions) and through monitoring interactions over time between the user and others, the AI program or device could help clarify understanding of the relationships. Perhaps the AI could offer personalized prompts for responding to interactions. It might recognize that the person seeming irritating is a friend, and prompt a question and then a supportive response about the bad day.
AI offering emotionally based personalized responses is already on the market- a virtual assistant called Emospark is said to be able to read its owner’s emotions and offer suggestions it knows have worked for helping the owner to feel better, like looking at vacation pictures or listening to music to relax, which it can show or play. A virtual BFF.
Obviously, technology will first be developed where the money is – in business – but eventually it is likely that sophisticated AI tools will be adapted to benefit the many individuals who need help decoding the social world. It’s a complex task to integrate so many component factors, but I would never underestimate the ingenuity of researchers and developers.