ADOLFO NETO: Hi, everyone. I am Adolfo Neto, a professor at the Federal University of Technology, Paraná, and this is the Professor Adolfo Neto podcast. This podcast is part of the Emilia's Network of Podcasts. In this podcast, I interview professionals from the field of computing and related areas. Today, I welcome Nicole Novielli to the show. Nicole is an associate professor at the University of Bari, Italy, and a member of the Collaborative Development Group. Her expertise lies in affective computing in human aspects of software engineering with a focus on the emotions of developers. Nicole, welcome to the podcast. Thank you for accepting my invitation.
NICOLE NOVIELLI: Thank you, Adolfo, for inviting me. It's my pleasure to accept your invitation and I'm honored to have this opportunity to contribute to your podcast.
ADOLFO NETO: Okay, Nicole, before we dive into the topic of emotions in software engineering, let's get to know you a little bit better. Could you tell us a bit about your background, what led you to focus on computer science and specifically affective computing?
NICOLE NOVIELLI: Okay, so I have to say that I accidentally stumbled upon computers and programming when I was a teenager and I was in high school because I had this opportunity to join this experimental project at high school when we had like 10 years of computer science per week. So it was pretty much, it was a significant amount of hours. So this gave me the opportunity not only to learn like basics of fundamentals of programming, but also to learn more broadly about computer science, databases, theoretical computer science, computer architecture. So I became very much, I didn't expect, you know, this would have triggered to this extent my curiosity and interest. So I suddenly became passionate about this topic and I decided to continue studying computer science. So I basically studied computer science for my bachelor, for my master's degree, and of course I did a PhD in computer science related topics. And in particular, I was working on dialogue analysis and recognition of user communicative intention, which is something which is pretty much, you know, popular right now, also thanks to technologies, the state of the art technologies, as I said, child GPT, but back then was pretty novel topic and rather unexplored. And so, yeah, and affective computing, as for my interest in affective computing. Right after my bachelor, I started collaborating with a group which was working on intelligence interfaces and in particular on natural language based interfaces. And they were involved in a research project in a European network of excellence, which was called Humane back then. And I joined the group as a research assistant. So during my master, I started doing research on this topic, on emotional recognition mostly, and also on the recognition of emotion from text and how people interact, different kind of attitudes, like I can be friendly, hostile towards the machine and the systems and how this impacts the effectiveness of the dialogue itself in terms of whether this is useful or not for reaching the goal of the user and all kind of related topics. And it was very exciting because this was, of course, a multidisciplinary research domain. So I had the opportunity to interact with psychologists, with linguists, and actually this is also what triggered my interest in a research career. So this is how it all started.
ADOLFO NETO: Oh, that's great. I have read some of your papers and this interdisciplinary aspect of your research is very interesting. So you are right now an associate professor at the University of Bari, right? And how did you become interested in studying human factors in natural language interaction?
NICOLE NOVIELLI: Right. So this was like a natural follow up of my initial research activity as a research assistant that I was talking about before. And almost 10 years ago, in 2013, I joined the group, the Collab Research Group, which is led by Professor Filippo Anubil at the University of Bari, and has been always interested in software engineering and also in collaborative software development, which is strictly related to human aspect as well. And so I found this as a very natural fit, given my background and expertise. So I suddenly started working on topics related to mining software repository, especially analysing the developer communicative traces. I've been working on a project on the role of emotion and the communicative style on technical Q&A and in particular stack overflow. And I've also been working on recognising the emotion of software developers as well, and how this emerged also in written communication, such as an issue tracking system, or also in face to face dialogue, for example, in retrospective meeting, which is what we're working on right now. So this was like a kind of, yeah, I got interested in this topic as a result of a combination of aspects of both my background and original research interests, and of course, what the collaboration group has been working on in the last decades.
ADOLFO NETO: Yes, and I saw, I read in some of your papers that you studied, you and your collaborators studied, for instance, commit messages. And also, there is a study where the developers used a wristband, right? And something on their foreheads. But my question before the next question is, did the pandemic change the way you do your studies? I mean, are people now working remotely more and then this had to change the way you do your studies?
NICOLE NOVIELLI: Well, I have to say that, of course, accessing the lab was an issue during the pandemic. So data collection in particular. So okay, I have to say something else before. So my focus on emotion of software development is actually related to both written communication as developers mostly collaborate also through collaborative platforms. So there is a lot of developer communication traces on GitHub, on discussion on Stack Overflow, on issue trading system, and code reviews have been mining all sorts of data as well. So this part of the research was not affected very much by pandemic, because this was like, by design already focusing on remote collaboration. So I didn't see any difference in how to approach my research. And also now developers actually interact with each other. But of course, when biometrics, as you were saying, is concerned, then you have to design and implement experimental protocol, which involve physical presence, because you have to collect data using sensors that the developer wear while doing their programming task. And so this involves accessing the lab. So of course, this was not possible during the pandemic. So data collection has been an issue for a while, for sure. So yeah, definitely, yes. So this was one of the main aspects that was impacted by the pandemic.
ADOLFO NETO: Okay. And then another question is related to emotions in the software development process, because maybe some people think that they are not that important for software developers, because, okay, what you have to do as a software developer is to produce code, and code has no emotions. There is even a phrase that software developers use is, show me the code, because that's what makes the computer do what you want it to do. And why do you believe that emotions are essential to consider in software engineering?
NICOLE NOVIELLI: Okay. Emotions are part of our everyday life. And evidence from psychological research has shown that emotions are definitely linked to cognitive skills. And emotions might influence job performance. And this was studied a lot in psychology. And of course, this is also true for software development, because software development involves creativity and problem solving skills, which are cognitive processes that are known to be affected by emotion. So first of all, this is one of the motivations. So like being, trying to, being able to implement early detection of negative emotion is crucial to remove those stages of frustration that might be associated to learning new technologies. So it might be something you want to avoid when you want to learn something new. Or also, negative emotion might be a proxy for also for technical problems. So if I complain about some technical issue I'm experiencing and nobody pays attention to my request, this frustration might build up until anger has actually reached. And we also have been studying anger in particular, together with Professor Alexander Serebrennik from the University of Eindhoven, for example, on the issue tracking system, in engineering in particular, an issue tracking discussion. And we observed that anger in that case was seen as a proxy for a different problem. Like it could be an indication of frustration. So anger directed at myself. Or it could be also an indication that something is not adequate in terms of hardware or equipment or documentation is not accessible. So it could be a proxy for something that you want to fix. And also anger at each other, like at my peers and my colleague, and in general emotion and collaborative settings are important because they might impact communication which might not be effective in the end. Like I might not be able to reach cooperative goals because negotiation or communication is affected by negative emotion, by a wrong way of communicating them at least. So for example, on Stack Overflow, I have been asserting that neutral communication style is preferred. So even positive emotions are not appreciated. So because in that particular platform, this is what the community considers as appropriate, for example. So there is not, of course, a recipe for all settings. So I'm not just saying that there is one kind of intervention for all application scenarios. But being aware of the fact that software is by people is definitely part of the motivation to answer your question. So why is it important to study emotion? Because this kind of emotion can impair both productivity, well-being at an individual level, but also the effectiveness of collaborative process by affecting communication as well. Yes. And the software developers care
ADOLFO NETO: about it because I remember I interviewed a software developer back in 2019 and there was a part of the podcast where I asked her to suggest a book and she suggested a book called Nonviolent Communication. And of course, there are also some bad examples in the community. I believe I can talk about this because he now recognized that he was wrong in the past. He is the creator of Linux, Linux Torvalds. He was famous for his outbursts. He was angry when someone suggested something even in written form. So but OK, these are some more extreme. This is one extreme example. But can you share some other less extreme examples of how emotions can affect software development practices and the dynamics within engineering teams?
NICOLE NOVIELLI: OK, so I have been investigating this part, as I say, of a long lasting collaboration with Professor Serebrennik as well. And we've been investigating the emotion also in a field study with a professional developer from different companies involved. This study was about investigating the perceived productivity as well as the self-reported emotion during the workday. And on top of this, the developers also used biometrics to collect their physiological data during the day so we could use supervised machine learning to try to recognize positive and negative emotions. So let's now focus on the link and the impact of emotion. So we observed that indeed there is a relationship between well-being, self-perceived self-reported productivity and emotion. And in particular, this correlation apparently was stronger in the afternoon because it's probably mediated by fatigue. So also fatigue is somehow the correlation between fatigue and emotion is also a topic which is worth investigating. So because we might be more or less focused, more or less engaged because we are experiencing negative emotion or the other way around. And fatigue sometimes is a factor which impairs our ability to regulate emotion and restore positive mood and thus focus and concentration. And this clearly emerged from a correlation study based on the data that we collected in this field study which was published two or three years ago at TSC. So it's on my website if somebody is interested in checking it. And another thing that we obtained based on the answer of the developers involved in the study, which by the way, they participate in the study for two or three weeks, depending on the agile development cycle that was implemented at their company. So we collected quite a lot of data and we obtained a taxonomy of reasons, causes for positive and negative emotion. And most of them actually are associated to productivity. So whenever people are feeling stuck, for example, they experience negative emotion. And whenever they report being in flow or concentrated or focused, they experience positive emotion. And also whenever they have trouble in collaborative with fear, negative emotion might arise. And also whenever they have like troubles in understanding other code that other people code that they are supposed to modify, for example, to change. So all these kind of causes emerged. And in a way, this could be an indication for themselves, but also for people who are organizing the teamwork for a good starting point for improving the organization of work, for example, and therefore also the process of software development itself.
ADOLFO NETO: Yes, yes. And as a researcher, you probably have faced some challenges in your research, but maybe also the practitioners have faced such challenges when they were trying to incorporate emotional aspects into software engineering. Could you tell us a little bit about this challenge?
NICOLE NOVIELLI: Of course. OK, so we have several problems. Some of them are actually purely methodological. Like, for example, whenever you want to work with written data, for example, you need to assess the emotions which are conveyed through written text. And this is sometimes affected by the fact that people who are writing, who are labeling the data might not be native speaker with respect to the language which is being annotated, which is normally English. And so this might influence the perception of emotion. Also, when you're labeling data, let's say for building classifier for emotion by based on the analysis of text. And another aspect is also when you're collecting data using biometrics, for example, even if methodologies exist and techniques exist for normalizing the data, we have all different profiles for biometrics. So the classifier might not be very well at generalizing with new subjects when new participants join the study. So there are some issues related also for biometrics to these individual differences in between subjects. So even if the recognition of emotion based on sensor data is feasible, this might not be directly usable in practice, at least not yet. And data collection is not trivial as well. And so a huge amount of data is required to try to overcome the problems due to the difference between individuals. So we're not also experimenting with the classifier based approach to try to see whether we can do some more robust learning. But, you know, this is still, you know, having 20 subjects, 20 participants per study poses an additional challenge because this kind of data collection is very expensive in terms of time. And another concern that I had when starting doing this kind of studies, but then it didn't actually turn out to be a problem at all, was privacy. So I was concerned about people not willing to participate in the study because, you know, wearing sensors means that you have to give access to research or to your biometric data. And this might trigger some privacy concerns. But actually, people do not seem like, you know, as long as they're like reassured that the data will be treated in an anonymous forum, they're not concerned at all. And this is also something that we were concerned about when we were trying to recruit in professional developers and companies. So we say, OK, people will be like, you know, I don't know if I feel like I want to join. But actually, they were very much interested and they were also very interested in learning about the outcome of the study as well. So they perceive some value in what we were doing. And I don't know if this answers your question. So if you have any other follow up question.
ADOLFO NETO: Oh, yes. Maybe one additional challenge is not specific to emotions in software engineering is to obtain the collaboration of companies. Right. Because you want to do this kind of research within companies. And regarding privacy, I think that we got used to being observed by all those big tech companies that, oh, if it's a university doing that, it's more than OK. But based on your research on what you know about the others in your perceptions that may not be backed by research right now. Can you answer how software engineering teams can become more aware of the emotional dimensions of their work and what steps can they take to address this effectively?
NICOLE NOVIELLI: OK, so this is something we're actually working on right now. So I have I'm supervising a PhD student here in Bari, Daniela Grassi. And again, we are doing this in collaboration with Endovan, with Professor Serebrennik. And we have designed and implemented a tool for collecting data about emotions, self-reported and biometrics as well. And which software developers are now using in the context of agile development. So what they do before the retrospective meetings, they watch at the data and they try to get insights on what happens in the two weeks before during the spring. That is what's reporting and discussing in the meeting. So we had a very early result paper published last year. And now we have an extended study which is not published yet. And we are continuing also the replication with additional teams also in a training project at the University of Endovan. And so for sure, we are trying to investigate whether this kind of approach can support gaining more awareness of our own emotion and whether developers think it is worth sharing the insights they get by looking at the emotional report from the two weeks before in order to enrich the discussion during retrospective meetings. So this is one of the potential scenarios we were thinking about. Actually, based on the interview with participants, more scenarios are emerging related to emotional self-awareness, which is probably worth investigating in the future. So this could be one of the potential scenarios that we are currently investigating. And as for intervention strategy, I'm afraid I don't know. Because not being a psychologist, I'm not the one who is supposed to design effective communication or intervention strategies. But at least what we are trying to do is to try to provide developers with protocols and tools that at least help them gaining emotional awareness.
ADOLFO NETO: Yes. And my next question is related to the fact that, OK, we have a limited time here. There's so much more to explore about emotions in software engineering. And even psychology, I know that there are lots of pop science books about psychology that even some software developers read. And I also know that there is this replication problem in psychology. I believe there were some papers published in the last few years where one result that was very well-known in psychology and then though someone tries to replicate that study. Oh, it's not true. And it's interesting because some of those pop science books and now not only books, but all kinds of media, they keep repeating those false results. So suppose someone that is listening, watching us, is interested in emotions in software engineering. Where would you recommend they start learning about it?
NICOLE NOVIELLI: OK, this is a tricky question because, of course, if somebody is specifically interested in emotions in software engineering, of course, related work which has been published recently on the topic is also very important to check. So back in 2016, together with some colleagues, with Professor Walid Malay, with Andrew Bagel, Alexander Serebrennik again, we started this workshop which was called C-Motion, which was co-located with Dixit. But now the workshop is not there anymore. So some early studies were also published there. But now also the main track of the software engineering venue has many papers about this kind of topics related to emotion and sentiment in software development. So this is also a place where to look for. And what is very important to keep in mind is that whenever we are trying to define the theoretical framework for this kind of study, we have to keep in mind our research goal and intended use. Because let's say also seeing negative emotion doesn't say enough. Like negative emotion can be anger, can be aggressive behavior, can be sadness and frustration, and all of them might require different interventions. So let's say you want to use emotion detection for moderating your community. You don't want to murder it like sadness expression. You want to control like toxic behavior or hate speech, for example. So also some literature about psychological literature, about theoretical framework for emotion, I guess is required to check. Because different concepts, different constructs might require different operationalization and also different approach for measurement. So this is also very important to check. And of course, depending on the actual need of themes, I would recommend checking also with prediction. Because we also had a lot of interaction with some companies, also some of them here in Bari. And they start getting interested, for example, in the well-being and emotional well-being of developers based on some experience that developers report about how they perceive their productivity and also how this is affected by the environment, by the organization of work. So they might also have some very specific needs based on their specific experience, for example.
ADOLFO NETO: Okay. So my next question is about your... You were invited to be a keynote speaker at the Brazilian Conference of Software, Practice and Theory, which is going to happen here. Right now I'm here in Curitiba, Brazil is where I live. And have you thought about what you plan to present at the event?
NICOLE NOVIELLI: Yes. So thanks for mentioning this because I would like to thank the organizer of the conference for inviting me. I'm very eager to travel to Brazil. It will be my very first time in Brazil. And so I know that is a big community of software engineering. So I'm very excited to join the conference. And yes, I will be reporting mostly about our research journey into the emotional software engineering, software developers. And I would like to provide an overview of our recent findings on developers' emotion, the relationship with self-assessed productivity. And also I will try to argue in favor of the emergence of tools and practices to support developers' emotional awareness, both at an individual and team level. Also towards improving productivity and resilience to failure, well-being in the broad sense. And I will also report more on recognition of emotion using sensors. And what are the promises and challenges of this research domain?
ADOLFO NETO: And I believe you mentioned ChatGPT before, right? And do you feel there is going to be new research related to how software developers interact with either ChatGPT and Copilot and those other tools, specifically regarding emotions?
NICOLE NOVIELLI: I would say so. I would say it's a relevant topic because this, based on my perception, this is kind of related to the broader domain of comprehension, of program comprehension. So whatever relates to cognitive ability to process code and extract information from code and understand the code and change it in an effective manner is always relevant to emotion and other psychological states which might play a role in the effectiveness of the task execution.
ADOLFO NETO: Okay. And I'm also curious about your other research interests. Maybe you do some work besides emotions in software engineering. So are there other areas you focus on? And suppose someone who is listening, watching this interview, he or she is from Brazil or even from other countries and want to do a PhD under your supervision at the University of Bari. I have seen on, I haven't visited Bari yet, but I visit Italy, but not Bari. And I saw on YouTube, it's a beautiful city. They want to do a PhD under your supervision. Which other areas can they focus on?
NICOLE NOVIELLI: Sure. So yes, Bari I can say is definitely a nice city and our university is also always welcoming visitors or students interested in starting a PhD, of course. And beyond emotion, more broadly, I focus on AI for software engineering. So I'm currently, for example, I'm currently leading a project on quality improvement of AI based systems. So the problem is about a topic associated to the evaluation also after deployment of AI based system, including also data quality issues, as well as the continuous assessment of performance of AI based system with specific reference, of course, to software engineering topics and the software development process in particular. And more in general, I normally work a lot also on natural language processing and on mining software repositories. I've been mostly focused on developer communications, so code review, issue tracking discussion, Q&A, thread of interaction. But we are now also working a lot with issue report classification and labeling. One of my students is currently working on the use of large language model, for example. Well, he's currently visiting the NASA Goddard Space Center in US. So he's working on issue classification and related topic as well there on some other databases, other data that they will be providing. And so all kind of topic related to AI for software engineering is definitely something we're interested in. And in Paris, to start a PhD, you need to already graduate to be a master student, graduated. And so if you hold a master, you can apply. And it's a three year path, which can be extended, of course, and up to four. And yeah, and we also have we actually I'm actually moving to different school right now. So if you're interested in what to check in Italy, we have this new thing about national PhD school. So beyond the School of Computation, Computer Science and Mathematics in the University of Bari, there is also the possibility to apply for a position at the National School of PhD in Artificial Intelligence, which I am part of the board. And this is a multidisciplinary board. So it's a very interesting option because you have people from different backgrounds that you can ask for support during your PhD project development. So this is also another opportunity. And the University of Bari is part of this national project together with other universities all around Italy.
ADOLFO NETO: Oh, that's great. Here in Brazil, and more specifically in the state where I am, there are lots of people from Italian ancestry, and maybe they want to follow that path. And my final question, which I ask all my guests, it's about is about learning. And it's I know, of course, you are a professor just like me, and we have to read and we want to read a lot of papers. Maybe I feel that there are more papers that I want to read than those that I can read. But besides that, how do you stay updated in your field? And do you rely on books, blog posts, podcasts and other resources?
NICOLE NOVIELLI: OK, yes. And of course, as you say, as academics, we are very passionate about classical venues such as books or academic papers. But I also consider it very important to check magazines and podcasts as well. And also on my personal experience is that interaction with colleagues is a great source of inspiration and learning. Like when I understand that many cases, this is also related to travel funding, but like attending conferences and something which is a really great boost for collaboration and for research in general, because you get to talk to many people working on different areas that can provide also by simply by serendipity, you know, just hints for like they just open a door on a completely different landscape and starting from a small project in collaboration, you can just end up, you know, years later, I work making evolving, you know, this collaboration into something bigger. And so this forces you to study, to search for more papers and more research paper in order to learn more for the purpose of the collaboration. So in general, yeah, interaction. And we are blessed by this possibility to travel and participate and attend scientific events, which is interesting, interesting both for staying up to date in terms of research findings that are published, but also for the opportunity to talk to people and which is a great way to start collaboration and learning.
ADOLFO NETO: I want to talk to all people that we are going to have the largest software engineering conference in Brazil in Rio de Janeiro in 2026. So as you mentioned, the conference. And thank you, Nicole, for sharing your insights with us today. Is there anything else you'd like to add before we wrap up?
NICOLE NOVIELLI: Thank you. I just want to thank you again for inviting me. This is a podcast which I already knew and I was like, you know, happy to be invited and to join this project because I think it's very interesting. This is definitely one of the other ways to stay current and up to date to learn how what people are doing research on. And so thank you again. Thank you for inviting me and see you in Curitiba.
ADOLFO NETO: Thank you again, Nicole. And a big thank you to all our listeners and viewers for joining us today. We appreciate your time and attention. If you want to leave a comment below, please do. And until next time and goodbye. To the next level, check out my affiliate link to the ELX Pro course in the episode description.
Top comments (0)