IETF 115 was held in London from 5th November through 11th November 2022, with members of the "Streamlining Social Decision Making for Improved Internet Standards" (sodestream) project taking part in activities throughout the meeting.
To start the week, project members, colleagues, and students from the University of Glasgow (Elizabeth Boswell, Stephen McQuistin, Mairi Sillars Moya, and Ivan Nikitin) and Queen Mary University of London (Ignacio Castro, Mladen Karan, Prashant Khare, Hugo Ramirez, and Ravi Shekhar) took part in the IETF Hackathon.
The focus of our work at the Hackathon was on analysing the sentiment of postings to the firstname.lastname@example.org mailing list. This mailing list provides a forum for the broad discussion of IETF-related topics. Sentiment analysis techniques could be useful in characterising the tone, and levels of toxicity, in the interactions that take place on this, and other, IETF mailing lists. With historical data, these trends can be tracked over time, providing insight into how the IETF community is evolving. The team managed to generate a dataset of sentiment scores, passing e-mail through the VADER library to understand if messages are broadly positive, negative, or neutral. In addition, they started to plot broad trends over time, and for individuals, and sketched out improvements to tooling documentation and packaging.
There initial evidence of some interesting trends: relatively low levels of negativity, with more negativity on found on postings made on weekends and via person e-mail addresses, and relatively more positivity on Monday's. Broadly, however, the team found that sentiment analysis over technical text is difficult. For example, phrases like "dropped packets", "killed process", and "abort transmission" are neutral technical phrases that are scored negatively by the sentiment analysis library. It is essential to build up a lexicon of technical phrases to avoid misclassification.
Ignacio Castro presented a summary of our recent work to meetings of the Internet Engineering Steering Group (slides), the Working Group Chair Forum (slides), and the Measurement and Analysis of Protocols Research Group (slides, recording). These presentations highlighted our findings, including that conversations seem to be getting more complex, publishing is harder, and that the relevance of a minority of influential participants is growing. Our findings suggest that these are interconnected.