Talkatu!

This article was originally posted on Patreon and has been brought over here to get all of the Pidgin Development/History posts into one single place.

Talkatu is a library we created for Pidgin 3 which is a collection of GTK widgets for chat clients. We created it as a separate library with the hope that other GTK based chat clients might adopt it and help develop it. The main components of it are a widget for the conversation history as well as the input widget. The latter is what actually drove the creation of this library and those origins are what we're going to cover in this post.

For as long as I've been with the project, Gaim/Pidgin's input widget was a custom widget named GtkIMHTML. It is a custom GtkTextView that, you guessed it, renders the HTML that's used in instant messaging. This widget was used for both the conversation history and input.

When I got started in the project the current release was Gaim 0.67, but it wouldn't be until Gaim 0.74 that GtkIMHTML would get WYSIWYG (What You See Is What You Get) support. Before this, if you wanted to make your text bold, you could use the formatting bar that would insert the HTML for you or manually type it in yourself. Regardless of which way you chose, you'd end up with message like the following in the input widget:

omg.. I'm SOO bored!!

The 0.74 release was a tough one due to the addition of WYSIWYG. There was a LOT of bugs. So much so that I threw together another tracker for them so we didn't have to deal with the SourceForge issue tracker which was what we were using at the time.

Over time additional bugs would be found and fixed, but most of the code was jammed in gtkimhtml.[ch], with additional HTML helpers in gtkutils.[ch]. This made it very hard to work with, so adding new features weren't really high priority for anyone. So the widget just kind of sat that way for awhile.

Then Adium came out using a WebKit WebView as its conversation history with fancy themes that included animations! There was of course talk and even work to adding support for these themes in Pidgin, but that would mean moving to a WebKit webview.

I'm not sure if there was ever an attempt to migrate to a WebKit WebView in Pidgin 2, if I had to guess, I would say there wasn't because making this work on Windows would have been a lot of work. But this work did actually happen on the Pidgin 3 branch.

A summer of code project in 2009 converted the conversation history and input widgets into WebKit WebViews as well as added a plugin to be able to load Adium themes. It took some work getting everything locked down, but everything worked fine and the plan was that Pidgin 3 was going to use this going forward.

But then we hit the block. I'll do another post about the block in the future, but basically with Pidgin 3 we were planning on breaking backwards compatibility. As this doesn't happen very often, we started stacking more and more things into it. So much so that it basically derailed from scope creep and what I've spent the past 6 years trying to salvage.

Due to this block, this code sat. For a very long time. So much so that WebKit 2 had stepped up to succeed WebKit 1. Similar to the Pidgin 2 to Pidgin 3 change, the WebKit 1 to WebKit 2 update completely changed the way you interacted with the WebView. So updating got put on the back burner as we had plenty of other things to do in the mean time.

But then came the deprecation and removal of WebKit 1 for Linux distributions. I'm not sure who was first, but a Bug was created in the Fedora Issue tracker on 2016-09-14 stating that WebKit 1 would not be available in Fedora 27 which was scheduled for release in about a years time and that all users should port to WebKit 2. The reasoning for this deprecation and removal was due to a large number of security issues that just were not getting fixed.

Initially the plan was just going to be to port to WebKit 2 like everyone else and be on our way. It was then that we learned of a dirty little secret. There was no active WebKit 2 port for Windows. Being a cross platform application, this was a huge problem for us. The idea of just porting to WebKit 2 now also included, bringing the Windows port of WebKit 2 up to date. It should come as no surprise to anyone here, but I am not a Windows developer, updating that fork was going to take some extensive knowledge about Windows' graphics and networking APIs which is not something I was prepared to do.

We spent a lot of time looking at our alternatives. We even considered trying to bring back GtkIMHTML, but after skimming the code again I decided that wasn't going to happen. For a while we were really considering using litehtml. But that didn't have JavaScript support, so we'd have to give up on the Adium theme support which was the main thing driving the WebKit support in the first place.

That made me stop and think about why we were still trying to use a WebView in the first place. Historically WebKit has been a security nightmare. The number of video game consoles and mobile devices getting rooted because of it alone should be enough to give anyone pause. We also have had a long stance on supporting themes over skinning, and the Adium themes are basically skinning at least for the conversation window.

It was at this point, in November of 2017, I started playing around with rebuilding GtkIMHTML, but with a more modular approach and using modern API in GTK. This work was done in a separate repository due to its experimental nature. By the end of the month I had a basic version that supported copying and pasting. This seemed like the way to go, but there was something like 14 different uses of WebViews in Pidgin 3 and making sure we could support everything in the new widgets was going to take awhile.

Also at this time, WebKit 1 was getting removed from distributions. I was able to save a copy of the Debian packages and provide them for the other developers. But for other distributions the recommendation was to somehow disable WebKit 1, which I believe QuLogic did for awhile, or just not upgrade.

By happenstance, in February of 2018 I had purchased an AMD Threadripper 2990 WX so that I could build Pidgin 3 much faster and thus spend more time developing and less time compiling. Not only did this take my Pidgin 3 builds down from ~90 seconds to about ~15 seconds, it also meant I could compile WebKit much faster as well. So I started building RPMs and APKs of WebKit as well for the other Pidgin developers. If I recall correctly, the wall time for me to build WebKit was around 15 minutes, but the user + system time was over 14 HOURS!!

We continued working on this new version of GtkIMHTML for a number of months. In June of 2018 we officially named it Talkatu. The name is a pun of the parrot in Super Mario Odyssey who's named Talkatoo who will give you hints. It's also a bit of "IM speak" so that it also means it can be read out as "Talk At You". All of this seemed very fitting as you're using it to talk to people. I also have a guideline for naming things that there needs to be at least two puns in it for it to be a good name!

Talkatu was officially added to the Pidgin build on September 3rd, 2018. But it wouldn't be until July 19th of 2019 that we finally removed WebKit from the build. Since then we've added a number of features to Talkatu and even ported it to GTK 4 as well.

However, Talkatu's future is a bit in question. While it made sense to build it in its own repository and we would still love to see it become standard widgets for GTK based chat clients, it just doesn't seem like that's going to happen. We've also run into some interesting issues for advanced features like custom emoji and message reactions.

For example, since Talkatu depends on GTK we can't use it from libpurple (see the previous post about Pidgin's Architecture for some more context). So right now there is a TalkatuMessage interface that represents a message. This is used in Talkatu so that the input widget is literally a message, but it also gives us a type to use when displaying the messages in the history. However, libpurple needs to have its own representation of a message, and it does this via PurpleMessage. So to make this all work, there's a PidginMessage that implements the TalkatuMessage interface and maps those calls to a PurpleMessage that it's wrapping. This is not an ideal setup in any way.

But as I mentioned earlier, custom emoji are the real problem. When you start stacking protocols like Slack, Discord, Twitch, etc. The number of custom emojis explodes. So wrapping every single one of them is not only a giant use of resources, it's also adding additional operations when searching for emojis.

There's also an issue with the way Talkatu handles the author of a message. At the time of its design, we just followed what we had done previously and used a string to represent the author. Since then, we've created PurpleContactInfo (which I'll be covering more in a future blog post), that libpurple uses to keep track of available contacts.

In modern chat protocols, any time a user is referenced it is done via an opaque identifier. PurpleContactInfo contains these identifiers, as well as the contact's username, alias, and display name. This gives us a single source of truth that the protocols and user interfaces can update and everything should just update because the property changed.

But since Talkatu can't use this object as it's from libpurple and Talkatu is a standalone library, that means we'll need to handle the accounting of this and we've been avoiding doing this in everything now by binding properties, so this isn't something we really want to do, which means Talkatu would need some way to represent a Contact.

Due to these reasons, we are most likely going to be folding Talkatu back into the Pidgin code base and simplifying that interface quite a bit. While I'd love to have a library we can share with others that isn't so tied to libpurple and Pidgin, I just don't think we can engineer around it. But if this sounds like something of interest to you, please let me know soon, because we have to make this decision before the Pidgin 3.0.0alpha1 release, which doesn't have a date yet, but does have some official tracking finally. This list will ebb and flow over time, but when it reaches 0 open issues we should be ready to release an alpha!

I hope you're enjoying these posts! Remember they go live for patrons at 9AM CST on Mondays and go public at 12AM CST on Thursdays! If you'd like to support my work, you can join find a list of ways to do so here.