This is my next progress post about my Google Summer of Code project of implementing Stateless File Sharing (sfs)

Storing sfs sources in a database

Like everything else we receive, we need to store the sfs sources in a database. In this case, we are in a unique position: Not only are there different kinds of sources, but even http sources on their own are not trivial. For now, we only use the url, but later we need to store much more to receive all kinds of http files. Whether it’s cookies, necessary headers or authentication data, it would be a hassle to store everything in their own column or extra table.

Instead, we decided it would be easier and better with forward compatibility if we use some kind of serialization to be able to store any such data. For that we create a table SfsSources with file_id, source type and data. The idea is that we store the sources we obtain in a serialized data field. This way we can store elements we don’t use yet and have a unified table for everything. However, this also means that data can’t be searched properly with database queries.

Vala interfaces

This paragraph will compare elements of the programming languages Vala and Rust.

Being kinda new to vala, I struggled to map sfs sources into a class. The main issue is that it needs to support multiple variants.

In Rust, I would immediately go with a Rust enum, which allows you to store different data in each variant. For this, I was told that I should use a Vala interface. This confused me, because reading the documentation, they seemed to me a lot more like Rust traits: They define shared behavior that other classes can implement and then be passed generically as the interface type.

The big difference between Rust traits and Vala interfaces, however, is that you can easily cast the generic interface type back into its original class. Using this, they can also be used similar to Rust enums.

ALL the compatibility

I guess we are adding a new file transfer method. Well then, let’s maybe try to not lose compatibility with all the other clients?

Fallback body

When we send a sfs file transfer with a http source, we want to allow the receiver to see Out of Band Data. Exactly to support backwards compatibility, sfs can be attached to any other message, simply as another subnode. All we have to do here, is to create a normal message stanza with Out-of-Band-Data, and send the sfs along with it.

Double Receive

Well darn, now we receive that file transfer twice, because two modules registered an incoming file transfer. Who could’ve guessed.

Dino has about 4 entry points for message listeners

  1. A Signal that listeners can connect to. Owned by MessageModule (module xmpp_vala)
  2. A Pipeline of listeners owned by MessageModule (module xmpp_vala)
  3. A Signal that listeners can connect to. Owned by MessageProcessor (module libdino)
  4. A Pipeline of listeners owned by MessageProcessor (module libdino)

As we can see, there are two types of listeners, and each type is present in each xmpp_vala and libdino.

  • Signal: listeners are called in an arbitrary order
  • Pipeline: listeners are called in specified order, used when listeners depend on each other. Listeners can also drop messages so that they don’t get processed further
  • xmpp-vala: Lower level, access to raw xml elements
  • libdino: For text messages, called after messages are stored in the database

Out-of-Band-Data is retrieved via a ReceivedMessageListener that connects to MessageModule’s signal and sfs via MessageProcessor’s signal. To avoid the double-receive, we create a sfs MessageFlag, that gets attached when we successfully received sfs from a message. The Out-of-Band-Data listener first checks whether that flag is set, before parsing the message itself.

Unifying http file transfers

We of course still expect to receive Out-of-Band-Data messages. However, it is not very nice to need to handle two variants of http file transfer differently in databases and other code. The answer is rather simple here: When we receive Out-of-Band-Data, we convert it into an incoming sfs and handle it as such.

Next up

The next To-do I will approach is image and video specific metadata. As always, you can track my progress on my stateless-file-sharing branch