App.net Object Sync

I recently released my first app on the Mac App Store, Toki, and I decided that talking about the inner workings of the sync mechanism I’m using would not only be interesting, but helpful for me to think about some of the problems I’m having1.

Toki uses an App.net sync mechanism that I’ve been thinking about and working on for a while — I thought that Toki would be a good low-profile app to test out some ideas in real-world use.

Although Toki is a production app, the sync portion is not feature-complete yet. I’m working on ironing out the basic bugs (planned to be released in 1.0.1 and 1.0.2 releases), before moving on to the more advanced features. My “end goal” for the App.net sync mechanism is an open-source multi-platform library that anyone can use in their own app to use the App.net backend to keep their objects synchronized2. From now on (and until I think of a better name), I will call this sync mechanism “App.net Object Sync” or “ADNOS”.

ADNOS uses the private message / channel feature of the App.net API, using a channel for the app namespace, and a message representing each object.

There are two fundamental steps of the sync protocol (as of version 1.0.2, still in development):

  1. Pulling remote changes: ADNOS pulls all new messages from the channel, processing them one by one, creating or updating local records as necessary. If the record is deemed “duplicate”, it will overwrite local changes3. If the record was newly created in the local database, ADNOS will also record the App.net Message ID in the local database.
  2. Pushing local changes: ADNOS makes a new message post for each record that does not have a App.net Message ID recorded in the local database. When the message is successfully created, the message ID is set.

This has some limits, some inherent in the protocol, and some inherent in the App.net backend:

  1. Message creation is rate-limited to 20 messages per minute4.
  2. Each message is limited to 8192 bytes. Less, actually, because of serialization and metadata.
  3. Recreating the database requires a full playback of messages — this takes time, especially on databases with more than 200 objects.

These limits are pretty much unavoidable. I work around the rate-limiting problem by queueing sync up requests, and pausing the queue when X-RateLimit-Remaining gets dangerously low. The queue is re-started after X-RateLimit-Reset elapses. Any app that plans on using ADNOS will want to be aware of these limitations before implementing it.

And, now for the features I’m planning on implementing (in no particular order):

  • Proper conflict resolution.
  • Streaming support for pulling new changes.
  • Updating and deleting previously synced items. This isn’t implemented yet, but should be relatively easy.

I’ll be writing more as I progress. I hope to get some sort of open-source library out there in the near future.


  1. This is inspired by the “Vesper Sync Diary” series of blog posts by Brent Simmons

  2. There already has been some work been done with a Ruby-based command-line client.

  3. Conflict resolution is not applicable in the context of Toki, but is high on the list of features I want to implement.

  4. This is also why Toki works with such low-resolution data.

App.Net Comments Widget on WordPress

I noticed that most of the good discussion I was having about blog posts were on App.net. Turns out, many other people feel the same way, and there’s even an official App.net comments widget!

I’ve been testing a WordPress plugin to replace the default WordPress comments with ADN comments on this site for quite a while, so I decided to publish it in the WordPress Plugin Directory. Just search for “ADN Comments”, and it should be the first result.

ADN Comments

Installing this plugin will completely replace WordPress comments. You won’t even be able to see them from the admin dashboard — the data is still there, but just hidden. If you don’t like ADN Comments, just disable the plugin and all your old comments will reappear.

If you’d like to display your username as the default “reply-to” username, please set your App.net username in the “Contact Info” section of your profile.

WordPress Admin => Users => Your Profile => Contact Info => App.net Username

Source code is available under the GPLv2 (or later) license. Code contributions, bug reports, etc, are welcome at the GitHub repository.

Thoughts about App.net

I’m sure you have heard of App.net by now. The paid, non-advertisement-supported “live social stream” (read: Twitter). They set a $500,000 goal, and raised more than $800,000 – a clear indicator that some level of demand is there for such a service.

There are two levels of membership, the $50 “User” level and the $100 “Developer” level. A developer account will give you the necessary API keys to build apps that connect to App.net. I was very interested in how these prices were chosen, and thankfully App.net published their logic (“How did you come up with the pricing tiers?” in the FAQ).

I agree with Marco Arment that $50 a year is too expensive for regular users, but he didn’t have anything to say about the developer tier. Here’s an excerpt from the FAQ that irks me:

The developer price is inspired by the amount charged by the Apple Developer Program, $99. We think this demonstrates that developers are willing to pay for access to a high quality development platform.

The decision using the Apple iOS Developer Program as inspiration for the App.net developer tier is, I believe, fundamentally flawed – there are too many differences and not enough parallels:

  • The iOS Developer Program includes hosting of your app.
  • The iOS Developer Program introduces your app to all iOS users
  • Et cetera..

For App.net to convincingly (using their present argument) charge $99 for their developer tier, I believe that the following services are necessary:

  • A central repository of App.net-enabled apps (this looks like a beta)
  • A much larger user base – there were approximately 8,000 “User” tier backers. At $1.99 (revenue of $1.40) of 20% of these users, you can expect $2,240 of gross income. With over 2,000 registered developer accounts, 20% is extremely optimistic.

Of course, if you believe that App.net will balloon, the developer tier is moderately priced. Otherwise, it’s just another lucrative investment.

(Yes, I did back App.net)