Treat with Jermolene

The organisers have posted the video from the talk that I gave at the Next08 conference in Hamburg earlier this month. It doesn’t seem to be possible to link directly to the video; you’ll need to scroll down until you find “Jeremy Ruston”:

http://www.next-conference.com/next08/program.html

I was a bit more nervous than usual because I was standing in at short-ish notice for my boss. I added some laconicity, but the material is essentially his, as you can see in this video of JP giving the talk last December at LeWeb 3 in Paris. Osmosoft helped out with the slides, using RippleRap and TiddlyWiki to maintain my strict PowerPoint quarantine.

Happily at least one person liked it.

Along with several of my Osmonaught colleagues, I will be attending OpenTech 2008 in London on Saturday July 5th 2008. It’s a very low cost event (£5 to get in!), and is focussed nicely on the hacker community. I went to the last one in 2005 and had a great time.

This time, I’m giving a talk entitled “TiddlyWiki Tales” which will be a round-up the latest TW developments, both from Osmosoft and the wider community. Later on in the day, my colleague Paul Downey is talking about “The Web as Agreement“.

We’re also holding a competition with a TiddlyWiki flavour and some interesting prizes:

The TiddlyWiki OpenTech competition gives you the chance to win a BUG (plus four modules)! The BUG base station and modules use open source software and snap together to create whatever device you want. There’s also a great second prize of a GP2X Linux powered handheld games console. A stack of O’Reilly books will be the third prize.

For your chance to win one of these great prizes, all you have to do is show us your TiddlyWiki skills. TiddlyWiki is written entirely in javascript, and has a great plugin architecture, so just like the BUG you can twist it to do whatever you want. It can pull in RSS feeds, and because it can be a local file (and you don’t have to worry about same domain policies), it’s great for mashups. There are already 400+ plugins you can use, but don’t let that stop you from writing your own new ones!

(See the Osmosoft site for full details).

So, here at Osmosoft been lucky enough to have access to an iPhone to play with for the last few weeks. Our first discovery was that TiddlyWiki works just fine on the iPhone - apart from being excruciatingly slow. TiddlyWiki is so meaty you really feel the 20x speed difference between a MacBook Pro and an iPhone. Anyhow, I’m confident that we can slim TiddlyWiki down to the extent that it runs at a reasonable speed on these new portable slow AJAX devices. (If you try the leaner and meaner first version of TiddlyWiki on an iPhone, it’s a good deal snappier).

The other problem is that saving doesn’t work properly because the TiddlyWiki is being accessed via http. We can expect that TiddlyWiki adaptations that support serverside saving (like TiddlySpot) will work, but that doesn’t really speak to the TiddlyWiki vision of untethered access to the information that you need.

I quickly discovered a potential solution: the iPhone supports bookmarks whose URL is a data URI. The procedure is to use something like Hixie’s data: URI kitchen to encode a file (HTML, images or PDFs all seem to work fine; possibly Word and Excel Documents too) into a data: link. Then right-click on the link and select “add to bookmarks”. You’ll end up with a bookmark that starts something like this:

 data:text/html;base64,PCFET0NUWVBFIGh0bWwgUFVCTElDICItLy9XM0MvL0RURCBY....

When that bookmark is invoked, the base64 data is converted back into a binary file, which is then displayed by the browser - all without recourse to the Internet.

The really interesting thing is that Safari/iPhone support bookmarks of up to 32 megabytes or so, which suggested the possibility of embedding an entire TiddlyWiki in a bookmark. And, it turns out that it works — giving you offline read-only access to TiddlyWiki files on an iPhone.

Here are the steps to convert any TiddlyWiki to use offline on the iPhone

1. Locate the TiddlyWiki file you want to convert (it can be stored locally or on the internet)

2. Go to Hixie’s data: URI kitchen in Safari

3. Make sure the ‘base64′ checkbox is enabled

4. Either enter the URL of the TiddlyWiki file in the text box, or use the ‘Choose file’ button to select a local file

5. Click ‘Generate’

6. Wait; it can be quite slow to do the conversion with a big TiddlyWiki file

7. Right click on the resulting link and select ‘Add Link to Bookmarks…’

8. Go to iTunes and make sure that you’ve selected to sync your Safari bookmarks with the iPhone

9. Sync the iPhone

10. On the iPhone, select the bookmark

This is all still quite rough and ready; there’s still a bunch of work to do to get a first-class TiddlyWiki experience on the iPhone. In my next post I’ll be looking at optimising the TiddlyWiki user interface for the iPhone, to make it look and feel more like a native application.

I’ve started regularly hanging out in the TiddlyWiki IRC channel. It’s an open chatroom that anyone can join; there’s half a dozen or so regulars, including FND and Saq, and several of my colleagues at Osmosoft.

The channel is called #tiddlywiki and it lives on the server irc.freenode.net.

IRC is quite old-school; there’s a bunch of different clients available for various platforms. I’m using something called Conversation for the Mac.

So, I started TiddlyWiki back in September 2004. It is now, by many measures, a moderately active and successful open source project. The forums are busy, the ecosystem is vibrant, and many people get a great deal of satisfaction and enjoyment from using TiddlyWiki, improving it, and helping others to do so.

Sometimes, people ask me how they can start a similar project, particularly how they can build an open source project around an idea that they’ve got. For a long time, the question used to make me stop short and give the glib answer that I didn’t know, because it had happened to me by accident. I’m acutely aware that I knew next to nothing about open source until long after I first published TiddlyWiki, and I felt that my experience was so specific to TiddlyWiki that it wouldn’t help anyone else.

Now, though, with the benefit of hindsight, there might by now be a few things I can generalise from my experience. Perhaps. Anyway, I’ll try for three things fairly simple observations for the moment.

1. You have to be fearless, if not reckless. In my case, for the two years or so that I was precariously working full-time on TiddlyWiki, I didn’t earn anything like the money I’d earned in my previous jobs. The only way I could survive was not to put away any money to pay the tax bills that I was building up but to spend everything I was earning on day-to-day living and bills. You can juggle things in the UK so that you don’t pay tax on your income for two years if you’re careful, but my cunning plan was clearly not something that a fully trained accountant would advise anyone to do. Especially given that I had no indication of any future payout that a happy-go-lucky angel investor would recognise, let alone one to impress the friendly tax man. Living hand-to-mouth is never fun, but I got huge motivational sustenance from the enthusiasm of the other people who were putting so much into the project. I reasoned that dozens of strangers wouldn’t be investing in it if TiddlyWiki were without serious merit, and my natural blind optimism filled in the leap that something of serious merit would eventually allow me to pay my bills.

2. You have to build the thing before they will come. An idea alone is not enough. In the case of TiddlyWiki the first proof-of-concept version that I released in September 2004 has around 850 lines of JavaScript code, much more than a trivial proof of concept or a wireframe sketch. I then worked on TiddlyWiki for around 9 months as a pretty much solo effort, driven by the bug reports and feature suggestions that I was receiving. For all of that period, it would have been hard for me to describe TW as an open source project in anything other than license terms (back then there was no subversion repository, no mailing list and no bug tracking system). It was only when TiddlyWiki itself was mature enough to sustain a user community that the demand built up for it to become a proper open source project.

3. You have to make it easy for your users to find and use your source code. In the case of TiddlyWiki, the website is the product is the code is the documentation, which has neatnesses of its own. But there’s more than that: one of the things I’ve learned from working with the TiddlyWiki community is that there are an awful lot of almost-programmers out there: people who couldn’t reliably tell you much about the syntax of JavaScript but have enough wit and common-sense to splice and modify samples of code. Especially if they have the means to try out their changes easily and safely. Give them these tools, and these people can do wonders. In contrast, many open source projects have much more complicated installation, build and execution requirements. It’s often surprisingly hard just to locate the source code and browse your way to a particular part of the codebase.

A great example of how successful TiddlyWiki has been at lowering the barriers to contribution is the Digital Supplement for “Web Campaigning” by Kirsten Foot and Steven M. Schneider” (MIT Press 2006). The authors have published their source materials as a TiddlyWiki file that fellow researchers can download and use to begin their own work, use as a template or indeed study while on an aeroplane. The really interesting thing is the way that Steven assembled their own unique customisation of TiddlyWiki, consisting of a handful of off-the-shelf plugins and one that he wrote himself to handle their unique requirements for footnotes and endnotes. The macro isn’t particularly complicated in programming terms, but it radically simplifies the annotation of data, and demonstrates a beautiful blurring of boundaries between programming and writing.

I’m delighted to announce that the mighty BT has acquired my tiny little company Osmosoft Limited. I’m joining BT as Head of Open Source Innovation, and I’ll be building a crack open source web development team called BT Osmosoft. To say the least, this is big news for me personally, and I hope will have a positive and lasting impact on the future of TiddlyWiki.

BT is becoming a remarkable thing: a truly internet-scale consumer company that doesn’t rely on owning “secret sauce” software for it’s business. At the most senior levels, there’s an appetite to embrace open source that wouldn’t disgrace a web 2.0 startup. I’ll be working with a great many talented and interesting people, and I’m looking forward to it immensely.

Meanwhile, TiddlyWiki has benefited from something rather magical: a global community of eager people who have gathered around it and generously contributed to it, striving to make it better in a spirit of good-natured sharing. I’m regularly astonished by the inventiveness and resourcefulness of this community; I feel a part of something much bigger and more significant than I could ever manage on my own.

I’ve always kept TiddlyWiki fiercely independent — for instance, not carrying advertising (or indeed accepting venture capital investments). I feel that to do anything else would be disrespectful to the grass roots users and enthusiasts who make TiddlyWiki so useful and intriguing. Now that I’m taking up a commercial position it’s necessary to take certain steps to enshrine that independence more formally.

I have therefore legally assigned my copyrights in TiddlyWiki to an open, non-profit foundation called UnaMesa. I think that TiddlyWiki is at once too fragile and too important to be wholly owned by any one player in the ecosystem; common ownership allows everyone to work together on a level-playing field. There’s a lot more to say about UnaMesa, and I’ll return to it in a later post.

I’m looking forward to being able to improve some areas of TiddlyWiki that have not received enough attention in the past - like a better plugin catalogue, automated testing, better accessibility, and easier security. This won’t be by BT taking over the project, but rather by supporting the open source process and helping out with resources when and where it can.

I hope BT’s endorsement of TiddlyWiki will open up new applications that we haven’t thought of yet. To meet the challenges that they bring, I’ll continue to strive to keep the core of TiddlyWiki true to it’s origins as a lean, efficient non-linear personal web notebook.

I’ve been using a couple of google groups for TiddlyWiki for two years or so, and there’s been a steadily accelerating spam problem. I now get sent loads of email notifications requesting manual moderation of potential spam messages — which are actually 99% not spam. Anyway, to keep within good response times for all the people who post to the group, as the volume of requests has got higher I’ve had to start adding other people as managers to help field the requests in the middle of the night.

The problem is that notification system for these moderation requests is email; now, when you get one of them there’s a good chance that one of the other volunteers has already got there and approved the message for you.

The situation has just got good deal worse, too: up until recently, the notification emails had the original message appended on the end. That meant that if you could already see the message in your inbox, you didn’t need to click through on the link because you knew that someone else had already approved it. Over the last couple of days, Google have changed things so that the original message doesn’t appear in the email notification message. This means that one has to click through every single time.

Now I’m in a quandary: the more people I add as managers to deal with the avalanche of spam, the more often all of the managers have to put up with the notification system firing blanks. The best solution I can think of would be if we had an RSS feed of the notifications - and ideally a single personal feed that aggregates all the groups that I manage.

http://groups.google.com/group/TiddlyWiki/pendmsg?hl=en

A slightly frustrating irony at the moment is that the page does actually already offer an RSS icon in Safari’s address bar, but that feed actually turns out to be the main group feed. Bah.

UPDATE: I’m now also getting reports of posts being wrongly filtered as spam, and not showing up at all. I wonder if it’s time to start looking for alternatives.

UPDATE: The more I think about it, the more frustrated I am that recently Google groups has managed to gain a bunch of shiny new features (like editable pages) but evidently little effort has been spent on basic spam control technology.