Treat with Jermolene

Today, my colleague Paul Downey has made an impressive contribution to Osmosoft’s mission to explain by preparing a paper about TiddlyWiki for submission to the W3C Workshop on Security for Access to Device APIs from the Web. It gives an overview of TiddlyWiki for technical audiences that already know and understand the mainstream web and its underlying technologies.

Paul’s paper can be found here:

http://osmosoft.com/~psd/TiddlyWikiDeviceAccess/

I’m loving it in lots of different ways. It’s a really solid example of what I call a “manifesto-style TiddlyWiki”, a tightly knit mesh of content that together communicates a system of belief. I like his tone, too, nicely sober and authoritative.

The challenge that Paul has taken on isn’t to be under-estimated. I’ve had the chance to talk about TiddlyWiki to many different audiences around the world, but the most unexpected reaction I encounter is from people in that same, technical audience and it is, bizarrely, outright disbelief. Picking it up in the middle of a demo, the conversation goes something like this:

Jeremy: Here I am clicking the ’save changes’ button, and now watch as I refresh the page in the browser, and you can see that the changes that I made have been persisted to disc

Expert: Right, that’s very interesting. What serverside technology are you using? I bet it’s Java, isn’t it?

Jeremy: No, actually, as I said at the beginning, TiddlyWiki is a standalone HTML file, with the application logic written in JavaScript

Expert: Ah, I see, so you’re using something like Rhino to run the JavaScript on the server?

Jeremy: No, not at all. TiddlyWiki runs entirely on the client browser, and takes advantage of special privileges accorded to HTML files loaded from “file://” URIs to be able to save itself directly to disc

Expert: That’s neat. So do you use Google Gears to handle going offline?

Jeremy: Sigh, no, not quite.

And so it goes, on and on. It sounds ridiculous, but it’s happened too many times or me to avoid thinking about it. It’s seems that the idea of a standalone HTML file is sufficiently far away from many people’s experience of the web for it to be invisible to them.

The frustration for me is because it feels as though TiddlyWiki has something important to say about the different things that happen if you let people hack the web without needing server infrastructure. My experience is that you can foster some intense innovation by radically lowering the barriers to the Copy/Modify/Share cycle that is at the heart of community innovation.

“SinglePageMode” is a typically neat little TiddlyWiki plugin created by Eric Shulman. It restores the more conventional wiki behaviour of having a single page open at once, so that clicking on a link replaces the current page, rather than inserting it below, as TiddlyWiki usually does.

Eric’s plugin has been deservedly highly popular. It seems that for a significant proportion of situations/users, the default TiddlyWiki behaviour of having multiple tiddlers at once is just too complicated. Having a community plugin to ‘fix’ this is a great solution, allowing TiddlyWiki users to choose which behaviour they prefer.

But, from an interface perspective, one of the things I find troubling about the current implementation of SinglePageMode is that it rather takes away the whole point of TiddlyWiki: after all, it started out as an experiment in having multiple wiki pages at once, with a view to making it easier to refactor content between pages. It’s also frustrating to have to have two separate modes of operation, with slightly inconsistent behaviour.

I’ve given quite a lot of thought to how one might somehow unify these two modes, to give the clarity of single page mode without giving up on the ability to quickly flick between multiple open tiddlers. I just came across some sketches buried in an old virtual machine from 2006 that show one approach that still looks like it might be interesting.

When a single tiddler is displayed, it looks much like an ordinary TiddlyWiki screen:

When a tiddler link is clicked on, the new tiddler opens in front of the previous one, building up breadcrumbs as a sort of rolodex of tabs:

Then the idea was that by riffling the mouse over the breadcrumbs you’d get a popup that allowed you flick back and refer to an earlier tiddler:

I’m not sure why I drew the popup at a reduced size, I’m not sure that that’s a good idea. I’d also intended to draw a frame showing a similar popup when hovering over any tiddler link.

I took some care to try to make it possible to implement this design with minimal changes to the TiddlyWiki internal Story object, it should be possible to do most of the work with CSS. I was imagining that there might be an ‘expand’ button over near the breadcrumbs that floops the various tiddlers out to their fully expanded size, looking like an ordinary TiddlyWiki.

Although I never implemented this design, I’ve not stopped thinking about these issues, and am exploring some of them in Project Cecily.

UPDATE: There’s some further discussion on the TiddlyWiki developers group.

I’m looking forward to speaking at next week’s Oxford Geek Night at The Jericho Tavern. I’ll be talking about TiddlyStuff, including Project Cecily, for the first time showing it off on the new Firefox 3.1 beta.

Gosh, I just blogged.

Osmosoft is the team of a dozen people that I lead within BT. It is the result of BT buying my tiny open source company in May last year. To the consternation of their corporate finance people, they did this despite there being no staff, no revenue, no sales, no customers in the conventional sense, and no “intellectual property” (the rights having already been made over to a non-profit organisation in Silicon Valley).

I was told that the rationale for the acquisition was something like this:

“We see innovation happening on the edges of open source communities, and we need to understand how it works”

It’s pretty refreshing that it talks about communities and innovation, and not, say, just something about lowering license fees. That statement was one of the reasons that I pursued the opportunity with BT. After a few conversations with my boss, I arrived at a basic set of four goals to sit under that rationale:

• Drive adoption and improve the BT Web21C SDK by developing open source applications to exploit it
• Build a world-class open source web development team, exploring how open source can make it easier to find and assess talent
• Contribute to the innovation and adoption of the TiddlyWiki system, exploring the central question of how innovation occurs in communities 
• Do all of these things transparently and publically, so that others in BT can follow us and learn from our mistakes

After a few months it became clear that we needed to improve our approach to the governance of open source usage and contributions within BT. I was given responsibility for it, and we added a fifth goal:

• Evolve governance policies and procedures to remove the internal obstacles to working with open source, to maximise the benefit from doing so, and to protect BT from any attendant risks

In doing all of that, we adopted a few basic principles to guide us:

• Show, rather than tell. The best way to learn about something is to experience it directly
• Communicate with our colleagues across BT using the public internet to ensure the widest audience, and to allow collaboration with other organisations involved in similar endeavours
• Put developers together in the same room to maximise the collaborative bandwidth
• Treat TiddlyWiki as a microcosm of the kinds of open source projects BT might be interested in, particularly in terms of the innovation that can be seen in the TiddlyWiki community

In subsequent posts, I’ll talk about our progress against those five goals, and some of the work we’re doing to evolve decent metrics for the success of our work.

The organisers have posted the video from the talk that I gave at the Next08 conference in Hamburg earlier this month. It doesn’t seem to be possible to link directly to the video; you’ll need to scroll down until you find “Jeremy Ruston”:

http://www.next-conference.com/next08/program.html

I was a bit more nervous than usual because I was standing in at short-ish notice for my boss. I added some laconicity, but the material is essentially his, as you can see in this video of JP giving the talk last December at LeWeb 3 in Paris. Osmosoft helped out with the slides, using RippleRap and TiddlyWiki to maintain my strict PowerPoint quarantine.

Happily at least one person liked it.

Along with several of my Osmonaught colleagues, I will be attending OpenTech 2008 in London on Saturday July 5th 2008. It’s a very low cost event (£5 to get in!), and is focussed nicely on the hacker community. I went to the last one in 2005 and had a great time.

This time, I’m giving a talk entitled “TiddlyWiki Tales” which will be a round-up the latest TW developments, both from Osmosoft and the wider community. Later on in the day, my colleague Paul Downey is talking about “The Web as Agreement“.

We’re also holding a competition with a TiddlyWiki flavour and some interesting prizes:

The TiddlyWiki OpenTech competition gives you the chance to win a BUG (plus four modules)! The BUG base station and modules use open source software and snap together to create whatever device you want. There’s also a great second prize of a GP2X Linux powered handheld games console. A stack of O’Reilly books will be the third prize.

For your chance to win one of these great prizes, all you have to do is show us your TiddlyWiki skills. TiddlyWiki is written entirely in javascript, and has a great plugin architecture, so just like the BUG you can twist it to do whatever you want. It can pull in RSS feeds, and because it can be a local file (and you don’t have to worry about same domain policies), it’s great for mashups. There are already 400+ plugins you can use, but don’t let that stop you from writing your own new ones!

(See the Osmosoft site for full details).

So, I’ll be attending BlogTalk 2008 next week in Cork, along with most of the Osmosoft contingent. I’ll be running a brief demo of some of the latest TiddlyWiki wizardry, and we’ll have a little exhibition stand too. Being a webby sort of conference, we’re running a RippleRap server so that people can create and share notes about the conference sessions (regardless of the state of the wifi!).

I went to BlogTalk 2006 in Vienna as an independent hacker, and thoroughly enjoyed my first exposure to a crowd of people who are generally at the forefront of thinking about social software. Thomas and John do a tremendous job of bringing people together, and so I’m delighted that this time around BT are one of the sponsors of the event.

So, here at Osmosoft been lucky enough to have access to an iPhone to play with for the last few weeks. Our first discovery was that TiddlyWiki works just fine on the iPhone – apart from being excruciatingly slow. TiddlyWiki is so meaty you really feel the 20x speed difference between a MacBook Pro and an iPhone. Anyhow, I’m confident that we can slim TiddlyWiki down to the extent that it runs at a reasonable speed on these new portable slow AJAX devices. (If you try the leaner and meaner first version of TiddlyWiki on an iPhone, it’s a good deal snappier).

The other problem is that saving doesn’t work properly because the TiddlyWiki is being accessed via http. We can expect that TiddlyWiki adaptations that support serverside saving (like TiddlySpot) will work, but that doesn’t really speak to the TiddlyWiki vision of untethered access to the information that you need.

I quickly discovered a potential solution: the iPhone supports bookmarks whose URL is a data URI. The procedure is to use something like Hixie’s data: URI kitchen to encode a file (HTML, images or PDFs all seem to work fine; possibly Word and Excel Documents too) into a data: link. Then right-click on the link and select “add to bookmarks”. You’ll end up with a bookmark that starts something like this:

 data:text/html;base64,PCFET0NUWVBFIGh0bWwgUFVCTElDICItLy9XM0MvL0RURCBY....

When that bookmark is invoked, the base64 data is converted back into a binary file, which is then displayed by the browser – all without recourse to the Internet.

The really interesting thing is that Safari/iPhone support bookmarks of up to 32 megabytes or so, which suggested the possibility of embedding an entire TiddlyWiki in a bookmark. And, it turns out that it works — giving you offline read-only access to TiddlyWiki files on an iPhone.

Here are the steps to convert any TiddlyWiki to use offline on the iPhone

1. Locate the TiddlyWiki file you want to convert (it can be stored locally or on the internet)

2. Go to Hixie’s data: URI kitchen in Safari

3. Make sure the ‘base64′ checkbox is enabled

4. Either enter the URL of the TiddlyWiki file in the text box, or use the ‘Choose file’ button to select a local file

5. Click ‘Generate’

6. Wait; it can be quite slow to do the conversion with a big TiddlyWiki file

7. Right click on the resulting link and select ‘Add Link to Bookmarks…’

8. Go to iTunes and make sure that you’ve selected to sync your Safari bookmarks with the iPhone

9. Sync the iPhone

10. On the iPhone, select the bookmark

This is all still quite rough and ready; there’s still a bunch of work to do to get a first-class TiddlyWiki experience on the iPhone. In my next post I’ll be looking at optimising the TiddlyWiki user interface for the iPhone, to make it look and feel more like a native application.

I’ve started regularly hanging out in the TiddlyWiki IRC channel. It’s an open chatroom that anyone can join; there’s half a dozen or so regulars, including FND and Saq, and several of my colleagues at Osmosoft.

The channel is called #tiddlywiki and it lives on the server irc.freenode.net.

IRC is quite old-school; there’s a bunch of different clients available for various platforms. I’m using something called Conversation for the Mac.

So, I started TiddlyWiki back in September 2004. It is now, by many measures, a moderately active and successful open source project. The forums are busy, the ecosystem is vibrant, and many people get a great deal of satisfaction and enjoyment from using TiddlyWiki, improving it, and helping others to do so.

Sometimes, people ask me how they can start a similar project, particularly how they can build an open source project around an idea that they’ve got. For a long time, the question used to make me stop short and give the glib answer that I didn’t know, because it had happened to me by accident. I’m acutely aware that I knew next to nothing about open source until long after I first published TiddlyWiki, and I felt that my experience was so specific to TiddlyWiki that it wouldn’t help anyone else.

Now, though, with the benefit of hindsight, there might by now be a few things I can generalise from my experience. Perhaps. Anyway, I’ll try for three things fairly simple observations for the moment.

1. You have to be fearless, if not reckless. In my case, for the two years or so that I was precariously working full-time on TiddlyWiki, I didn’t earn anything like the money I’d earned in my previous jobs. The only way I could survive was not to put away any money to pay the tax bills that I was building up but to spend everything I was earning on day-to-day living and bills. You can juggle things in the UK so that you don’t pay tax on your income for two years if you’re careful, but my cunning plan was clearly not something that a fully trained accountant would advise anyone to do. Especially given that I had no indication of any future payout that a happy-go-lucky angel investor would recognise, let alone one to impress the friendly tax man. Living hand-to-mouth is never fun, but I got huge motivational sustenance from the enthusiasm of the other people who were putting so much into the project. I reasoned that dozens of strangers wouldn’t be investing in it if TiddlyWiki were without serious merit, and my natural blind optimism filled in the leap that something of serious merit would eventually allow me to pay my bills.

2. You have to build the thing before they will come. An idea alone is not enough. In the case of TiddlyWiki the first proof-of-concept version that I released in September 2004 has around 850 lines of JavaScript code, much more than a trivial proof of concept or a wireframe sketch. I then worked on TiddlyWiki for around 9 months as a pretty much solo effort, driven by the bug reports and feature suggestions that I was receiving. For all of that period, it would have been hard for me to describe TW as an open source project in anything other than license terms (back then there was no subversion repository, no mailing list and no bug tracking system). It was only when TiddlyWiki itself was mature enough to sustain a user community that the demand built up for it to become a proper open source project.

3. You have to make it easy for your users to find and use your source code. In the case of TiddlyWiki, the website is the product is the code is the documentation, which has neatnesses of its own. But there’s more than that: one of the things I’ve learned from working with the TiddlyWiki community is that there are an awful lot of almost-programmers out there: people who couldn’t reliably tell you much about the syntax of JavaScript but have enough wit and common-sense to splice and modify samples of code. Especially if they have the means to try out their changes easily and safely. Give them these tools, and these people can do wonders. In contrast, many open source projects have much more complicated installation, build and execution requirements. It’s often surprisingly hard just to locate the source code and browse your way to a particular part of the codebase.

A great example of how successful TiddlyWiki has been at lowering the barriers to contribution is the Digital Supplement for “Web Campaigning” by Kirsten Foot and Steven M. Schneider” (MIT Press 2006). The authors have published their source materials as a TiddlyWiki file that fellow researchers can download and use to begin their own work, use as a template or indeed study while on an aeroplane. The really interesting thing is the way that Steven assembled their own unique customisation of TiddlyWiki, consisting of a handful of off-the-shelf plugins and one that he wrote himself to handle their unique requirements for footnotes and endnotes. The macro isn’t particularly complicated in programming terms, but it radically simplifies the annotation of data, and demonstrates a beautiful blurring of boundaries between programming and writing.