July 24, 2008

Simon Willison

Dojango version 0.3 released

Dojango version 0.3 released. A reusable Django application that provides Dojo, helper functions (dojo.data integration) and tools for switching between Dojo versions.

July 24, 2008 12:47 AM

July 23, 2008

JP Stacey

Remember, remember

The recent pitiful release of Home Office yoof shilling site My Life My Id came to few people’s attention—why should it, being a pointless exercise in “consultation”?—but those who did spot it were mostly No2ID aficionados. They moved in rather swiftly, and the “recent posts to the forums” on the front page quickly vanished, as forum post after forum post was desperately moved (for which read: brushed under the proverbial carpet) into a “miscellaneous” forum.

Being a geek, though, I was most surprised to spot that they were using Drupal; in fact, they’re on the mostly patched Drupal 5.7 version at the time of writing. This piqued my interest further, and I emailed No2ID to mention that I’d been reminded to join them—which I ought to have done years ago—by the very people whose perniciousness they were trying to militate against.

At the same time, I remembered from OpenTech that No2ID had begun a recruiting drive, and I was sure that Simon had blogged about it but couldn’t at the time find the post. I actually asked Guy Herbert at No2ID about this, and rather politely he explained that, no, it wasn’t them: it was the Open Rights Group. He refrained from adding “you buffoon.” But now, thanks to mylifemyid.org, I’m a member of both No2ID and ORG, two organisations I should have joined long ago.

In all its years of oppression, subjugation, scaremongering and offences against the person, the Home Office has never spread so much love. Thanks, Home Office: you’re a pal!

by jps at July 23, 2008 07:30 PM

Simon Willison

Quick OAuth Notes

Quick OAuth Notes. Yesterday’s XMPP Summit resulted in a proposed standard for using OAuth to authenticate XMPP streams.

July 23, 2008 06:14 PM

window.name Transport

window.name Transport. The cleverest use of the window.name messaging hack I’ve seen yet: Dojo now has dojox.io.windowName.send for safe, performant cross-domain messaging.

July 23, 2008 04:25 PM

How Dopplr learns

How Dopplr learns. Dopplr uses global and personal trip histories to disambiguate place names, and your friends’ schedules to help disambiguate dates in airline confirmation emails.

July 23, 2008 04:17 PM

A quote from Fredrik Lundh

(It’s probably just me, but every time I stumble upon some thread involving people from the so-called “security community”, it’s like watching a Jerry Springer episode.)

- Fredrik Lundh

July 23, 2008 09:28 AM

Drizzle, Clouds, "What If?"

Drizzle, Clouds, “What If?”. Exciting news in the world of MySQL: Drizzle is a new project to produce a massively stripped down version of the database server—InnoDB/UTF8 only, no permissions, views, stored procedures or triggers, simplified field types, optimised for the common subset of functionality used by web apps. MySQL’s Firefox?

July 23, 2008 12:30 AM

July 22, 2008

Graham Lee

Microblogging

For a long time, I deliberately avoided microblogs like twitter. I thought that they were simply an acknowledgement that people want to be published more than they want to have something to say. However, it would be rude of me to completely disavow the medium without actually giving it a go.



To that end, I may indeed be iamleeg on twitter, as soon as twitter actually finishes processing the signup form.



I'd like to point out that one problem I'm going to have is brevity - I have spent 650 characters telling you what my username is. Constraining myself to SMS-sized wibblings will indeed be tricksy.

by leeg (noreply@blogger.com) at July 22, 2008 09:33 PM

Simon Willison

Email Address to URL Transformation (EAUT) specification now available!

Email Address to URL Transformation (EAUT) specification now available!. Allows OpenID users to login using their E-mail address, which is converted in to an OpenID URL based on rules specified in an XRDS document attached to the root domain. Seems like a good idea to me.

July 22, 2008 07:30 PM

Python BoF and Django Drinkup

Python BoF and Django Drinkup (via). At OSCON? Come along to the Jax Bar tonight (Tuesday 22nd) from 7pm to 10pm to hang out with fellow Pythoneers and Djangonaughts.

July 22, 2008 06:48 PM

Replacing Django's Template Language With Jinja2

Replacing Django’s Template Language With Jinja2. Part of Wil Larson’s series on taking advantage of Django’s loose coupling.

July 22, 2008 05:18 PM

ComicVine.com

ComicVine.com. Also powered by Django, Whiskey Media’s comic book encyclopedia and community. 43,000 characters and 94,000 issues and counting.

July 22, 2008 07:12 AM

GiantBomb.com

GiantBomb.com. Launched today, powered by Django—a combination of (mostly ex-Gamespot) quality editorial content and a massive structured wiki of every computer game ever released. This is going to be a lot of fun—all of the crazy detailed content that Wikipedia tends to reject.

July 22, 2008 07:09 AM

Django 1.0 alpha release notes

Django 1.0 alpha release notes. The big features are newforms-admin, unicode everywhere, the queryset-refactor ORM improvements and auto-escaping in templates.

July 22, 2008 06:04 AM

Django 1.0 alpha released!

Django 1.0 alpha released!. Not meant for production use, but a pretty solid preview of what’s coming in 1.0 proper. The beta is scheduled for August 5th.

July 22, 2008 06:01 AM

What the Heck is the Open Web?

What the Heck is the Open Web?. Brad Neuberg is seeking a two sentence definition. Bonus points for answering the following: “If Adobe were to open source Flex/Flash, or Microsoft Silverlight, would that be the Open Web? If so, why? If not, why not?”

July 22, 2008 01:33 AM

July 21, 2008

Simon Willison

New feeds for Project hosting on Google Code

New feeds for Project hosting on Google Code. Finally! I’ve been wanting these ever since Google Code launched. Should make it much easier to integrate personal projects hosted on Google Code in to tumblelog style sites.

July 21, 2008 09:45 PM

Graham Lee

Common sense writ large

Looking at the bottom of Apple's Q3 results, as indeed with any similar publication from a US publicly-traded company, we see the following text.



This press release contains forward-looking statements including without limitation those about the Company’s estimated revenue and earnings per share. These statements involve risks and uncertainties, and actual results may differ. Risks and uncertainties include without limitation potential litigation from the matters investigated by the special committee of the board of directors and the restatement of the Company’s consolidated financial statements; unfavorable results of other legal proceedings; the effect of competitive and economic factors, and the Company’s reaction to those factors, on consumer and business buying decisions with respect to the Company’s products; war, terrorism, public health issues, and other circumstances that could disrupt supply, delivery, or demand of products; continued competitive pressures in the marketplace; the Company’s reliance on sole service providers for iPhone in certain countries; the continued availability on acceptable terms of certain components and services essential to the Company’s business currently obtained by the Company from sole or limited sources; the ability of the Company to deliver to the marketplace and stimulate customer demand for new programs, products, and technological innovations on a timely basis; the effect that product transitions, changes in product pricing or mix, and/or increases in component costs could have on the Company’s gross margin; the effect that product quality problems could have on the Company’s sales and operating profits; the inventory risk associated with the Company’s need to order or commit to order product components in advance of customer orders; the effect that the Company’s dependency on manufacturing and logistics services provided by third parties may have on the quality, quantity or cost of products manufactured or services rendered; the Company’s dependency on the performance of distributors and other resellers of the Company’s products; the Company’s reliance on the availability of third-party digital content; and the potential impact of a finding that the Company has infringed on the intellectual property rights of others. More information on potential factors that could affect the Company’s financial results is included from time to time in the Company’s public reports filed with the SEC, including the Company’s Form 10-K for the fiscal year ended September 29, 2007; its Forms 10-Q for the quarters ended December 29, 2007 and March 29, 2008; and its Form 10-Q for the quarter ended June 28, 2008, to be filed with the SEC. The Company assumes no obligation to update any forward-looking statements or information, which speak as of their respective dates.



Erm, like, duh. Stuff which we say might happen in the future, might not actually happen. Really? You've got to get out of the financial industry, there's a lucrative career ahead of you in construction.

by leeg (noreply@blogger.com) at July 21, 2008 09:04 PM

Simon Willison

MySpace To Join OpenID, Bringing Total Enabled Accounts to Over A Half Billion

MySpace To Join OpenID, Bringing Total Enabled Accounts to Over A Half Billion. Another 200 million OpenIDs—but the important difference between this and the Yahoo! and AOL announcements is that MySpace users know what their profile URL is. Whenever people have told me OpenID is flawed because people don’t understand URLs I’ve answered “sure they don’t, but they know their MySpace page”.

July 21, 2008 07:42 PM

Ben Werdmuller

Elgg in education

Elgg has a long history of being used effectively in education. The University of Brighton made headlines a couple of years ago when they used Elgg to become the first institution in the world to roll out a social network to students and staff across campus, which is now being emulated throughout the UK higher education sector. In California, Jim Klein has an award-winning Elgg network for the Saugus Union school district, which is also serving as a model throughout the country.

So why is Elgg suited for education, and how can you use it?

A lot of the software designed for an e-learning context is aimed at formal learning: classes, courses and submissions. However, there is a lot more that goes on in an institution, which is rarely captured or harnessed. If you look back on your university career, you'll probably remember that some of the most valuable experiences were the things that happened around the edges: study groups, hallway interactions, unofficial discussions with tutors.

Elgg captures both formal and informal learning beautifully, but also - through cross-site tagging - clusters content together in useful ways. You can find documents, students or colleagues relating to the topics you're interested in; you can set up a collaborative group for you and three other researchers; you can create groups for classes and allow students to both discuss the content and discover related resources. Students maintain control of their own digital identities, and can create customised profiles that truly reflect their personalities and interests. They can blog, upload photos, record podcasts, participate in forum discussions and manage tagged wiki pages. All this while maintaining strict, powerful access permissions (you can control exactly who can see any given resource) and maintaining an audit trail using a full activity log.

In an industry first, Elgg also has full import and export capabilities using the Open Data Definition. That means students, if you allow it, can take their content from one Elgg community to another, for example if they change institutions. If you have to look after several different communities, you can also use the Open Data Definition to aggregate activity across them. They could be running Elgg, or any platform that supports the Open Data Definition.

Elgg's interface is extremely easy to use. In fact, we think it's by far the easiest social networking environment to master, which leaves education professionals to get on with the business of teaching and learning. Speaking of which, while use of Personal Learning Environments, digital storytelling and e-portfolios are all examples of how Elgg can be used in education, Elgg also works very well as a way to enhance traditional, face-to-face learning. We've written the software; it's up to you to decide what to do with it. Forthcoming features will include wikis, SMS functionality and more, so there's a lot of power at your disposal.

It doesn't just have to be about learners and tutors. Jim Klein uses Elgg effectively between teachers at the Saugus Union school district, and many others are following his lead. It's a great way to share teaching resources and experience, securely and efficiently. The network is closed, and you can further restrict resources to be read by as small or as large a group as you please, so you can be sure that your discussions are secure.

We're going to be talking a lot more about Elgg in education. If you'd like to find out more, please get in touch, either here in the comments or at info [at] elgg.com.

July 21, 2008 06:18 PM

Andrew Godwin

LastGraph Fixed

LastGraph has been fixed, at least as far as I can tell; the last.fm team fixed the API bug pretty quickly on Monday, so thanks to them.

If your profile was affected (that is, if its timeline is flat), email me and I’ll delete its data file.

Finally, I would have ended with a cheap phrase like “Keep on LastGraphing!”, but that would have been cheesy and silly.

by Andrew Godwin at July 21, 2008 05:08 PM

July 20, 2008

Simon Willison

newforms-admin branch has been merged into trunk

newforms-admin branch has been merged into trunk. Congrats to Brian Rosner for the merge. django.newforms has been renamed to django.forms as well—1.0 grows ever closer.

July 20, 2008 11:17 PM

DjangoCon 2008

DjangoCon 2008. The official DjangoCon site is up, along with a mostly complete schedule.

July 20, 2008 08:49 PM

A quote from Danny O'Brien

If we want people to have the same degree of user autonomy as we’ve come to expect from the world, we may have to sit down and code alternatives to Google Docs, Twitter, and EC2 that can live with us on the edge, not be run by third parties.

- Danny O'Brien

July 20, 2008 09:00 AM

July 19, 2008

Simon Willison

Jinja2 Final aka Jinjavitus Released

Jinja2 Final aka Jinjavitus Released. The Jinja template engine now has auto-escaping as an optional feature, disabled by default. Worth considering as an almost drop-in replacement for Django’s template language if features such as macros and compilation to Python code appeal to you.

July 19, 2008 11:52 PM

We're added some iCal views to BBC Programmes

We’re added some iCal views to BBC Programmes. BBC /programmes is one of the most exciting public facing parts of the BBC’s online offerings.

July 19, 2008 11:48 PM

HeatMapAPI

HeatMapAPI (via). Cool (or should that be hot?) API for adding heat maps to any Google Maps application.

July 19, 2008 11:46 PM

The Truth about Web Navigation

The Truth about Web Navigation. Jeremy Zawodny on regular users understanding the browser address bar: “They don’t. And they never will.” Then they’re going to get phished, and there’s absolutely nothing we can do to help them.

July 19, 2008 11:42 PM

Simple Top-Down Parsing in Python

Simple Top-Down Parsing in Python. Eye-opening tutorial on building a recursive descent parser for Python, in Python that uses top-down operator precedence.

July 19, 2008 11:37 PM

php: rfc: closures

php: rfc: closures (via). I never thought I’d see the day, but a patch adding closures to PHP has been both proposed and accepted! Looks like a solid implementation—the syntax is similar to JavaScript but makes explicit which variables are to be captured. As with much of PHP, values are copied in to the closure by default but you can use an ampersand to specify JavaScript-style pass-by-reference instead.

July 19, 2008 10:58 PM

Andrew Godwin

Yay, backwards compatability…

It seems that the launch of last.fm’s new site has strangely broken the older API that LastGraph uses. I knew I was going to have to update to the new API, but I didn’t think it would be this soon!

For the two eagle-eyed people who’ve contacted me about this, well done - I’m working on a fix now (the precise problem is that the old API now always returns the last week’s data, no matter when you ask for, hence the very flat graphs).

Update: After a quick three minute URL change, it seems the new one is ALSO broken, and other things such as the lovely Last.fm Spiral are broken too. Time to go make sure they know about this at last.fm HQ.

by Andrew Godwin at July 19, 2008 01:16 PM

July 18, 2008

Graham Lee

Designing a secure Cocoa application

That's the title of next month's CocoaHeads Swindon, and I'll be leading the presentation/discussion. So if you want to learn a little about how to ensuring your Cocoa app doesn't give away the keys to the kingdom, or have some experiences to share with the rest of the group, come along! We'll be at the Glue Pot, which is nice and near the train station as well as reasonably close to a car park. We'll be congregating at 7:00 but will wait for everyone to be settled with a beer in their hand before starting ;-).

by leeg (noreply@blogger.com) at July 18, 2008 08:56 PM

Jon Hicks

Listen with Delia

86385

Delia Derbyshire has long been considered ahead of her time. One of the earliest creators of electronic music, she is most famous for her work with the BBC’s Radiophonic Workshop in Maida Vale, at which she created the haunting original Dr Who theme, in an age without synthesizers:

A lot of modern musicians like Orbital, Stereolab and Spacemen 3 cite her work as an influence, and it’s easy to to see why.

What sparked this blog post was the news that more of her work has been discovered, some 267 tapes to be exact! All of this is going to be digitized and made available, but in particular was this experimental dance track that she created in the 60’s. Made decades before ‘electronic dance music’ really happened, and yet it sounds like something created today.

This clip from a BBC Four documentary gives some insight into how she created music with reel to reel tapes:

Sadly she died in 2001 at 64, just after rediscovering her love of electronic music, working with Peter Kember (Spacemen 3, Sonic Boom), of which she said:

“Working with people like Sonic Boom on pure electronic music has re-invigorated me. He is from a later generation but has always had an affinity with the music of the 60s. Now without the constraints of doing ‘applied music’, my mind can fly free and pick-up where I left off.”

Discover more about her life at delia-derbyshire.org

dave-judgement_delia_derbyshire

by Jon Hicks at July 18, 2008 07:43 PM

Garrett Coakley

What we look like from out there

Sometimes you see things that make the hairs on the back of your neck stand up.

The Deep Impact spacecraft, with it's original mission complete and now undertaking science under the EPOXI moniker, turned it's cameras back towards the Earth from 50 million kilometers away and over the course of several hours caught The Moon transiting The Earth!

Phil Plait sums it up beautifully:

While there is science galore in these animations, I think their real impact is the visceral one from simply seeing them. As Carl Sagan once said: everyone you have ever met, every human who has lived and died, lived out their lives on that blue ball. And yet here we are, in the 21st century, plains apes allowed to evolve and satiate their curiosity, now with the ability to lob metal proxies into deep space, look back, and see ourselves.

Science. I love this stuff.

by garrett at July 18, 2008 12:52 AM

July 17, 2008

JP Stacey

Firefox/Sage bookmarks to Google Reader import

Want to migrate your RSS bookmarks from Firefox (or its RSS-reading addon, Sage) to Google Reader? I did, just now.

Christopher Hinze has written a great Firefox addon that exports bookmarks to OPML 1.0. Unfortunately, OPML is a bit of an anything-goes specification. So although Hinze’s plugin produces valid OPML, it isn’t the same sort of valid OPML that Google Reader expects. Google Reader, in fact, gags and chokes on Hinze’s OPML, and refuses to import it.

The main problem is that the <outline/> element, the basic hierarchical building block for OPML, will take any attributes. What does that mean in practice? Well, here’s what Hinze’s export produces:

<outline text=”Coding”>

  <outline type=”link” text=”Joel on Software” url=”http://www.joelonsoftware.com/rss.xml” />

</outline>

and here’s the result of Google Reader exporting its own store of RSS bookmarks:

<outline title=”Coding” text=”Coding”>

  <outline text=”drupal.org - Community plumbing”

    title=”drupal.org - Community plumbing” type=”rss”

    xmlUrl=”http://drupal.org/node/feed” htmlUrl=”http://drupal.org”/>

</outline>

To a computer, these are fundamentally two different data formats: the URLs are stored in different attributes, and there are attributes on each that either have different values or are not present on the other. Someone did a DTD for OPML: looking at those two apparently analogous fragments above you have to ask yourself why they bothered.

Help is at hand, though. This sort of problem is bread and butter to XSLT, and here’s an XSL transform for converting Firefox OPML to Google Reader OPML. If you have xsltproc installed on your system, you would type:

xsltproc http://www.jpstacey.info/applications/google/ff2gr_opml.xsl bookmarks.opml > fixed_bookmarks.opml

Or download the XSLT—it’s released under GPL2—and run it locally, changing that URL there to a local file location.

One thing to note: the XSLT will remove an outline wrapped around your bookmarks with title “Sage Feeds” (case-sensitive). So you can export that branch of your bookmarks, and the XSLT will strip the wrapper off and you won’t import a load of bookmarks tagged “Sage Feeds”. If you don’t like this behaviour then either rename your Sage bookmark container, or learn XSLT: it won’t kill you.

by jps at July 17, 2008 07:04 PM

July 16, 2008

JP Stacey

The Straight Edge minimalist Wordpress theme

As promised, I’m releasing the Straight Edge theme used on this blog under GPL2.

There’s a brief README.txt in the zipped archive linked above, but the theme’s main features are:

  • XHTML compatible (in core theme files)
  • Minimal, semantic markup
  • No sidebar
  • Excerpts on archive and category pages
  • Implicit RSS feeds: the only orange icon is in your browser chrome
  • Adaptive top navigation
  • Separate pages for archives, categorisation and blogrolls
  • Next/previous rel links in header
  • Support for special pages e.g. blogroll, tag cloud

The todo list includes:

  • Implement my Blogthis! plugin, while trying to keep minimalist
  • Unobtrusive hiding of elements, using jQuery
  • Improve styling

The theme is in a fairly alpha state. The PHP is fairly straightforward, apart from some neat theme functions, but don’t blame me if everything goes bang.

by jps at July 16, 2008 09:19 PM

Hardy Heron and the Dell Precision M4300

In brief: the problems discussed here and here go away under the most recent Ubuntu release, Hardy Heron, which I can generally recommend.

Alsa seems stable and graphics support is present from installation onwards. Enabling fancier 3D compiz effects requires the nvidia-glx-new package; compiz spots this, however and prompts for installation. All very smooth. Wireless works; my VoIP headset works; but I haven’t yet tested Bluetooth.

The only problem was in upgrading from Gutsy: my previous peregrinations had rendered my hybrid distribution shafted and incapable of upgrade. This isn’t a problem, though, if one has installed the /home directory (and in my case the /music one too) on a separate partition: the Ubuntu Live CD will blat the root partition with Heron, but leave the other partitions alone if you so require. Don’t resize any of your partitions during installation, though, or you’ll lose everything. Everything!

by jps at July 16, 2008 07:16 PM

July 15, 2008

Ben Werdmuller

Translations and views in Elgg 1.0

The new Elgg has been open for beta testing from the user's side for a week or so now, and we've had a lot of very valuable feedback that's helping us shape the system and fine tune the way it presents itself to users. Let us know if you want to join in by sending an email to info at elgg.com.

During this period, I've been asked two questions more than any others, and I thought I'd deal with them both here. They're explored in more depth within Elgg's documentation, which we'll be making available soon.

How do translations work in Elgg?

For Elgg 1.0, we've moved to a more lightweight translations system. That's lightweight in terms of drain on system resources; it's not any less fully-featured.

The core engine and each plugin have their own set of language files, which translate from a simple identifier to the full-text string. For example, in a plugin you might call:

elgg_echo('myplugin:saved:successful');

In the English translations file this would be defined as:

'myplugin:saved:successful' =>

'Your example plugin object was created successfully.'

This way, because full-text string are separate from the logic or display of the site, translations can easily be generated for new languages without worrying about how this will impact design or logic flow.

How do I create an alternative interface, for example for mobile devices or the sight impaired?

This separation goes deeper between logic and display. Elgg has its own advanced template API, which allows the display of entities to be distinct from how they're stored or acted on. Each object may have a standard HTML view, as well as other views for RSS, mobile devices, accessibility-optimised sites and so on.

Because this blog is powered by Elgg, it's a great way to see this process in action. By default, you view the list of blog posts in the standard HTML view. However, by adding ?view=rss to the end of the URL, we flip into the RSS viewtype, and you see the RSS views on everything, from the shell of the page (which becomes the shell of the RSS feed) to each blog post. The logic to display both versions is exactly the same; it's only the display that differs.

July 15, 2008 11:36 AM

July 14, 2008

Graham Lee

Ah, the sweet sound of my own voice

The title is a linky to the press release regarding the edition of Sophos Podcasts I recorded with Carole, and which has now (clearly) gone live. In it we mainly talk about the data theft and Macs technical paper I've already posted about. This is the first podcast I've ever been involved in, so any feedback you have (apart from telling me that I sound drunk in my first sentence, I'm not sure what that's all about) will be most welcome!

by leeg (noreply@blogger.com) at July 14, 2008 01:04 PM

JP Stacey

Give a green finger

People at Torchbox, both singly and as a company team, are committed to helping fight climate change. As a web company, we think we can help most (in our small way) by helping our clients, especially environmental charities, fulfill their potential on the web, and by encouraging green behaviour among our staff: cycling breakfasts, cheap-bike schemes, car sharing, not flying to meetings or on work jollies.

Still, fighting the good green fight can feel pretty lonely, and sometimes you forget not just how many people are on your side in the next office or town, but also of the billions of people who have something to lose to climate change. So as a bit of fun, and to remind ourselves what it’s all about, we’ve recently given the green finger to climate change. Here we are, doing just that:

Want to join in? You can respond to the original YouTube video.

by jps at July 14, 2008 08:39 AM

July 13, 2008

Jon Hicks

Geneviève Gauckler on CBBC?

I’m big fan of the French artist/illustrator Geneviève Gauckler, having discovered her work via the Guardian Angel Room project.

Now, I’m not sure why, but I was really surprised to see her work (or at least her style) in the backgrounds of CBBC, the site for BBC’s youth output.

I’ve tried to do a bit of searching, but no information is forthcoming, and I can’t see a credit anywhere. I’m surprised, can anyone tell me if these really are her work?

bg-1

bg

There’s also wallpapers available too:

1280_games_wallpaperfair

by Jon Hicks at July 13, 2008 09:01 PM

Simon Willison

A quote from Robert Lofthouse

DjangoCon 2008. Venue: Gooleplex, San Francisco Bay Area. Dates: 6th and 7th Sept. Official post will be on djangoproject.com soon.

- Robert Lofthouse

July 13, 2008 04:50 PM

July 11, 2008

Ben Werdmuller

Embrace the chaos

In its simplest form, the Web is one of the biggest ideas ever created by human civilization. That's not hyperbole - the idea of an interlinking, distributed publishing network that anyone can write to and anyone can read, no matter who they are or where they are in the world, has enormous tranformative power. It's a leveller, an enabler, a force that has the capability to reshape global society into a more democratic system. Yet the biggest money has been invested in taming it rather than harnessing its full, chaotic potential.

Facebook is a walled garden; I think we can all accept that. Aside from getting us to the front door, the Web plays very little part in its success - it shares more in common with CompuServe or Prodigy than an interlinking, distributed web of conversation and information. Their application platform was a genuinely innovative step on first glance, but appears to have been expertly designed to make sure people add their content to Facebook rather than anywhere else. This isn't a criticism - it's a good business model, at least in the short to medium term - but I don't think it's in keeping with the spirit of the Web, and they'll fall out of favour as soon as the next big thing comes along.

More and more, this year has been about privacy and openness in the tech press. The result has been initiatives like OpenSocial, Facebook's open sourced platform and MySpace's Data Availability initiative. OpenSocial has huge promise as an idea, but each version of the standard isn't backwards compatible with the last, and the vast majority of widgets appear to have been coded for Ning. (Plaxo also supports OpenSocial, but they need to manually whitelist your app before it can be made available.) Facebook's open source platform requires you to compile Firefox 2.0.0.4 on the server - and therefore its own dependencies and requirements - which make it a dubious choice for most systems (we're experimenting with it; here's the PDF release notes). MySpace's Data Availability initiative requires that you don't ever cache any of the data it sends you.

In very obvious ways, these are all token gestures, aimed more at PR and stock prices than any kind of real openness. They're boxes to check off rather than a statement of belief in the platform and how it should be.

Yet this is a platform that has the potential to change the world we live in. For me, what we do is my way of trying to make the world slightly better than how we found it. It's a way of pushing things forward, not just technologically, but from a very human angle. That's why Elgg is open source. That's why we're developing the OpenDD. That's why we've been using open standards, shouting about data portability and enabling user choice since we began.

At the same time, we're not a charity. I think there are business consequences to accepting the nature of the Web as well: taming the Web is like taming a herd of infinite cats. Anyone can build a new page on the Web, anywhere, in any language and using any formatting they choose. It's called freedom of speech. You can do your best to erect fences and force people to walk distinct lines, or you can create ways for them to do what they want to do, share what they want to share, connect with the people they want to connect with. It should be easy; it shouldn't be elitist; it should be the path of least resistance to doing what people want to do. I believe that companies that embrace the chaos - and let you interact with it in a manageable way - will outlive the ones that don't.

July 11, 2008 07:41 PM

The Elgg data model

As we bring in some beta testers and kick Elgg 1.0's tyres, I thought I'd write an overview of the architecture and some of the decisions we've made. Be warned: this is a very technical post!

Everything we've done has been to work towards the dual goal of making the system more flexible, and making it the easiest social networking platform to work with. This means coding plugins is faster, and that you can do more with them, and that end users also see the benefits through extensive interface improvements and more consistent design.

Entities and data

One of the biggest improvements we've made to Elgg is the data model. Previously, plugins were responsible for introducing their own database tables, which meant that their data was effectively placed in a ring-fenced bucket away from everything else. They could plug into search, but those results were segmented; there was a section for blog posts below a section for users, and so on.

In the new Elgg codebase, everything runs on a unified data model, based on atomic units of data called entities. Plugins are strongly discouraged from dealing with database issues themselves, which makes for a more stable system that also has visible benefits for the end user. Content created by different plugins can be mixed together in consistent ways, which are programmed using generic principles - in other words, plugins are faster to develop, and are at the same time much more powerful.

So what does it look like?

Elgg classes

(Click for larger.)

Every entity in the system inherits the ElggEntity class. This controls access permissions, ownership and so on; Elgg 1.0 allows you to run multiple sites on the same install, so it also stores the site each element belongs to.

ElggEntity has four main specializations, which provide extra properties and methods to more easily handle different kinds of data.

  • ElggObject - objects like blog posts, uploaded files and bookmarks
  • ElggUser - each user in the system
  • ElggSite - each site within an Elgg install
  • ElggGroup - multi-user collaborative systems, which were called Communities in prior versions of Elgg

Each of these have their own properties that they bring to the table: ElggObjects have a title and description, ElggUsers have a username and password, and so on. However, because they all inherit ElggEntity, they each have a number of core properties and behaviours in common.

  • A numeric GUID.
  • Access permissions. (When a plugin requests data, it never gets to touch data that the currently logged-in user doesn't have permission to see.)
  • An arbitrary subtype. For example, a blog post is an ElggObject with a subtype of "blog". Subtypes aren't predefined; they can be any unique way to describe a particular kind of entity.
  • An owner.
  • A site.

Each of these also implements an interface called Loggable, which means that actions relating to it are stored in the system log (and potentially displayed in the profile river), and one called Exportable, which means that it can be represented using the Open Data Definition, as well as JSON, serialized PHP and XML.

Everything in the system can also have metadata and annotations attached to it, which in turn have an owner, access permissions etc. Metadata is set-once data like tags, profile items, etc; annotations are things like comments and ratings.

Relationships between entities

One of the most interesting things we've done is allowed generic relationships to be estabished between any two entities. A very common example is the relationship called "friend", which always occurs between two users - but there's no reason it has to end there. The system allows developers to establish an arbitrary relationship between any two things in the system, following a familiar pattern: subject, predicate, object. These could then be traversed by a plugin to help create accurate recommendations, or perhaps to fine-tune the accuracy of search results. We think that social usage data can help establish relationships behind the scenes, as well as weight them, and we'll be doing more work on that as time goes on.

Views and templating

Of course, none of this matters if you can't get the data to the user in the way you need to. Luckily, Elgg comes with a powerful templating system.

Each entity in the system can be displayed using a view. Each view has a name, which can potentially be split into subdirectories; for example, 'css', 'page_elements/header' and 'object/blog' are all valid view names. In turn, there's the concept of a view type - by default this is standard HTML, but it could be mobile HTML, RSS, etc. Just as you can create your own views, you can create your own view types. Views are stored in subdirectories under their view type - so a standard HTML version of 'object/blog' is stored in 'views/default/object/blog.php'. The RSS version is then contained in 'views/rss/object/blog.php'. Plugins can extend views or replace them with their own data.

Everything in the system knows how to draw itself; if we're trying to view an ElggObject of subtype 'blog', the system will automatically look for a view called 'object/blog'; similarly, 'user/admin' and so on. Elgg is smart enough to take a stab at creating RSS representations of every entity. (Plugin authors can override the default views of their particular subtypes of entity.)

To get a representation of an entity, you can call:

$fomatted_entity_for_display = elgg_view_entity($entity);

To get a paginated list of blog posts, you can simply call:

$formatted_list = list_entities('object','blog');

And as if by magic, the RSS feed for the same page will be generated for you and linked up in the head section of the page so that the user's web browser knows it's there. View and logic are kept separate in the new Elgg, so creating a mobile web view, RSS, JSON or other versions of the same page are extremely simple. With common formats like RSS, we've made that even simpler, by doing most of the legwork for you.

Because all entities are essentially the same, you also don't need to do anything at all to hook your features into search. If you search for 'foo', you'll get a list of everything matching foo, in one, unified list. (With RSS, natch.) Because access permissions are integrated at a very deep level, users only ever see the items they're been given access to - so the same search will yield a different results set when you're logged in to when you're browsing anonymously.

July 11, 2008 07:31 PM

July 10, 2008

Jon Hicks

Opera Web Standards Curriculum

wsc_468x60

Gawd Bless™ Chris MIlls and Gawd Bless™ Opera, for together with authors like Norm they have begat the Opera Web Standards Curriculum.

I linked to this in my sidenotes a couple of days ago, but really felt it deserved a bit more attention, considering the extraordinary amount of work that has gone into it.

Opera’s new Web Standards Curriculum, released in association with the Yahoo! Developer Network, is a complete course to teach you standards-based web development, including HTML, CSS, design principles and background theory, and JavaScript basics. It already has support from many organizations (including Yahoo! and the Web Standards Project) and universities. The first 23 articles are currently available, with about 30 more to be published between now and late September.

Finally, there is non-reference resource to point people to, where they can actually learn in stages what web standards mean, and how to adopt and use them to build a better web. Heartily slapped backs to all of the contributors!

by Jon Hicks at July 10, 2008 07:55 PM

JP Stacey

OGN7 writeup at the Google OS Blog

The writeup for OGN7 is live on the Google OS blog. If you were there, then you probably won’t learn anything new; if you weren’t, then what was your excuse?

by jps at July 10, 2008 09:18 AM

July 09, 2008

JP Stacey

The Mashup Song

Inspired by the title of the relevant Slashdot article, to the tune of My Bonny:

Your mashup is probably legal.

Your mashup is probably sound.

Your mashup is probably legal,

So pass all that data around!

Stuff here

Stuff there

And something mashed up in between (be-tween!)

Stuff here

Stuff there

And something mashed up in between

Your mashup is probably legal;

You could monetize it as well!

But though I contend it’s all legal,

Remember I-A-N-A-L!

[Repeat chorus]

by jps at July 09, 2008 11:45 AM

Simon Willison

ftputil

ftputil. Python’s built-in ftplib is ridiculously low level, requiring you to send RETR commands and even assemble downloaded chunks yourself using a callback. ftputil looks like a really solid high-level interface to that module with file-like objects and plenty of convenient methods.

July 09, 2008 10:51 AM

July 08, 2008

Simon Willison

Opera Web Standards Curriculum

Opera Web Standards Curriculum. Opera commissioned an impressive sequence of articles from a bunch of very talented people to help address the monstrous learning curve for modern client-side development.

July 08, 2008 02:22 PM

JP Stacey

Trailing commas and unfeasibly high line numbers

In Javascript, trailing commas are to be considered harmful. Strictly speaking, they’re not allowed by the syntax, but this wouldn’t be such a problem were it not for the fact that some browsers (including Firefox) will quietly ignore them, pretending briefly that Javascript’s syntax is Pythonic or, um, Rubric. The safest route to take is to avoid trailing commas wherever possible. I’ve mentioned the general problem before, and the hard core among you would probably go so far as to change their formatting to highlight trailing commas.

But what makes it all far, far worse is that IE6 and IE7 can sometimes produce error messages which, as is usual for those browsers, contain no useful information whatsoever.

Here’s a recent example:

Line: 64432871

Char: 2

Error: Expected identifier, string or number

See that impossibly high line number? That’s a result of the parser being struck soundly on the head by the syntax error, and consequently dribbling its way past the end of the Javascript file. IE’s incomprehensible “English” error message conspires with circumstance to make the whole report of no help at all.

Something like JSLint, on the other hand, is of tremendous help, and we’ll be running as much Javascript through it as possible in future. JSLint is just as unforgiving, but as it’s reliably unforgiving and incredibly communicative with it then that’s a bonus rather than a detriment.

It’s sometimes a little too strict to be useful, complaining about implicit global variables (even when that variable is called window). My suggestion is to ignore those reports, though; but watch those commas!

by jps at July 08, 2008 01:04 PM

Simon Willison

Apple just gave out my Apple ID password because someone asked

Apple just gave out my Apple ID password because someone asked. “am forget my password of mac,did you give me password on new email marko.[redacted] @yahoo.com”. Classy.

July 08, 2008 10:10 AM

A quote from Matt Cutts

Question: how do you upgrade servers when you need to pass new information between them? It’s a fool’s game to try to upgrade both servers at the same time. So you need a communication protocol that is not only backward compatible (a new server can speak the old protocol) but also forward compatible (an old server can speak the new protocol). Protocol Buffers provide that because new additions to the protocol can be ignored by the old server.

- Matt Cutts

July 08, 2008 09:11 AM

Protocol Buffers: Google's Data Interchange Format

Protocol Buffers: Google’s Data Interchange Format. Open sourced today. Highly efficient binary protocol for storing and transmitting structured data between C++, Java and Python. Uses a .proto file describing the data structure which is compiled to classes in those languages for serializing and deserializing. 3-10 times smaller and 20-100 times faster than XML.

July 08, 2008 08:20 AM

July 07, 2008

Graham Lee

CocoaHeads Swindon

Just got back from the first meeting of Swindon CocoaHeads, featuring a bunch of people who live nowhere near Swindon, some good beer and the occasional discussion of Cocoa. Special mention to Scott who came all the way from sunny Warsaw to be with us!



Tonight's event was an informal, "what do we want from Swindon CocoaHeads?" event, but it looks like being successful enough that we'll be doing it again. The format will be a presentation or directed conversation, followed by general chit-chat about all things Cocoa. In fact, um, I may have volunteered to give the first presentation at the next meeting. The subject is: well, that would be telling, wouldn't it… ;-). You'll have to find out by coming along to the Glue Pot in Swindon at 8pm on Monday, August 4th. Look out for further announcements and a mailing list over at Scotty's place in the forthcoming month!

by leeg (noreply@blogger.com) at July 07, 2008 11:47 PM

Simon Willison

Growing the ORG community

Growing the ORG community. The Open Rights Group want a fiver a month from 750 new people to support their excellent work fighting for digital civil liberties in the UK political arena. Going by their past performance this is a very worthwhile investment.

July 07, 2008 06:42 PM

A quote from Me, on Twitter

It looks like the first ever Django conference will take place in early September in the San Francisco bay area.

- Me, on Twitter

July 07, 2008 05:14 PM

Django Unit Tests and Transactions

Django Unit Tests and Transactions. If you’re using a transactional database engine (MySQL with InnoDB, Postgres or SQLite) you can speed things up by running each of your unit tests inside a transaction and rolling back in tearDown().

July 07, 2008 02:14 PM

OpenTech 2008 "Impossibox" presentation

OpenTech 2008 “Impossibox” presentation. One of my favourite Open Tech sessions—Tom Loosemore describes the “Impossibox”, a cloud of PVRs collaborating to transcode and share “all decent UK TV for a year” via BitTorrent.

July 07, 2008 02:11 PM