Australian Lefty on Politics, Governance, Science and Info Management

The major inhibitor of open government and commercial efficiencies?

Posted by Dave Bath on 2008-12-07

When there is a trivial technical enhancement to a ubiquitous type of utility that could dramatically improve efficiency and governance, both of societies and within organizations (even commercial ones), and even make life easier for individuals, why doesn’t it happen?

Consider email messages, and the huge proportion of the information base of an organization contained in email rather than normal reports, etc.

Now consider how difficult it is to find information within a mailstore by detailed topics: not just the subject lines, but proper metadata that is required under recordkeeping acts and regulations in web pages of agencies.  Consider the costs of this, especially in relation to FOI requests.

What is stopping this level of metadata being included in email?  Almost nothing but a few hours by coders of user presentation layers of mail clients.

The silly thing is, all the components are in place, and mail clients from the Elder Days had the capabilities in user interfaces.

Here’s the technical overview, hopefully understandable to non-geeks:

  • Email messages have headers.  Some of these you all know about (Cc, Subject, Reply-To), others are necessary but you don’t normally see (like the audit trail of how a message got to you, maintained by each hop from sender to receiver), and others are extensions (and begin with "X-"). See IETF RFC 822 if you are a geek.
  • The Australian-government mandated AGLS and AGIFT standards defines audience, standard keywords for function in government thesauri, publisher, jurisdiction, visibility (e.g. publically-visible, commercial-in-confidence, cabinet, yada yada yada).
  • It is trivial to define an extended header that drills down into AGLS/AGIFT keywords, by a convention that anything after a header with a name starting with X-AGLS- is analogous to the corresponding AGLS/AGIFT element
  • It is trivial to define a convention that the information after the X-AGLS- label has a value similar to that which would be contained in the equivalent META item of a web page
  • It is trivial for a client presentation layer mail utility to have a button or menu item that prompts the user for an extended header and its value.

Here is a trivial example (taken from the AGLS v2 manual) : If a government document is aimed at children, it is supposed to have something like the following embedded inside it, so on a web page it should have the following:

<META NAME="AGLS.Audience" SCHEME="agls-audience" CONTENT="Children">

A corresponding email header element could be:

X-AGLS-Audience: SCHEME="agls-audience" CONTENT="Children"

Note that the AGLS/AGIFT rules allow specification in encodings that a document discusses contract management of water supply within a given geographical region.  Just imagine how cheap this would make fulfillment of an FOI request!

At the very least, every public servant should want the ability to have an extended header that marks an email as social (like one saying "We are having our Xmas drinkies at…." or "Footy tipping results are…") so they don’t need much archiving, while managers can make searches for real work (especially with FOI requests) much more efficient.

Exactly the same considerations apply to a company that want to be able to find all emails (and only those emails) relevant to a given purpose, so they can get the information needed to make an effective decision.  Ditto for the discovery costs for legal actions.

So, given the huge payoffs and trivial implementation costs, when will the Department of Finance and Deregulation (which includes the Australian Government Information Management Office – the Oz agency for which I have most respect) lean on both open- and closed- source email client providers (whether standalone like Thunderbird and Outlook, or hosted like gmail) to allow addition of arbitrary headers when creating an email message)?  The same goes for other Oz and international governments, as well as commercial entities.

Or don’t these organizations really want information discoverable, but remain obscurable, even though there are major efficiency gains on offer?  If a government is serious about open and efficient government, and wants to be able to offer cheap and reliable FOI capabilities, there is no excuse for not going down the path of using X-headers in email.

Though an anti-Microsoft bigot, if MS-Outlook provided such a capability when constructing messages, I’d recommend we all swapped to Windows and Outlook until other clients provided the same capability.  Now that’s saying something!

A good example of hiding information with shoddy metadata is a transcript from the Victorian Government "Inquiry" into Melbourne’s Future Water Supply:

  • Title: Microsoft Word – Hansard 08092008.doc (exactly the same as other transcripts from the same day
  • Author:sdinsber
  • Subject: (blank)
  • Comments: (blank)

2 Responses to “The major inhibitor of open government and commercial efficiencies?”

  1. […] anything?  EMPTY EMPTY EMPTY!  See my most recent whining post about this: "The major inhibitor of open government and commercial efficiencies?" (2008-12-07).  Posted in Australia, Civil rights, Economics and Business, […]

  2. […] "The major inhibitor of open government and commercial efficiencies" discusses how poor management of email metadata (lack of use of "X-Headers" in email, mainly because most email clients provide less functionality than old ones) screws everybody over except the inefficient and the corrupt: and perhaps the Gov2.0 Taskforce would consider funding development X-Header-savvy plugins (that reference Dublin Core, AGLS and government thesauri) for common email clients (including Thunderbird and Firefox) given that the taskforce has that power (but I don’t know how full the chequebook is!)  […]

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

%d bloggers like this: