Can Algorithms Break the Law?

Here’s a breakdown of how 2 of the most prominent search algorithms on the web appear to be presenting information that it would be illegal for you or I to communicate.

  1. The first is the Twitter search algorithm, tweaked recently in a way which (inadvertently) increases the likelihood of people finding out ‘illegal’ information.
  2. The second is a Google algorithm, and 2 user interface changes which (again inadvertently) very much increase the chance of ‘illegal’ information being communicated.

4″ />

The Background – a Very Sad Story

In the entire history of the British legal system, there are apparently only four prisoners who have been given new identities. One of them was a boy (now a man) who used to be called Jon Venables. When he was a child, Jon Venables & another boy (Robert Thompson) killed a very, very young boy (James Bulger). If you live in the UK, you will almost certainly know this.

Because of their crime, Jon Venables & Robert Thompson are very likely to be in extreme danger if the general public can identify them. Therefore, they were given new identities, and those identities were protected.

Since Jon Venables was given a new identity, nobody is allowed to know his new name: It is illegal to publish anything claiming to identify him, or even to ‘purport to identify’ anyone as him whether it is him or not.


Photos of Jon Venables were apparently published on Twitter recently. This hit the front pages of most newspapers in the UK (including, above, The Guardian).

The Twitter Search Algorithm Change

Until recently, Twitter’s search algorithm only returned very recent tweets. A few weeks ago, that was altered to also include much, much older tweets. That coincided with the 20 year anniversary of Jon Venables’ crime. As a result, lots of very old tweets and photos claiming to ‘out’ Jon Venables identity suddenly became visible far more easily, at a time when many were searching for his name. Many of these photos were retweeted; some new ones were posted, some were taken from Twitter and posted on other websites, and many users posted names that they claimed were Jon Venables’ new identity.

That hit the front page of most UK newspapers (though none made the link with Twitter’s algorithm update), and – of course – an investigation was carried out.

Twitter Search Last Week: Jon Venables

Here’s what happened if you searched Twitter for ‘Jon Venables’ a week ago:


You can see there, Twitter has pulled a ‘Top news’ box covering the investigation. It has automatically pulled a ‘Top’ tweet that was Retweeted 49 times, claiming to out Jon Venables’ new identity & urging readers to spread it.

There are 2 big red blocks there too where I’ve obscured 2 important items:

Item 1: Top Photos

On the left (item 1) I’ve obscured 2 photos showing an adult man, roughly the age Jon Venables would be now. Those were posted by users, but the algorithm picked them out as ‘top photos’.

Item 2: A man’s name

Across the top (item 2) I’ve obscured a man’s name, flagged as being a ‘Related search’ for ‘Jon Venables’. Again, as with the photos, this was posted by Twitter users, but Twitter’s algorithm picked it out & highlighted it as being a related search.

Clean Up?

Since this all came out, the UK Attorney General has announced that there will be ‘contempt’ proceedings launched against people who posted images ‘purporting’ to be Jon Venables.

Twitter has obviously made a substantial effort to clean up the photos too.

Twitter Search Now: Jon Venables

Today, if you search for ‘Jon Venables’ on Twitter, the photos on the left (which previously showed an adult man) have gone. BUT, the ‘Related’ search term, showing a man’s name, is still in place:


If you click through on the man’s name in the obscured ‘Related Search’ up there, it leads you through to this:


Twitter Summary

In the first instance, Twitter’s algorithm automatically highlighted photos & details of an adult man whenever users searched for ‘Jon Venables’. Twitter users themselves posted the content, but the algorithm crowdsourced from that to highlight particular photos, and a particular name.

Twitter have obviously gone to some lengths to clean things up here. BUT, on searching for ‘jon venables’, the algorithm still leaves a fairly prominent trail toward a man’s name, and toward tweets linking to photos of an adult man.

And Google?

The Google story is much, much shorter. It involves 2 relatively recent additions to Google’s user interface:

  1. The ‘knowledge graph box’ – an area on the right of search results that is intended to reveal ‘Facts’ related to searches.
  2. Google’s updated Image Search results, which used to show images in the context of a web page, but now simply show the image on a black background on Google itself.

Here’s what happens when you search for ‘Jon Venables’ on Google today:


I’ve obscured an area on the right there (within the ‘knowledge graph’ box), where there are 2 photos (within a single image) of an adult man.

And when you click on that obscured area, it leads us to this quite scary screen:


A few weeks ago, clicking the image from search results would have taken you to the website in question (faded in the background), showing clearly that it was a website other than Google publishing it. Today, doing so keeps you on Google’s own property (note the ‘’ URL). It simply shows a black background, 2 photos of a man, and the name ‘Jon Venables’, the old name of a person it is illegal to identify.

4 Replies to “Can Algorithms Break the Law?”

  1. Feedback and info… ok.

    I take an amateur interest in cyberlaw; I did a bit of research on the Ched Evans rape case tweets, and this is another case that interests me.

    The big question, the elephant in the room no-one is talking about… what is the *point* of the UK AG ‘taking action’? What does it achieve to bring contempt proceedings against people in the UK, or order UK-based websites to take down photos, when *everyone else on the internet* – the other 98% of the human population – can tweet and post whatever they damn please, because they’re *not bound by a UK court injunction*!?

    Facebook and Twitter may or may not be cooperative; they’re obviously not UK operations so they don’t have to. But there are a lot of other websites that won’t do a damn thing. Think 4chan would heed a British injunction, or even a polite request to play nice? The genie is well and truly out of the bottle and isn’t going back in again.

    I think we have to start calling the AG Dominic ‘Canute’ Grieve!

  2. On another aspect… the AG’s office, when this story broke, issued a statement saying there was a ‘worldwide injunction’ against publication. Virtually everyone seems to be repeating this unthinkingly and uncritically, without stopping to realise that the very concept is a load of tosh.

    I’m astonished the AG would make such a facile error. UK courts only have jurisdiction within the UK; how do you think the injunction would play out in America, for instance, when it comes up against the First Amendment? In the USA it would laughed out of court as an unconstitutional prior restraint on free speech.

    An American writing in America is bound by American law. They can’t be prosecuted for writing about something that’s prevented by an injunction in the UK, contrary to UK law, any more than they can be prosecuted for posting naked photos, contrary to Saudi law!

    The injunction in this case is a rare ‘contra mundum’ injunction. This literally means ‘against the world’ – and the AG’s press office appears to have taken that literally. Big error! What it actually means, of course, is ‘against everyone within the UK’ – to distinguish it from a common or garden injunction against specific named parties.

    This lawyer explains more fully:

    The Guardian touched on the ‘overseas media’ angle long ago:

  3. Wow,

    Ive taken a interest in the case and WOW has anyone noticed that 2 weeks ago there were a 1000 images of J** V******** on google images.

    Now there is 2

    The UK govmnt have managed to clean the internet

    I bet my computer is being monitored.

    Two weeks ago I found the police interview of those two evils people. Now it has disappeared with a very random message.

    I found hundreds of piccies of one of the called criminals but now they have been cleaned up

    What are our taxes being used for. No doubt this message will be deleted by the authorities.

    But wow f**ing wow how do they manage to remove almost all traces of his image ?

Leave a Reply

Your email address will not be published. Required fields are marked *