Watermarking AI: analysing drops in an ocean of plagiarism

Attempts to enforce the provenance and use of AI-generated content may inadvertently entrench the most successful of those who pinched it in the first place.

by Rob Corbidge

Published:

In the course of writing a eulogy for a dearly loved and recently departed relative last week, I had cause to marvel at the sheer complexity of the human writing process. The speech in memory of their life was well received by all those who heard it, and indeed was praised by many, it making them by turns happy, sad, and reflective, which is the purpose of a eulogy as far as I'm concerned.

Yet as I told my partner that I, as a writer of some modest skill was pleased with the result of my efforts, and she said "I'm glad you're happy with what you've written", I immediately responded "Well, truth is I'm never really happy with anything I've written, and if I'd written it today it would be almost entirely different."

It's true. Even if the aim of the writing - evoking the memory of the departed as warmly and effectively as possible - was an irreducible purpose, the characterisations used, the words, the structure, could all have been different. The human brain still yet has a few tricks left up its cerebral sleeve in its almost endless capacity for creation.

Which leads us today on to the fast-warming topic of content generated by AI, how to identify it, and moves to make it easier to do. And why it may encourage the very things we are trying to avoid.

Spy vs sp-ai

Anthropic announced this week that it would be complying with new EU legislation on transparency of AI-generated content, stating that from August 2 when the act came into force "Claude models launched in the EU will support machine-readable marking at launch. Generated text will carry embedded watermarks, and generated files will include digitally signed provenance metadata where supported." This means Claude-generated files in formats such as jpeg or svg will be signed with provenance metadata conforming to the open standard established by the Coalition for Content Provenance and Authenticity (C2PA).

I feel obliged here to speak for many publishers by pointing out you can't hide the origin of stolen goods just by sticking a new label on them, and until content stolen for training purposes is traceable and reconciled with its owners, anything post the point of looting feels tainted. In short, we'd like to watermark their watermark.

As you would expect, Anthropic are being rather secretive about how exactly their watermarking system works. There are fairly reliable ways of adding a traceable element to an LLM-created text, a good explainer of one such is here for those interested, but essentially the choice and placement of words can act as an identifier. But as The Register have noted,  "Anthropic's insistence that its text marking scheme 'doesn't change the meaning' of Claude's output" would suggest they are using a different method. Time will tell.

Whatever method they are using, they make no claim as it being foolproof. It is without irony, a concept that simply suffocates in the world of Big AI, that Anthropic say that "Detecting a Claude mark tells you that the content may have been processed by Claude. It does not, on its own, confirm the full provenance of the content." The full provenance of the content surely being something they don't wish us to know in their transformative world.

There are other caveats against traceability too, such as if "the text has been heavily edited, paraphrased, translated, or mixed into other writing", nevertheless for the vast majority of LLM-created text - and research by Google using the known watermarking method mentioned above has backed this up - it will remain possible to trace it, even if there is a scientific paper out there actually called Watermarks in the Sand: Impossibility of Strong Watermarking for Generative Models.

Key access chain

As it stands, only Anthropic currently possess the keys to unlock its own watermark. It's understood that Google gives the US government the ability to trace its watermarks in AI-generated content, so it's to be seen if Anthropic follow suit. All it has said so far is that it is "working to enable users and other third parties to detect Claude’s embedded watermarks and provenance metadata," with technical documentation "forthcoming".

One final thought is around the effect of legislation - particularly reactive legislation - and that in the haste to get something on the statute books, the second and third order consequences of doing so are not properly considered. While Anthropic's watermarking will almost certainly satisfy European regulators, it has also created a bar to market for smaller rivals who must add such a requirement to their development costs, as well as potentially making open source LLMs far more attractive to those with no interest in traceability.

This then holds the prospect of the market leaders - all American - having their dominant position reinforced due to superior abilities to comply with regulations, and also see the bad actors who don't wish to leave a trail head off to use other systems. 

Now you can say what you want about my writing, and about the multitude of ways I could have written the same thing, but it's supremely likely that the work could be identified as mine. 

Humans... we watermark ourselves just by being us.