Pray tell, oh oracle - how much did thine internet search cost?

Under attack from new AI rivals, projects such as Google's SGE aims to streamline information retrieval. What might this mean for publishers, asks Rob.

by Rob Corbidge

Published: 16:38, 23 November 2023

Rob Corbidge is Head of Content Intelligence at Glide Publishing Platform, applying the latest knowledge about advances and ideas in the publishing industry to our own product and helping clients get the most from their content.

What does an internet search cost? And how far into the ether do we follow the thread of the "cost of a search...". Well, luckily there is a term for that: Delphic Cost, which attempts to represent the entire cost of the information search process online. 

It is summarised in the 2023 paper which coined the term, as "access costs (for a suitable device and internet bandwidth), cognitive costs (to formulate and reformulate the query, parse the search result page, choose relevant results,etc), interactivity costs (to type, scroll, view, click, listen, and so on), and obviously for all of these activities plus waiting for results and processing them, time costs to task completion."

Notably, the research paper in question came out of Google Research, proving once again that while the company's commercial activities show little love for the content people who provide its search results, such an expansive and well-funded business produces some interesting thinking which is useful for everyone.

Why are we interested in this Delphic Cost? 

Put simply, it is the drive to reduce this cost yet further which lies behind current efforts with AI-enhanced search, such as Google's SGE. For those interested, we'd really recommend going through this presentation by Google's Mark Najork on the historical development of information retrieval, and what it is that technology such as SGE is trying to improve upon.

At the very least, such understanding provides protection to publishers from the SEO confusion which inevitably rides alongside any significant change to search. By search we mean Google, and so it is important to understand what will change if or when SGE becomes widely adopted. 

Pythia, the high priestess of the Temple of Apollo at Delphic, the Oracle of antiquity herself, would answer any question, but supplicants needed to ask the right question. Anyone who has read Google's own documentation may appreciate the irony in such a term being coined from their own laboratories.

I've actually worked as a human search tool. Before the world of news journalism sucked me into its unforgiving but oddly enjoyable orbit, I worked in a couple of large city centre bookshops, when such shops were not so unusual, and thus I found myself in a world of fuzzy queries as a capitalist librarian, so to speak.

Your professional worth in a bookshop is largely predicated on your ability to send customers who come in with a question away with a book, or at least an order for one. This process can start with "The American woman who died last month, she wrote about dogs, do you have her book?" to "I need the main text book for my psychology course, but I don't have the title".

Such queries are resolved by a) keeping up to date on author news, and b) asking which college they go to and knowing what the text book they use. Don't forget, this is before the internet had worked its way down to the sales floor, or anywhere else public really. Such knowledge is lightly worn if you love books and even love the ones you don't like.

However, the human search tool also has its limits. Twenty minutes spent trying to work out "it's got a blue cover and it's about vampires" is a Delphic Cost indeed. There were many variations upon such vagary, and much satisfaction in resolving at least a portion of them.

Is it to be expected that this next generation of search or the next, could provide direct answers to such direct yet tricky questions? I do believe it is, such is the sophistication of systems such as SGE and the multiple reference points they use to retrieve answers.

Interestingly, Najork makes a point in his presentation that is more or less the sum of all search fears for publishers (well, aside from yet another traffic-skewering algorithm change): "Direct answers will affect how users interact with search engines themselves. Users are less likely to leave the search engine while on their information-seeking journey; on the other hand, journeys may go father and deeper due to the lowered Delphic Cost."

In this content, "not leaving" means not going to our websites and reading the things we have written. However, an optimistic parallel view is that if this search experience is both satisfactory and rapid, then publishers could benefit as dwell time gets an increase from those truly interested in a subject.