Web Directions Year round learning for product, design and engineering professionals

Back in early September 2022, so over three years ago now and weeks before ChatGPT came out, I wrote this article. For some reason, I never published it. I’m not entirely sure why. Perhaps a version of it ended up in my newsletter. I don’t know. But in it, I said to people, playing with the GPT playground, which was a precursor to ChatGPT. And I also observed right at the end

If I were advising a younger person than myself where to focus my attention, I’d certainly pay close attention to what is happening here.

Well, turns out I did follow my own advice, and a lot of the area, a lot of what I focused on over the last three years or so has been AI-related. Our conferences have increasingly focused on this, sometimes as AI-focused conferences, sometimes with AI-related content for our existing conferences. There’s a whole water that’s coming up in our next three conferences. In fact, I’d love you to check them out.

But I thought despite it having aged in some ways, there’s a lot of value in this being out there. So here you go. Take a look at what I was thinking about in terms of AI, as the world was about as before we got ChatGPT and the modern explosion of AI and machine learning.

Since its very early days, Artificial Intelligence researchers and pioneers have expressed optimism that within some relatively short time frame machines will match, if not surpass, human intelligence and capabilities.

The idea (and project of) Artificial Intelligence as a simulacrum of human intelligence is very long standing–exemplified in the note of the Turning test–devised by Alan Turning (a personal idol of mine of whom I’ve written before), and originally named ‘the imitation game’ (also the name of a not so good movie about Turning’s wartime efforts in cryptanalysis–if you’re interested in Turing’s life, the play and hard to find TV adaptation ‘Breaking the Code‘ is far superior).

Turing’s idea was “If a machine could carry on a conversation (over a teleprinter) that was indistinguishable from a conversation with a human being, then it was reasonable to say that the machine was ‘thinking.”

So when most of us hear the phrase ‘artificial intelligence’ it’s likely we think of something like us–intelligent and sentient. Artificial General Intelligence.

And even though the dreams of a machine that thinks, and communicates like us started to flounder in the 1970s, this idea persists.

Learning not Intelligence

However, researchers in the early part of this century started to take a different tack. Rather than trying to devise a set of rules that machines could apply even in the absence of very much data (after all it was the prevailing view that humans learn language from a tiny number of imperfect examples and so underlying this capability must be a set of rules), researchers started to take statistical approaches to learning from increasingly number of large data sets-for example large sets of translations between languages.

In popular imagination the idea of sentient, conscious machines continues, with films like Her or recent claims by a Google software engineer that a chatbot had become sentient (they were subsequently fired), in research (and application), however, machine learning has focussed on much narrower subsets and domains. Indeed even the term ‘learning’ being applied to these techniques is not something all researchers are comfortable with.

If you’re a developer, Github’s co-pilot, trained (not uncontroversially) on public, open source licensed code bases on Github (it’s far from clear that all these licenses do permit the use in every instance that co-pilot makes of this code) will be well known. Co-pilot is developed using GPT-3, from OpenAI.

In essence, GPT-3, trained on staggering amounts (45Terabytes) of text based data (including wikipedia, internet based corpuses of books, large numbers of websites) statistically predicts, given a sequence of words, what might come next. It turns out this is very good for a large number of text based machine operations, including translation, speech recognition, classification, summarisation and more.

In a similar vein are text-to image-generators, like DALL•E (again from OpenAI, and again using GPT-3 again not uncontroversial in terms of its use of copyright material), and the just released Stable Diffusion from Stability.ai. As Simon Willison writes “Stable Diffusion is a really big deal“.

I have been paying attention to what has been happening in AI/ML for some time. It’s in fact one of the key reasons I studied computer science way back in the 1980s (that’s a story for another day). Much more recently, we hosted two conferences focussed on applications of AI in 2017 and 2018-as always perhaps a little earlier than the mainstream was ready for, back too when AI driven chatbots and conversational interfaces seemed to be perhaps the next step in the evolution on UIs.

I think it’s a genuinely interest and valuable area of focus, more than ever.

There are certainly ethical (and deeper societal) challenges to consider, something I’d like to return to in future articles, and there are also opportunities to explore.

I have started thinking of these sorts of AI as being like power tools for the mind. In a similar way that machinery made humans far more productive at processes we had been carrying out for centuries if not millennia (weaving, threshing, harvesting), I can see tools like GPT-3 leading to the applications that currently, or until recently, required humans (think of speech recognition, or translation) and making these far more affordable, enabling use cases that previously would have not been economically feasible.

In our own efforts with captioning conference presentations, we used to pay many thousands of dollars a year for human translations that might take hours or days to be completed, and which still (given the often technical and niche nature of presentations at our conferences) required considerable expert editing.

In the last couple of years we’ve moved to machine learning based captioning, at a tiny fraction of the cost, where transcripts are ready within at most minutes, and which have a far greater and (to my mind at least) an ever increasing accuracy. I’d say ML based transcription now significantly exceeds the accuracy of human based transcription from only a couple of years ago.

I’ve been continuing to explore applications of GPT-3 (such as summarisation, keyword extraction and more) to add more value to our archive of conference presentations at Conffab. You can explore some applications yourself at the OpenAI GPT-3 Playground.

ML not AR/VR or Blockchain?

While so much attention and hype surrounds AR and VR, not to mention blockchain and “Web3” “innovations”, my instinct is that perhaps when we look back to the early to mid 2020s in a decade or 3, applications of Machine Learning might be where we see the greatest transformation.

If I were advising a younger person than myself where to focus my attention, I’d certainly pay close attention to what is happening here.