I’m starting to think AI chatbots are like buses. You wait years for one and then three come along at once. In the past couple of weeks we’ve seen the launch of Bing Chat, the release of multimodal GPT-4 (younger and more talented sibling of GPT-3) and just last week, the somewhat hasty roll out of Google’s Bard. There’s a lot going on and chances are, by the time this article is published, there may even be a fourth on the horizon if Meta gets a wriggle on.
Before we jump in, here’s a quick recap on the big three:
What is GPT-4?
A large multimodal model created by OpenAI and the fourth in its GPT series. Currently available via a paid subscription to ChatGPT Plus. Will be able to read and analyse images as well as text.
What is Bing Chat?
Powered by GPT, the new Bing brings together data from the GPT-4 model and Bing to generate answers quickly that are based on real-time information.
What is Google Bard?
A large language model chatbot developed by Google AI. It is powered by Google’s own LaMDA, a language model that can generate text, translate languages and answer questions with real-time information.
Bear in mind that these are not the finished products and all three are at various points in their development journey. Discovering and comparing capabilities is a work in progress operation where the goalposts keep moving (such as the recent introduction of GPT’s new third party plugins), but having dipped a tentative toe in all three, here are some initial findings.
Comparisons of the Big Three AI tools
Following a week of experimenting with the three language models, I've begun to form an initial understanding of each one's strengths and weaknesses. None are flawless at this stage and Google, Bing and OpenAI all acknowledge the bots' propensity to "hallucinate" (i.e., provide incorrect or nonsensical responses), which reveals their current limitations in terms of reliability and authoritativeness.
“Bard is experimental, and some of the responses may be inaccurate, so double-check information in Bard’s responses.”
So the advice from Google is…to google Bard’s responses. Hmmm.
Taking this caveat into consideration, I wanted to explore three specific areas: accuracy, citations and recommendations. The latter being of particular interest due to the potential impact on businesses and their content marketing strategies.
Accuracy
All three language models have dropped some pretty big clangers along the way, with Bard’s promo video humdinger of a faux pas resulting in $100 billion being wiped from Google’s parent company’s market value. Now that the dust has settled a little, I was keen to get an idea of how accurate and united the responses were across the three platforms.
I posed the following question:
How many miles is it from Christchurch Dorset to Christchurch NZ?
Not a black and white query by any means, but one where I expected a similar ball park distance across the board.
Bing and GPT provided the following answers respectively:
Both reasonable responses, with GPT providing the distance in kilometres as well as miles, plus additional elaboration.
And here was Bard’s response:
Oh dear.
Things didn’t improve with the follow up response either:
As an aside, it's important to note that GPT-4 would be unable to accurately answer a more time-sensitive question, such as "how many goals were scored in the 2022 World Cup?" due to its training data only recognising events up to to September 2021. However, the recent option of installing supporting plugins will help to counter such limitations and make it a much more powerful option.
Citations
For this exercise, I used the query ‘What year was the digital marketing agency Adido founded?’
Bing gave a correct answer, using the official Adido website as its cited source, thus demonstrating to the user that the answer can be considered trustworthy and accurate:
GPT-4, like its predecessor GPT-3, does not provide citations, stating that it is “not designed to provide direct citations like an academic paper or a research article.” Instead, their information is scraped together from a large dataset of text from various sources, such as books, articles and websites, which all sounds disconcertingly vague in my opinion. So while the answer given was correct, there is no way of ascertaining what the source for this response is:
Bard provided the following answers:
Not only was the first answer incorrect, no citation was provided, even when prompted.
So why the reluctance to offer citations? Here’s what Bard had to say:
“Bard, like some other standalone LLM experiences, is intended to generate original content and not replicate existing content at length. We’ve designed our systems to limit the chances of this occurring, and we will continue to improve how these systems function. If Bard does directly quote at length from a webpage, it cites that page.”
What Bard does offer is a ‘Google it’ button, so users can search for validation themselves, reiterating that the model should be used as a ‘complementary experience to Google Search’ as opposed to two mutually exclusive entities. This suggests that Google wants to keep users engaged with traditional search methods (potentially to preserve precious ad revenue), however their reluctance to provide citations seems a curious stance to take, and has subsequently raised a few eyebrows among some well-known SEOs.
Recommendations
The way people search for products or services online may be changing. Instead of searching for specific products or services, users may simply ask a language model for recommendations.
The implications of this could be huge for e-commerce businesses, so I was keen to understand how the language models collated their recommendations, i.e. establish the data sources that their respective algorithms use. Does Bard, for example, link up directly with the local pack when asked ‘Can you recommend a chinese restaurant in Bournemouth?’.
The answer is currently no (as demonstrated below). Only one of the local pack features in Bard’s recommendations:
So just where do Bard and co get their data from in order to make recommendations? As a final experiment, I posed the following question:
‘What are the best UK ski tour operators?’
All three offered a similar selection of tour operators, with Bard and GPT providing accompanying info for each one, whereas Bing preferred to keep it short and sweet:
Although Bing can supply citations, the basis for recommendations and the data gathering methods employed by the other two remain unclear. To obtain further insight, I prompted the bots to elucidate on this matter with their respective responses highlighted in the table below:
The sources above indicate that AI models derive their recommendations from similar criteria as search engines, such as E-E-A-T (Experience, Expertise, Authoritativeness and Trustworthiness). This implies that SEO (or perhaps we should start referring to it as AIO??) remains equally vital for AI recommendations as it does for search engine rankings.
So how can websites harness AI potential?
If the search landscape is changing, then it’s imperative businesses ensure their websites are optimised to maximise their potential for appearing in AI recommendations. Language models can only present information from the data that they find, therefore by implementing SEO best practices and focusing on brand development today, businesses will likely reap significant benefits in the future.
Here are some ways you can get your website AI-ready:
Increase online visibility
Maintain a strong online presence through an informative and well-designed website, active social media profiles and participation in industry forums and blogs. This will help ensure that the AI language model encounters your business more frequently during its training.
Encourage customer reviews
Positive customer reviews and testimonials can significantly impact product recommendations. Encourage satisfied customers to leave reviews on popular review platforms, including your website and social media channels. This will make it more likely for your products to appear in the AI's recommendations.
Publish high-quality content
Create informative, engaging and well-structured content related to your products and industry. This includes blog posts, articles and press releases that the AI language model can learn from. Ensure that your content is optimised for relevant keywords and phrases to improve visibility.
Develop partnerships and collaborations
Collaborate with influencers, bloggers and other businesses in your industry to increase your brand's reach and recognition. These partnerships can result in mentions and backlinks, which will raise the likelihood of your products being recognised and recommended by the AI model.
Feature FAQs and Testimonials
AI search users seek in-depth information and real-life experiences. FAQs and testimonials can deliver this information in a manner that AI search algorithms easily understand. Utilise FAQs to address common queries about your offerings, and testimonials to highlight positive customer experiences.
Conclusion
AI innovation is moving at such a rapid pace I’m in no doubt all three models will have evolved significantly by the time this article is barely a week old. Based on my findings to date, however, Bard appears to be lagging some way behind Bing and GPT-4 in terms of accuracy. Whether this is due to the haste in which Bard was rolled out, or perhaps Google's apprehension about AI's influence on conventional search methods, is food for thought.
One thing is certain though. The advent of AI language model recommendations heralds a potential shift in the way people will start searching for products and services. While I think traditional search will remain top dog for a while yet, businesses should ignore the rise of AI search and AI generated content at their peril. Tapping into its potential early on will not only serve to enhance the user experience but also help brands maintain a competitive edge in an ever-evolving digital landscape.
For the short term though, the Big Three are still trying to navigate that landscape without veering off course in the haze of a hallucinatory stupor…and where would that leave them? In Christchurch, New Zealand perhaps….just 9 miles east of Poole and 20 miles west of Southampton.
