Before it was launched, Google promised that its Gemini AI model would outperform OpenAI’s GPT-4. Gemini became publicly available in December 2023 but is it just another alternative AI chatbot that fails to match ChatGPT’s supremacy? I compared both models to see who the winner would be, the veteran AI model or the relative newcomer on the block.
What Is Google’s Gemini AI Model?
Gemini is Google’s most capable generative AI model, able to understand and operate across different data formats, including text, audio, image, and video. It is a unified and flexible AI model leveraging capabilities from different AI technologies. It is currently available in four variants:
- Gemini Ultra: The largest and most capable variant designed to handle highly complex tasks. Gemini Ultra was highly lauded by Google before launch and the model was accessible for a short while before it was silently withdrawn.
- Gemini Pro: The best model for scaling and delivering high performance across a wide range of tasks, but less capable than Ultra. It offers a one million context window and is available to users with a Gemini Advanced plan, although there is a way to use the model for free.
- Gemini Nano: The most efficient model designed for on-device task deployment. For example, developers can use Gemini Nano to build mobile apps or integrated systems, bringing powerful AI into the mobile space. It is slated to launch on Pixel and other Android devices in late 2024.
- Gemini Flash: The fastest and most cost-efficient multimodal model with great performance for high-frequency tasks. It is integrated into several Google applications, including Sheets, Gmail, Docs, and Drive. It is also available as a replacement for Google Assistant on Android.
How to Use Google Gemini AI
The Gemini AI chatbot uses the Flash model, by default. Users with the Gemini Advanced plan have access to the more powerful Pro model instead. You can access the Gemini chatbot by visiting gemini.google.com and signing in with your Google account.
Android users can download the Gemini mobile app, which doubles as a mobile assistant. On iOS, it is available in the Google app as a separate tab.
How Gemini 1.5 Flash Compares to GPT-4o
As it stands, ChatGPT is the gold standard that any other model is held up against. Using the default versions available in both chatbots, I compared Gemini’s ability in math, creative writing, code generation, and image creation to ChatGPT’s.
Want to know how Gemini Advanced compares to ChatGPT Plus? We’ve compared the paid versions, and there is a clear winner.
1. Solving Math
Starting with the easiest math question I could think of, I asked both chatbots a simple question:
Gemini and ChatGPT both got the correct answer (-1) on the first try.
Neither chatbot had trouble solving any of the other easy math problems I threw at it. So, I decided to take things up a notch with a more complex math problem from the Prompt Engineering Guide.
Suppose g(x) = f^(-1)(x), g(0) = 5, g(4) = 7, g(3) =2, g(7) = 9, g(9) = 6. What is f(f(f(6)))?
Both chatbots were able to parse the problem, even though it was in plain text. However, only one chatbot was able to give me the correct answer.
Only ChatGPT could derive the result of the composite function: 4. Gemini got lost pretty early in the process.
2. Creative Writing
Next, I compared both chatbots’ abilities to create new works of writing. Writing is the most common use for generative AI chatbots, and it stands to reason that they should be excellent at it. I asked them to create a poem this time.
Write a poem about Tesla (the electric vehicle brand).
Gemini focused on the brand as I requested, while ChatGPT’s poem was more centered on the cars produced by the company. ChatGPT seemed to take more creative liberty with the poem in ways that made no sense in some parts (case in point, “Ludicrous speed with just a glance”).
To me, ChatGPT’s poem reads like a junior literature student trying to impress the professor, and I prefer Gemini’s simple, straightforward take.
3. Image Creation
Gemini allows users to create unlimited images in the free version, while ChatGPT only allows you to create two for free and creating more requires a paid subscription.
I asked both AI chatbots to create an image, with the following prompt:
Create an image of King Kong swinging around in an industrial jungle
ChatGPT’s output was more detailed and could easily be a film still. On the other hand, Gemini’s creation was low-quality, low-resolution, and seemed to pay less attention to the prompt.
ChatGPT takes the lead here without a doubt.
4. Image Interpretation
Next, I compared the chatbots’ image interpretation abilities. I asked them to recreate a sign-up form in HTML and CSS using a screenshot.
Here’s the source screenshot:
Below are ChatGPT and Gemini’s attempts at replicating it using HTML and CSS:
When we compared the two chatbots’ ability to write code earlier, GPT-4 was the clear winner. Now, it seems Gemini has made reasonable progress in coding and can even produce a more visually appealing output compared to ChatGPT. Although, in the end, GPT-4’s recreation of the colors was more accurate compared to Gemini.
Language models are famed for inconsistent output, though, so your experience may vary.
Is Gemini Better than ChatGPT?
Gemini is not superior to ChatGPT in its current form. However, it is a capable AI chatbot capable enough to replace ChatGPT for most creative tasks. Gemini isn’t the ChatGPT-killer Google promised, but it’s still a useful model that benefits users and keeps OpenAI, the reigning champion, from resting on its oars.
