The field of artificial intelligence is booming, and it’s hard to keep up with all the available models and tools, or to even figure out what’s at our disposal. With so many names like Stable Diffusion, DALL-E, or Midjourney, it can be overwhelming to know where to start. Simple questions like, “Where should I go if I want to generate an image?” “And where if I want it for free?” “What options do I have?” can become more complicated than necessary.
The following list of resources aims to be an updated guide to the most significant tools in today’s AI landscape. I hope you find it helpful.
Proprietary models
Proprietary models are typically the most powerful, created and trained by companies that keep them in-house, and their source is not accessible to the public. These models are often hosted by the companies that produce them and made accessible through paid APIs, although they can be presented in other ways.
| Text to text | Text to image | Vision | Audio | Text to speech | Embeddings | |
| OpenAI | O1 GPT-4o / mini GPT-4 / Turbo GPT-3.5 Turbo | Dall-E 2 Dall-E 3 | GPT-4o GPT-4 / Turbo GPT-3.5 Turbo | Whisper (transcription) GPT-4o | TTS | Embeddings |
| Anthropic | Claude 3 | Claude 3 | ||||
| Gemini 1.0 Ultra Gemini 1.5 Pro Gemini 1.0 Pro Gemini 1.5 Flash | Gemini 1.0 Ultra Gemini 1.5 Pro Gemini 1.5 Flash | Gemini 1.0 Ultra | Embeddings | |||
| Midjourney | Midjourney | |||||
| Meta | Emu |
OpenAI
The company that has triggered the recent boom in artificial intelligence with its GPT models, introducing the general public to the capabilities of large language models with the launch of Dall-E and (especially) ChatGPT in late 2022.
OpenAI offers a multitude of models for different use cases: text and image generation, transcription, etc.
O1-preview / mini
OpenAI introduced the O1 model family on September 12, 2024. This new family breaks away from the name their language models had been using until now (GPT) to represent that it is something different. The focus of these models is on improving reasoning, making them especially useful for tasks that require logical analysis and complex problem-solving or programming.
For now, the preview model has been launched, which is expected to be soon replaced by improved versions, and the mini, which is smaller (and five times cheaper).
- Capabilities: Natural language model (generating text output from natural language text) oriented towards reasoning.
- Access mode: Web interface (free), API (PPU), Realtime API (PPU).
GPT-4o mini
Their newest model, lanuched on July 18, 2024. A smaller and more efficient model than GPT-4o, is cheaper than GPT 3.5 Turbo with better performance. Has replaced GPT-3.5 Turbo as the model underlying ChatGPT’s free tier.
- Capabilities: Multimodal model, which incorporates natural language and vision (according to OpenAI, audio will be rolled out later).
- Access mode: Web interface (free), API (PPU).
GPT-4o
Their most capable model, launched on May 13, 2024. In September, it also became accessible through Realtime API, which allows for real-time voice conversations (minimal latency).”
- Capabilities: Multimodal model, which incorporates natural language, audio and vision.
- Access mode: Web interface (subscription), API (PPU).
GPT-4 and GPT-4 Turbo
- Capabilities: Natural language and vision model (generate text output from text prompts in natural language and images).
- Access mode: Web interface (subscription), API (PPU).
GPT-3.5 Turbo
- Capabilities:Natural language model (generate text output from text prompts in natural language).
- Access mode: API (PPU).
Dall-E 3
- Capabilities: Image generation from text descriptions.
- Access mode: Web interface (subscription), API (PPU).
Dall-E 2
- Capabilities: Image generation from text descriptions.
- Access mode: API (PPU), web interface has been discontinued.
TTS (Text-To-Speech)
- Capabilities: Convert text into audio (readout of given text).
- Access mode: API (PPU).
Embeddings
- Capabilities: Generate embeddings (numerical vector representations) from text.
- Access mode: API (PPU)
Anthropic
Anthropic is an AI company founded in 2021 by ex-OpenAI researchers. Although it’s an independent company, it has received big investments from Amazon, and their models have specially good integration with AWS tools.
They have launched the Claude 3 family of models, with three different sizes: Opus (largest), Sonnet and Haiku (smallest), and more recently, 3.5 Sonnet, which at the moment is their most capable model. 3.5 Haiku and Opus are expected to be available before the end of 2024.
Free access offered through a web console, which oscilates between Sonnet and Haiku (Opus not available for free).
Claude 3.5 Sonnet
- Capabilities: Natural language and vision model (generate text output from text prompts in natural language and images).
- Access mode: Web interface (subscription), API (PPU).
Claude 3 Opus
- Capabilities: Natural language and vision model (generate text output from text prompts in natural language and images).
- Access mode: Web interface (subscription), API (PPU).
Claude 3 Sonnet
- Capabilities: Natural language and vision model (generate text output from text prompts in natural language and images).
- Access mode: Web interface (free), API (PPU).
Claude 3 Haiku
- Capabilities: Natural language and vision model (generate text output from text prompts in natural language and images).
- Access mode: Web interface (free), API (PPU).
Currently offers three language models within the Gemini family, Ultra, Pro and Flash, with Ultra being the most powerful; in addition to vision models and embeddings. It has recently launched Gemini 1.5 Pro, and 1.5 Flash, which is the lightest and fastest.
The main feature sets the Gemini models apart form from Claude and GPT is the large size of their context window, currently at one million tokens.
Gemini 1.0 Ultra
- Capabilities:Natural language model (generate text output from text prompts in natural language), and vision for both image and video.
- Speech recognition.
- Access mode: Web interface (subscription).
Gemini 1.5 Pro
- Capabilities:Natural language model (generate text output from text prompts in natural language), and vision for both image and video.
- Access mode: Google AI studio (free), API (free for low usage, or PPU).
Gemini 1.0 Pro
- Capabilities:Natural language model (generate text output from text prompts in natural language), and vision with a separate specific model.
- Access mode: Google AI studio (free), API (free for low usage, or PPU).
Gemini 1.5 Flash
- Capabilities:Natural language model (generate text output from text prompts in natural language), and vision for both image and video.
- Access mode: Google AI studio (free), API (free for low usage, or PPU).
Embeddings
- Capabilities: Generate embeddings (numerical vector representations) from text.
- Access mode: API (PPU)
Midjourney
- Capabilities: Image generation from text descriptions and other images.
- Access mode: Discord (subscription).
Inflection
Inflection is the company behind Pi, an assistant that comes across as more human and empathetic, due to its language model specially trained to enhance these traits.
Inflection-1
- Capabilities:Natural language model, especially trained for human connection.
- Access: Exclusively through the Pi assistant (free).
Meta
Meta has openly published their Llama models, but their image generation model, Emu, they keep proprietary.
Emu
- Capabilites: Image generation from text descriptions.
- Access model: Meta AI (free) in multiple channels including web console and Whatsapp and other meta products.
Open models
Open models are models that, once trained, are made available to the public. This allows anyone (in theory, although there may be limitations in the licenses) to download them, fine-tune their training for specific uses, host them on their own server, or even (if they have the appropriate hardware) run them locally on their machine.
Meta
Facebook’s parent company has made its Llama models public, with Llama 3.1 being the most recent (released in July 2024). At the time of their release, they are the most advanced open language models.
Llama 3.2
Launched in September 2024, the Llama 3.2 family includes 4 models split in 2 groups: lightweight (1B and 3B) and multimodal (11B and 90B).
The multimodal models can take both image and text as input, and generate text output, while the lightweight ones are only text for input and output, and are intended to be deployed on-device.
Llama 3.1
Family of open models launches in July 2024. There are three versions, 8B, 70B and 405B (8, 70 and 405 billion parameters, respectively). They are currently the most powerful open language models available.
Capabilities: Natural language model (generate text output from text prompts in natural language).
Llama 3
Family of open models launches in April 2024 and available in two versions, 8B and 70B, with the latter’s capabilities being comparable to those of Gemini Pro 1.0 or Claude 3 Sonnet.
- Capabilities: Natural language model (generate text output from text prompts in natural language).
In addition to the Gemini family of proprietary models, Google also has a family of open models: Gemma. They have recently launched Gemma 2, in 9B and 27B versions.
Gemma 1
- Capabilities: Natural language model (generate text output from text prompts in natural language).
- Avalilable in standard and CodeGemma variants, the latter one specialized in code generation; each with 2B and 7B versions.
Gemma 2
- Capabilities: Natural language model (generate text output from text prompts in natural language).
nVidia
On September 17, 2024, nVidia launched its family of open multimodal models NVLM 1. For now, only the first model of 72B is available, but preliminary comparisons put it on par with Llama 3.1 405B or GPT-4o.
- Capabilities: Natural language model (generating text output from natural language text) and vision.
Stability AI
Stability AI is main company behind the famous image generation model family Stable Difussion They publish hosted APIs to access their models on a pay-per-use fashion, like OpenAI or Anthropic, but they also publish them for anyone to download, self-host and modify.
Their latest image generation model, Stable Diffusion 3, is only available through their API, but they have committed to release it openly at some point (although, truth be told, this was posted by Emad Mostaque, who stepped down as CEO on March 23, 2024, so it may not be taken as binding).
Some of their released models:
- Stability Video 3D (video generation)
- SD-Turbo (image generation)
- SDXL-Turbo (image generation)
- Stable Video Diffusion (video generation)
- SDXL (image generation)
Mistral AI
Although the French company Mistral AI has also commercial models, they are better know for their open source ones. All of them are language (text to text) models.
Their open source models published to date are:
- Mistral 7b, their first dense model, released September 2023.
- Mixtral 8x7b, their first sparse mixture-of-experts, released December 2023.
- Mixtral 8x22b, their best open source model to date, released April 2024.
Microsoft
Microsoft has published a family of smaller language models, Phi-3. The family consists of three models, but for now only one, Phi-3-mini (3.8B), the smallest, is available.
xAI
Elon Musk’s artificial intelligence company has released its Grok-1 language model with open access (and Apache 2.0 license).
OpenAI
Most OpenAI models are proprietary, but the exception is their Whisper model, which they made public with an MIT license and can be freely downloaded.
Whisper
- Capabilities: Transcription; convert spoken audio into text.
- Access mode: API (PPU), open-source download.
AI integrators
In this category, I include companies whose offering is not based on creating their own models, but rather, uses models from others to create innovative products or services.
Hugging face
Hugging Face is the most important free distribution platform of AI models.
Groq
Groq is a company that has developed their own hardware to run model inference, making their execution ultra-fast. They offer a free chat to access a number of open source model running on their hardware, as well as model hosting services.
Perplexity
Perplexity is a chatbot with both a free version and a paid version. The free version operates on Perplexity’s own model, while the paid version allows for the usage of other models such as Claude-3, Llama 3, GPT-4, among others.
Besides serving as an interface for using other models, Perplexity has a special feature that enables it to perform internet searches to complement the information in its responses, which allows it to display more recent information, whereas other chatbots are limited by the information available at the time their training was completed.