Moving Beyond LLMs

Large language models (LLMs) have made artificial intelligence (AI) widely visible by enabling systems that understand and generate text. However, LLMs are only one category in a broad ecosystem of AI models. Other models perform tasks such as vision, decision-making, numerical analysis, and automation. This post introduces common types of AI models and explains how they complement LLMs to power intelligent systems.

Large Language Model

An LLM is a type of artificial intelligence system designed to understand and generate human language. It is trained on vast amounts of text data to learn patterns in grammar, meaning, and context. These capabilities allow it to answer questions, summarize information, write text, and assist with problem-solving. By predicting and producing language one piece at a time, LLMs can interact in a conversational manner. Organizations use LLMs when they require high-quality language comprehension, reasoning, or generation, especially for tasks that are too large, repetitive, or complex for manual human processing.

Small Language Models

SLMs perform natural language tasks with significantly fewer parameters and lower computational requirements than an LLM. SLMs prioritize efficiency, faster inference, and deployment in resource-constrained environments such as edge devices, mobile applications, or cost-sensitive systems. While they may not match LLMs in breadth of reasoning, SLMs are effective for targeted use cases like text classification, intent detection, summarization, or domain-specific question answering. Typical applications include on-device or near-real-time text processing where low latency, low cost, and data privacy are priorities.

Large Action Models

LAMs are a type of AI model designed to plan, decide, and execute actions rather than only generating text. While LLMs produce language, LAMs take goal-directed actions in physical or digital environments. A LAM understands user intent, breaks it into steps, selects the appropriate tools, and carries out those steps in order. For example, a LAM might book a meeting by checking calendars and sending emails, or manage an IT workflow by querying logs and deploying changes.

Large Quantitative Models

LQMs are computational models that use mathematics, statistics, and numerical data to analyze systems, make predictions, or improve decisions at scale. Unlike language models, LQMs operate on structured, quantitative data such as time series, signals, graphs, or tabular datasets. These models are common in finance, economics, operations, engineering, and science, where precise numerical reasoning is required. Typical applications include risk modeling, pricing, demand forecasting, fraud detection, and large-scale simulations. Their defining feature is a focus on quantitative accuracy rather than natural language interaction.

Computer Vision Models

A computer vision model is an AI model designed to interpret and understand visual data, such as images and videos. It learns patterns in pixels—including shapes, colors, textures, and motion—to recognize objects, detect features, or track changes. The goal is to enable computers to extract meaningful information from visual inputs to support automated analysis. Computer vision models are used in image classification, object detection, medical imaging, autonomous driving, and quality inspection. Modern systems often use deep learning architectures, such as convolutional neural networks (CNNs) or vision transformers, and can be combined with other model types in multimodal systems.

Multimodal Models

Multimodal models are designed to process and relate information from different types of data simultaneously, such as combining text, images, and audio. Unlike unimodal models that focus on a single input type, these systems learn a shared representation of data across multiple formats. This allows the model to perform tasks such as describing the contents of a technical diagram or identifying security vulnerabilities in a screenshot of a dashboard. Common examples include Vision-Language Models (VLMs) and speech-to-text systems. Organizations use multimodal models to build more intuitive interfaces and to automate the analysis of diverse data streams that would otherwise require separate, siloed processing steps.

Graph Neural Networks

GNNs are a specialized class of deep learning models designed to process data structured as graphs, such as social networks, molecular structures, or infrastructure topologies. While traditional models excel with grid-like data such as images or sequential text, GNNs capture complex relationships and interdependencies between interconnected entities. A GNN represents data as nodes and edges, using a message-passing mechanism to aggregate information from neighboring nodes. For platform engineers, GNNs are particularly valuable for anomaly detection in dynamic network environments, dependency mapping in microservices architectures, and optimizing resource allocation across complex cloud footprints.

World Models

World models are an emerging category of AI designed to build an internal representation of an environment to predict its future states. These models do not just generate content; they simulate physical or digital systems to understand the consequences of specific actions. By modeling concepts like causality and spatial relationships, world models provide a foundation for autonomous agents to operate in unpredictable settings. They are frequently used in advanced robotics, autonomous vehicle simulation, and digital twins for complex industrial processes. For technology leadership, world models represent a shift toward AI that can reason about the "what-if" scenarios of system changes before they are executed in production

A Quick Comparison

Modern AI systems rarely rely on a single model. By combining LLMs with specialized models for vision, action, and quantitative reasoning, organizations build solutions that are efficient and reliable. Understanding these distinct roles clarifies how AI delivers value beyond simple conversation.