Transformer architecture was the breakthrough that made AI chatbots possible

Transformer architecture can recognize and predict patterns in language

The underlying software powering text generation in AI tools associates each word or subword—called a token—with a set of values corresponding to how often it appears near other tokens. By recognizing these associations in prompts, the LLM can infer meaning.

AI gained mainstream attention with tools like IBM Watson and Apple’s Siri

With the release of ChatGPT in 2022, which drew over 100 million weekly users in just two months, natural language processing and understanding could be achieved at scale via machine learning. Unlike earlier artificial intelligence that could pull stored knowledge, generative AI produces text, images, or sounds.

Google invented the foundation of the AI movement—but ChatGPT was a shock

For years, Google was the cutting-edge of the AI world, including the development of the Transformer—the basis of contemporary AI chatbots. But since OpenAI debuted ChatGPT in 2022, the company was had to play catch-up. This is a four-hour deep dive into Google's AI history.

AlexNet was the AI tipping point that set the stage for ChatGPT

AlexNet is a neural network created in 2012 that laid the groundwork for modern AI models. Before AlexNet, the ideas behind neural networks existed, but the computational power to run them didn’t.

View a timeline of AI deployments from 2017 to 2025

The development of transformer architecture and the release of GPTs across AI labs led to rapid changes in LLM scale and power. Companies like OpenAI, Google, Meta, Microsoft, and Anthropic have invested billions of dollars to develop reliable, versatile models.

Stephen Wolfram on AI's rapid progress & the "Post-Knowledge Work" era

AI, as a concept, has been around since at least the 1950s, but it's only in the last year or so that the technology has been able to equal humans at many tasks -- and in many cases surpass humans. In this podcast, mathematician and computer scientist Stephen Wolfram provides an AI 101, explaining how we got here, how highly advanced chatbots like ChatGPT work, and what our world is in store for next.

'Context rot' explains why AI conversations degrade the longer they run

Research shows that as a large language model's context window fills up, performance declines, and models begin over-indexing on content at the start and end of a conversation while losing track of what's in the middle. The practical fix is simpler than you'd think: start a fresh chat more often.

The AI that creates any picture you want and how it works

Since January 2021, advances in AI research produced a plethora of deep-learning models capable of generating original images from simple text prompts. Researchers at OpenAI, Google, Facebook, and others have developed yet to be released text-to-image tools, and similar models have proliferated online. This video is a primer on how we got here, how the technology works, and its implications.

Agentic AI systems can proactively achieve user goals without user direction

Unlike large language models, which do not take follow-up actions after generating their output, agentic AI uses an initial prompt to take multiple steps, learn from its environment and outcomes, and continue working without follow-up human prompting.

Chat with famous, historical, and fantasy figures with this chatbot

Character.ai—a generative AI-based chatbot platform—allows you to have free-flowing conversations with a wide range of people. From celebrities to famous figures from the past, the platform relies on a neural language model to process huge amounts of text, learning from conversations with users to develop responses. A number of observers have referred to the platform as "making AI fun," with user time on site surpassing a whopping two hours per day.

Artificial general intelligence equals or surpasses human intelligence

AGI incorporates sensory perception, memory, and advanced logical inference systems to move beyond the narrow tasks seen in large language models and chatbots. One of the most significant hurdles in developing AGI is designing systems that learn and can flexibly apply learning across domains.

Understanding how Claude Code and other AI coding agents function

Under human oversight, a supervising large language model interprets user tasks and delegates work to subordinate LLMs, which can generate code, fix bugs, and run tests, often most effectively for proofs of concept. Incremental backups and versioning are crucial when using such agents, which can lose details during their work as a result of compressing context history to work around memory limitations.

Neural networks are modeled after an idealization of the brain

These networks—including the backbone of ChatGPT-3, which has approximately 175 billion parameters—naturally discover the implicit rules of grammar and syntax within language by identifying patterns. Deviations in word patterns add creativity to the text generation.

Generative AI tools excel at pattern recognition, not contextual accuracy

Large language models are trained on vast amounts of unstructured data, from which they develop parameters for grammar and associations between words. These connections can introduce errors due to inapplicable reasoning when used on new data in unfamiliar contexts.

Machine learning lets computers learn from data instead of relying on programming

ML is a core part of artificial intelligence and is foundational to applications like chatbots, fraud detection, and self-driving cars. Businesses are also adopting it to improve efficiency and identify patterns in data, though the latter may be inaccurate.

The work of Claude Shannon, for whom Anthropic's AI chatbot is named

Known as the father of information theory, Claude Shannon applied Boolean algebra—a subset of mathematics that quantifies true or false statements as 1 or 0, respectively—to circuit design to develop the concept of a binary digit. His paper "A Mathematical Theory of Communication" (1948) laid the groundwork for all modern digital systems.

AI systems identify patterns involving physics and time through 'world models'

These realistic virtual environments, such as driving simulations seen in video games, allow these systems to "learn" properties about reality beyond what can be captured in samples of text, images, and videos. Training AI in these simulated environments can help develop "physical AI" for automated robotic systems. (Some readers may experience a paywall.)

The hardware powering AI was originally designed for video game graphics processing

Graphics processing units are specialized computer chips that can process multiple data streams in parallel, including pixel colors for a display. Such processing is required to find connections across text data and generate outputs quickly.

How Model Context Protocol allows AI systems to communicate with databases

Developed by Anthropic, the open-source software serves as an integration layer that standardizes how large language models connect to real-time information sources or tools. Just as a USB-C port serves as a universal standard for many peripherals, MCP streamlines these integrations and provides context about external sources to LLMs in a way that AI can understand.

Using AI to restore speech 30 years after a stroke

Through a novel surgical procedure involving electrodes implanted in the brain, along with the help of artificial intelligence, a stroke patient is regaining her ability to speak. This technology converts her thoughts into speech through a digital avatar, allowing her to verbally express herself for the first time in decades.

What does it mean for AI to "understand" something?

This Quanta article discusses how programmers are working to create a reliable test for the depth and extent of the knowledge of machines. What does AI know and how does it know it, if it knows anything at all? There's been progress, but it hasn't been easy. Even humans can still be fooled some of the time.

AI expert shows the cyclical nature of AI hype—in 1988

Rodney Brooks wrote several decades ago about the boom and bust cycles of artificial intelligence. Starting from the 1950s excitement over ENIAC's multiplication through the second launch of then-novel neural networks, Brooks suggests AI progresses incrementally despite the hype.

The term 'artificial intelligence' was coined in 1956

Dartmouth professor John McCarthy introduced the term during a summer workshop aimed at exploring how to make machines perform human-like tasks. Two years later, Perceptron Mark I became one of the first neural network machines.

Teaching a robot to laugh

Researchers have designed a shared-laughter AI system that responds to human laughter in order to build a sense of empathy into dialogue. Learn how they did it with this explainer.

This site reaches millions of visitors who are tired of AI chatbots and seeking human connection. Pretend to be an AI chatbot and ask questions, provide answers, and draw for people in real time.

As of 2025, the internet is entering its fourth phase, with three more to go

Since the inception of the internet as a tool for connecting teams of researchers, technological and infrastructure advancements have progressively increased connectivity. Connections between computers, mobile devices, all devices, and now AI agents will be succeeded by an internet that integrates perception and the senses toward an eventual quantum internet.

Watch how algorithms can be designed to train themselves and self-improve without human input, to the point where the optimized code is too complex for programmers to understand.

English mathematician Ada Lovelace wrote the first computer algorithm a century before computers existed and hypothesized that computing devices would one day be able to compose music—foreseeing the generative AI systems of today.

A vending machine run with Anthropic's AI went out of business after it was convinced to give away items for free and sell a video game console.

Electrical grids worldwide must double their power delivery capacity by 2050 to meet surging demand from AI and electric vehicles.

The name 'Silicon Valley' arose from the founding of Shockley Semiconductor Laboratory in Fairchild Semiconductor in 1956-57, founded near Stanford University in Palo Alto

How Amazon operates a 'flywheel' business model, a self-reinforcing cycle that drives continuous growth

The first robot in literature may have been a 'mechanical man' in 1907's 'Ozma of Oz,' a book in L Frank Baum's 'Wizard of Oz' series.

The history of many kitchen appliances involves unexpected discoveries and applications, including wartime radar turned hamster defroster and replacing an 'ice king' with heat-stealing boxes.

As of 2025, the most cited papers in science include a review of graphene, methods to improve image recognition, streamlining electron density calculations, and optimizing protein studies.

Recent research found that the brain doesn't seem to massively rewire following amputation—upending earlier beliefs about the cause of phantom limb pain.

The animatronic shark from Steven Spielberg's 'Jaws,' the first modern blockbuster, was a complex underwater apparatus called 'Bruce.'

Crows are believed to have intelligence on par with a seven-year-old child and demonstrate complex behaviors, from problem-solving with tools to recognizing human faces.

Chaos theory—informally known as the butterfly effect—is the reason why weather forecasts are presented with probabilities rather than certainties.

The power and influence of the American presidency have expanded significantly over time—far beyond what the founders envisioned.

Sony set the standard for cylinder-shaped batteries when it repurposed manufacturing equipment used to make rolls of magnetic recording tape.

How fusion—which releases energy when two lighter atoms combine to form a heavier atom—differs from fission

Long before Major League Baseball began allowing instant replay for balls and strikes, an umpire reluctantly used the technology to determine whether a hit was a home run, a move that inadvertently ushered in a new era for the sport.

The word 'jazz' evolved from 'jasm,' 19th-century American slang for 'energy'

In recent years a child anxiety expert has created and tested a novel phobia and anxiety treatment approach for children, focused on changing how parents respond to the fears.

The Mona Lisa's celebrity status and repeated restorations have obscured what made the painting so revolutionary for its time

Learn how monoclonal antibodies, which mimic the body's antibodies, can fight diseases like Ebola.

Memecoins are a type of cryptocurrency—a form of digital currency with a wide variety of uses—created around popular 'memes,' or viral internet jokes.

Objects can be 3D printed with light by projecting 2D computerized tomography (CT) scans onto rotating cylinders of resin, which solidify upon exposure.

For centuries, it was common to sleep in two shifts: a few hours at the beginning of the night, and a longer period beginning around midnight