To understand computers, you first have to accept a physical reality. There is no magic inside a computer. Only electrical signals. More precisely, only high voltage and low voltage.

The base material for the chips we use today is silicon. Silicon is a semiconductor. Semiconductors have a physical property: by doping them with different elements and controlling the voltage on the electrode, you can control whether they conduct electricity or not. This tiny structure that can be turned on and off by electricity is called a transistor.

Modern CPUs contain tens of billions of transistors. TSMC’s current 3-nanometer process means these transistors are made so incredibly small that tens of billions can fit on a fingernail-sized piece of silicon.

You only need to remember one core function of a transistor. If you apply a high voltage to its control terminal, it conducts. If you apply a low voltage, it turns off. This is the bottom-line physical reality. We define high voltage as logical 1 and low voltage as logical 0. All of computer science is built on these two states.

With transistors, we can combine them into logic gates.

Suppose we connect two transistors in series. Current can only flow from one end to the other – outputting a 1 – when both control terminals receive a high voltage, i.e., both inputs are 1. If either one is 0, the current is cut off and the output is 0. This physical circuit is called an AND gate.

By the same logic, if two transistors are connected in parallel, current flows – outputting a 1 – as long as at least one input is 1. Only when both are 0 is the output 0. This is an OR gate.

Using a slightly different connection method, where inputting 1 gives output 0 and inputting 0 gives output 1, we create a NOT gate.

With AND, OR, and NOT gates, the physical world officially enters the mathematical world. I highly recommend a classic textbook here called Code, by Charles Petzold. This book has no fluff – it starts from the telegraph and explains how to wire up logic gates. It’s perfect for building that first layer of physical intuition. You should check it out after reading this post – it will absolutely blow your mind.

Now that we have logic gates, how do we make a computer do addition?

In binary, 1 plus 1 equals 10. That means the current bit becomes 0 and we carry 1 to the next position. This is actually two logical operations. First, determine the current bit: if both inputs are 1, the current bit is 0. If one input is 1 and the other is 0, the current bit is 1. This logic can be implemented by combining the basic gates we discussed – we call it an XOR gate. Second, determine the carry: only when both inputs are 1 do we carry, which is exactly what the AND gate does.

Connect an XOR gate and an AND gate together, and you’ve physically implemented the addition of a single binary digit. This is called a half-adder. Cascade multiple half-adders together, handling the carry signal, and you get a full-adder. If you connect 64 of these full-adders, you get a physical circuit that can add 64-bit binary numbers.

When you write the line of code a = b + c, what happens at the bottom level is that the electrical signals representing b and c surge simultaneously into this adder circuit made of tens of thousands of transistors. After an extremely short physical delay, the other end of the circuit outputs stable high and low voltages representing the result.

That is what the text mentions as the Arithmetic Logic Unit. It’s not some mysterious core component at all – it’s just a dense mesh of semiconductor logic gates, hardwired together according to mathematical rules that humans designed. Besides addition, through different wiring configurations, it can also perform subtraction, bit shifts, and even multiplication.

But being able to compute addition alone is useless. The result is just a bunch of electrical signals – once the power is cut or the input changes, the result disappears. A computer must be able to remember data. This brings us to memory and registers.

How do you use electrical signals to remember a state? Electronic engineers invented an extremely clever circuit called a flip-flop. This is a logic gate circuit with a feedback loop. Simply put, the output signal is fed back to the input. This design means that even after the original input is removed, the circuit can maintain its current high or low voltage state using internal feedback.

When you give this flip-flop a specific write signal, it updates its internal state and locks it. Unless you send it another write signal, it will hold that 0 or 1 indefinitely.

Arrange 64 flip-flops in a row, and you can store a 64-bit binary number. Inside the CPU, right next to the ALU, these ultra-fast storage units made of flip-flops are called registers. Their physical location is right beside the computation circuitry, meaning the electrical signal travels the shortest possible distance, so they are extremely fast. Of course, they’re also extremely expensive to make, so they can only be very small.

Here I have to forcefully recommend an absolutely legendary book: Computer Systems: A Programmer’s Perspective, better known in the CS world as CSAPP. This is the textbook used at Carnegie Mellon University. What makes it incredible is that it bridges the gap from assembly language to hardware architecture to operating systems. Once you understand it, you’ll know why some code runs fast and some runs slow – because you’ll start writing code with a physical-level mindset.

If you want to go further and systematically bridge operating systems, computer organization, networking, and other hardcore fundamentals all at once, I’ve put together a collection covering data structures and algorithms, computer organization, and more.

Over 1 million downloads across the web – computer fundamentals that will supercharge your skills.

Let’s keep moving outward. What if registers aren’t enough? We need larger storage. This is where the RAM sticks plugged into your motherboard come in. The technical name is Dynamic Random Access Memory, or DRAM.

The physical principle of DRAM is completely different from registers. Instead of flip-flops, it uses tiny capacitors to store electrical charge. A capacitor with charge is 1, without charge is 0. Because the capacitor structure is extremely simple – needing only one transistor and one capacitor to store one bit of data – the capacity can be made very large. Nowadays, dozens or even hundreds of gigabytes are common.

But capacitors have a fatal flaw: they leak charge. If left alone, the charge representing 1 leaks away completely within milliseconds, becoming 0. So the motherboard must have a dedicated control circuit that reads the data from memory and recharges it at extremely short intervals. This is called dynamic refresh. This is why when you unplug your computer, everything in memory disappears instantly.

Now we have an ALU that can do math, and registers and memory that can store data. The core question is: how do these lifeless things actually start moving? Who commands them to do addition first, then multiplication, and finally store the result?

This is the job of the control unit, and it’s the core mechanism that makes a computer a machine.

The motherboard has a crystal oscillator – a quartz crystal that generates a fixed-frequency pulse signal as long as power is applied. For example, if your CPU is rated at 4.0 GHz, it means this crystal generates 4 billion high-to-low voltage transitions every second.

This is the heartbeat of the computer. All the logic gates and flip-flops inside the CPU operate in strict synchronization with this heartbeat.

Inside the CPU, there is a very special register called the program counter. It holds nothing but a memory address.

When the first beat of the heartbeat strikes, the CPU does the first thing: fetch the instruction. It sends the address stored in the program counter to the memory through the metal traces on the motherboard. Memory sends back the long string of 0s and 1s stored at that address.

This long string of 0s and 1s is the instruction. Once the instruction returns to the CPU, it enters the control unit. The control unit is also just a bunch of hardwired logic gates – its job is decoding. For example, when it sees the first few digits of the incoming 0s and 1s are 1011, its internal wiring structure determines that these high and low voltages happen to activate the switch leading to the addition circuit in the ALU. At the same time, the rest of the instruction specifies which registers hold the two data values to be added.

When the next beat strikes, the execution phase begins. The activated addition circuit receives the data, electrical signals flash through the adder’s logic gate network in an instant, and the result is produced.

On the next beat, the result is written back to the specified register.

Finally, the program counter automatically increments by one, pointing to the next instruction in memory. Then the cycle repeats. Billions of times per second.

This is the complete bottom-level truth of how a computer runs. There is no advanced intelligence whatsoever – only a crystal oscillator blindly ticking away, a program counter mechanically incrementing addresses, a control unit robotically switching signal paths based on the 0s and 1s it reads, and an ALU passively converting incoming voltage levels into new voltage levels.

The Python code or Java code you write in your daily work is ultimately translated by a piece of software called a compiler into a long, tedious sequence of machine instructions specific to the CPU architecture – pure 0s and 1s.

At this point, you’ve actually completed the entire chain from sand (silicon) to program. Many people feel they understand at this stage, but in the real-world industry environment today, merely understanding traditional CPU principles doesn’t even meet the passing line.

Given the current hottest topic of AI large models, I have to expand your horizons. Let’s look at what the real computing power landscape looks like today.

Any team working on large models today, whether for training or inference, absolutely cannot use CPUs to compute. Why?

Because CPUs were designed to handle complex logic control. Look at a CPU: a large portion of its area is devoted to branch prediction, cache management, and the control unit. The portion actually used for pure numerical computation – the ALU – is not that large. It’s like the CPU is a highly educated intellectual who excels at complex management processes. Ask him to compute 1+1, he can do it, and he might even analyze whether there’s an error along the way – but he can only handle a few at a time.

The mathematical essence at the core of large models is extremely large-scale matrix multiplication. For example, multiplying two matrices of several thousand by several thousand. This requires hundreds of millions of multiplication and addition operations. The logic is extremely simple – multiply-add, multiply-add – with no complex branching at all, but the data volume is staggering.

If you used a traditional CPU to compute this, it could indeed execute billions of instructions per second, but faced with hundreds of billions of parameters, it would be a drop in the bucket.

To truly understand what large models are actually computing, if your mathematical intuition still relies on rote formulas, I strongly recommend you check out 3Blue1Brown’s “Essence of Linear Algebra.” To make review and derivation easier, you can pair it with this compiled set of 3Blue1Brown notes in both Chinese and English. It genuinely achieves intuitive visualization of advanced matrix multiplication and linear transformations. After watching it, you’ll have a real physical-level mental image of multiply-accumulate operations across thousands upon thousands of parameters.

3Blue1Brown Linear Algebra Notes: Possibly the best Chinese-English compilation on the web

The nature of this massive matrix computation gave birth to the GPU, the Graphics Processing Unit – which should now be called the General-Purpose Parallel Compute Unit.

The design philosophy of a GPU is completely opposite to that of a CPU. It cuts away most of the complex control circuitry and large caches, filling every bit of freed-up physical space with the simplest possible ALUs. A current NVIDIA H100 chip contains tens of thousands of compute cores. Although each core may run at a lower frequency than a CPU core and has weak control logic, they can all work simultaneously.

Think of a GPU as a formation of 100,000 elementary school students who only know arithmetic. You give them an extremely simple command: multiply the number on your left by the number on your right. All 100,000 students compute the result simultaneously on the same clock tick. For the massively parallel matrix operations required by large models, a GPU is hundreds to thousands of times more efficient than a CPU.

Actually, AI – just like the CPU’s adder – isn’t really that mysterious. It’s still just layer upon layer of logic and matrices. If you want to build your understanding of deep learning from the ground up, I recommend a book that many on Twitter have praised for thoroughly demystifying AI: Deep Learning with Python. Once you understand the tensor computations in forward and back propagation, you’ll realize there’s no magic inside large models either.

After reading this book, I was completely demystified about AI!

Back to the hardware. When we do low-level algorithm optimization, most of our effort goes into fighting with VRAM bandwidth. This is also where many high-level programmers trip up.

Because the ALU computes so fast. As mentioned earlier, memory modules store data by charging and discharging capacitors, and they are physically some distance away from the CPU or GPU – electrical signals take time to travel across the motherboard. The biggest challenge facing AI chips today is what academia calls the von Neumann bottleneck.

That is: thousands of compute cores finish processing the data in their hands in an instant, and then they can only sit idle, waiting for new data to be moved from slow memory. Even if you make the on-chip computation circuits faster, if you can’t feed the data fast enough, the compute power is wasted.

To solve this problem, hardware engineers have been pushed to extremes. Since ordinary memory modules are too far and too slow, why not package the memory and GPU chip directly on the same substrate? To increase the channel width for data transmission, instead of using traditional wires, they use a technology called silicon interposers – drilling thousands of microscopic conductive vias inside the silicon to directly connect the memory and compute cores. This is what’s been hyped to the skies as High Bandwidth Memory, or HBM.

If this part excites you deeply and you want to secure an irreplaceable high salary as an architect or low-level developer in the coming AI era, I strongly urge you to tackle Computer Architecture: A Quantitative Approach. This book was written by two Turing Award winners, and almost every hardware engineer in Silicon Valley has a copy. It uses massive amounts of real data and formulas to tell you why chips must be designed with specific cache architectures and exactly where the bottlenecks in multi-core parallelism lie. Although it’s incredibly dry to read, and some chapters are very difficult, it is the necessary path to bridging the cognitive gap in low-level understanding.

Back to your original question. Why do you still feel confused after learning all those terms? Because traditional teaching severs the flesh-and-blood connection between physical entities and logical abstractions.

When you treat the CPU as a magical black box, those terms feel like incomprehensible definitions. But if you follow the laws of physics upward:

• Because high and low voltages can form logic gates, we got arithmetic units that can do math.
• Because feedback circuits can lock voltage states, we got registers that can store data.
• Because we needed to rhythmically control the flow of electrical signals, we got crystal oscillators and control units.
• Because this system is too complex for humans to input voltage levels directly, we invented instruction sets, wrapping 0s and 1s into assembly language, and finally into C and Python.

As a practitioner who has been through the grind in this industry, I can tell you very responsibly: in this era, frontend frameworks come and go, backend microservice architectures have their moment for a few years, and even AI large-model architectures are iterating at breakneck speed. Many high-level application development techniques you learn today could be obsolete next year. In fact, large models themselves can already do many basic coding tasks better than the average programmer.

But the one thing that never changes is this set of fundamental computer operating principles rooted in semiconductor physics. No matter how software is abstracted, no matter how sophisticated algorithms become, everything ultimately comes down to voltage fluctuations on a piece of silicon. The more advanced the performance optimization, the more cutting-edge the compute scheduling, the more engineers need the ability to see through all software abstraction layers and directly perceive the underlying hardware’s operational state.

Don’t think this low-level knowledge is far removed from you. When you encounter a piece of extremely performance-hungry code, and because you understand the cache line mechanism at the bottom level, you adjust the arrangement of arrays in memory to dramatically improve CPU cache hit rates – making the code run ten times faster – in that moment, you’ll feel the immense sense of control that comes from mastering the underlying principles.