The Causal Boundary

1 The Causal Boundary

Purpose

Physical AI stack focus for this chapter.

Why can a better prediction still produce a worse physical action?

A learned model can recognize a task correctly and still act too late to stop a moving machine or prevent excessive contact force. Once a command produces motion or heat, replacing the prediction cannot undo the work already delivered. The engineer must therefore connect model quality to the body’s limits, the time available to respond, and the evidence supporting the next action. An independent permission path must be able to refuse a proposal and carry out a feasible fallback even when the model stalls. At the causal boundary, where commands become physical work, irreversibility makes these responsibilities part of the system’s architecture.

Learning Objectives

• Trace how machine learning systems evolved from disembodied prediction through digital agency to physical embodiment, and explain why that progression gives data, compute, evaluation, and deployment physical stakes
• Explain why an action that crosses the causal boundary requires reasoning about consequences that software rollback cannot undo
• Calculate how response delay consumes a moving machine’s stopping clearance
• Apply the scope test to distinguish physical AI from advisory learning and classical control
• Distinguish learned proposals from permission to execute them in the five-level machine
• Diagnose how perception, response timing, and missing redundancy interacted in the Tempe collision
• Relate the four laws to their established names in the field and to the book’s four parts

1.1 Artificial Intelligence Beyond Glass

When an image classifier mislabels a warehouse photograph on a server, a downstream filter can discard the label before any machine moves; when a mobile robot’s policy misjudges aisle clearance and drives its wheel motors, the chassis is already accelerating toward the rack. Across offline advisory prediction, digital agency, and physical embodiment, computation acquires different consequences as its outputs enter an action loop, and physical AI diverges from the other two in how agency, time, and recovery behave (table 1).

In the offline or advisory case, a model delivers predictions against a static dataset before any downstream action closes a feedback loop. Whether classifying an image, predicting customer churn, estimating a bounding box, or ranking search results, the model operates in open loop: its input distribution is exogenous because the predictions it emits do not alter the process that generates later test examples. Because those requests arrive independently, throughput-oriented serving can batch many inputs together (\(\BatchSize \gg 1\)) across pipeline stages, performing many arithmetic operations for every byte of weights fetched from memory (its arithmetic intensity). The advisory model exercises no agency of its own. Before a person or downstream service acts on the output, an incorrect label or irrelevant search result can be ignored, filtered, or recalculated without having done physical work.

The emergence of agentic machine learning marked an architectural transition from passive, single-shot prediction to active, sequential decision-making. Endowed with reasoning loops and test-time deliberation, an agentic model formulates multi-step plans, maintains internal state across execution traces, and autonomously invokes external tools: executing database queries, calling web APIs, running generated code in execution environments, and coordinating software subagents. In this regime, the execution loop is closed. An action taken at step \(\TurnIndex\) alters the state of the software environment, so subsequent observations \(\TrajectoryObservation_{t+1}\) depend directly on prior decisions \(\TrajectoryAction_t\). The system possesses genuine agency.

Within a controlled software environment, this agency can operate inside a protective envelope, “behind the glass.” Such environments can grant agentic systems three structural privileges, though external services and consequential tool calls may fall outside that envelope:

• Elastic virtual time: A controlled software environment can wait while the model computes. Whether an agent requires \(200\text{ ms}\) or \(30\text{ s}\) of test-time compute to plan its next tool call, the environment can hold its simulated state until the action is emitted. External services continue evolving on their own clocks.
• Transactional rollback and sandboxing: Tool execution can occur within isolated containers, virtual machines, or transactional databases. When an action produces an error inside this envelope, software exceptions catch the fault, database transactions roll back, or the container restarts from a clean snapshot. These mechanisms cannot undo consequences already delivered outside it.
• Discrete symbolic interfaces: Actions are structured strings, JSON schemas, or API payloads whose syntax and validity can be verified by deterministic compilers and linters before execution.

Physical artificial intelligence represents the arrival of agency in the physical world. A physical AI system does not invoke a software API inside a virtual sandbox; its outputs cross the causal boundary into physical actuators. A learned model’s output becomes a setpoint (the target value a low-level controller drives toward), the setpoint becomes electrical current in a motor phase winding (one of the stationary copper coils whose current turns the motor), and the current becomes electromagnetic torque that accelerates physical mass through gearboxes, linkages, and contact surfaces sharing space with people and objects. That physical embodiment strips away every digital privilege:

• Continuous time: Physical time advances under Newtonian mechanics regardless of processor state. The physical plant, the apparatus being controlled such as a robot’s wheeled base or arm, does not pause while a neural network deliberates. In a moving robot, computational latency is not a software delay. A machine moving at speed \(\BodySpeed\) that takes a time \(\PrebrakeDelay\) to begin braking covers \(\BodySpeed\,\PrebrakeDelay\) toward an obstacle under unguided momentum. Delay is physical distance.
• Mechanical and thermodynamic irreversibility: Physical state transitions admit no software rollback. Once software commands current through an inverter (the power electronics that switch a DC supply into the motor’s phase currents), electromagnetic force accelerates mass, dissipating irreversible Joule heat (\(\PhaseJoulePower = \MotorPhaseRmsCurrent^2 \PhaseResistance\) in each phase winding) and altering mechanical contact states. Kinetic energy (\(\KineticEnergy = \frac{1}{2}\Mass \BodySpeed^2\)) cannot be garbage-collected, and a mechanical impact cannot be undone by restoring a checkpoint. A command that has moved a machine can be answered only by another physical action, which itself consumes time, clearance, and energy.
• Single-batch, hard real-time execution: On the running machine, each new action proposal depends on observations shaped by prior motion, so the live loop does not accumulate independent observations into a throughput batch (\(\BatchSize=1\)). Control loops must also meet hard real-time deadlines across several coupled timescales: motor current loops at tens of kilohertz, loops near a kilohertz that check motion and regulate contact force, learned policies renewing at tens of hertz, and slower models interpreting the scene a few times per second. Section 1.7 gives the running machine’s rates. A missed deadline does not degrade user experience; it can destabilize the physical plant.

Losing these privileges forces an architectural response. Deep neural networks are statistical function approximators, vulnerable to distribution shift, sensor noise, and hallucination, so they cannot be granted unmediated authority over physical actuators. Physical AI therefore separates proposal from permission. High-capacity learned models act only as proposers in an unprivileged tier (the Brain), while an independent, deterministic permission path (in the Nervous System) checks each proposal against the machine’s physical limits, admits (accepts for execution) only the proposals that pass, and keeps an executable fallback ready on isolated real-time hardware before any action crosses the causal boundary into the plant (the Body).

Table 1: Three deployment regimes of machine learning systems: The selected arrangements progress from offline advisory prediction to digital agency to physical embodiment. The unit of engineering shifts from advisory prediction to digital tool execution, and finally to irreversible physical action across the causal boundary.

Deployment Regime	Execution Substrate	Primary Output	Environmental Coupling	Time Semantics	Failure & Recovery	Safety & Authority Model
Offline/advisory prediction	Cloud servers / accelerators	Advisory prediction (label, score, token)	Open-loop (static dataset, exogenous data)	Batchable in the selected serving case	Discard or recompute before downstream action	Advisory; downstream software or human decides
Agentic ML	Digital software environments	Tool execution (API payload, code snippet)	Closed-loop in software (actions alter state)	Elastic virtual time in controlled environments	Software exception, container restart, DB rollback	Sandboxed permission; API scopes and virtual gates
Physical AI	Embodied physical plant	Physical force and motion (current, torque)	Closed-loop in physics (actions alter world)	Inexorable continuous time; delay is distance	Irreversible work; energy dissipation; damage	Proposal vs. permission; deterministic runtime veto

Physical AI did not arrive solely through the scaling of language models. It represents the convergence of two long-standing engineering traditions: robotics and control, which learned to act reliably on a physical world that pushes back, and machine learning, which learned to extract representations from data at scale. Over eighty years, the learned model has moved step by step from the periphery of the machine toward its actuators, and each step shifted what the surrounding systems architecture had to guarantee. That history explains what physical AI inherits from classical control and digital learning, and why its central systems challenge is new.

1.2 From Prediction to Action

The acting line begins with feedback. Wiener’s cybernetics described animals and machines alike as systems that measure the result of their own action and use it to correct the next one (Wiener 1948), and control theory soon learned to estimate a moving system’s state from noisy measurements (Kalman 1960). The first industrial robots needed little of this at the level of the task. In 1961 a Unimate arm began stacking hot die-cast parts at General Motors by replaying step-by-step commands stored on a magnetic drum (Computer History Museum 1961). Nothing in that program could notice a person in the arm’s path, so protecting people meant keeping them out of its reach. The machine acted, but it did not perceive, and nothing in it was learned. The two lines this section follows, robotics and control learning to act and machine learning mastering prediction, run as the upper and lower lanes of figure 1 until they merge.

Wiener, Norbert. 1948. Cybernetics: Or Control and Communication in the Animal and the Machine. Wiley.

Kalman, Rudolph Emil. 1960. “A New Approach to Linear Filtering and Prediction Problems.” Journal of Basic Engineering 82 (1): 35–45. https://doi.org/10.1115/1.3662552.

Computer History Museum. 1961. 1961: UNIMATE, the First Mass-Produced Industrial Robot, Begins Work at General Motors. Timeline of Computer History.

Figure 1: From prediction to action: Robotics and control (top) learned to act on the physical world, and machine learning (bottom) learned to predict from data; the two lines merge in learned action. The band beneath the time axis shows where the learned output lands in typical deployed systems of each era, from no learned component, to learned perception feeding hand-written planners, to learned motor commands, to pretrained models that propose motion. The time axis widens after 2005.

Shakey, developed at SRI from 1966 through 1972, joined perception to planning (Nilsson 1984). It took camera images and range-finder readings, kept a model of its surroundings, and used a planner called STRIPS to compose sequences of actions that would achieve a stated goal. This organization, later summarized as sense-plan-act (S·P·A), treats acting as the last step of a computation. Shakey moved very slowly, and its designers treated its motion as running asynchronously to the programs directing it (Nilsson 1984); the world was assumed to wait while the computer thought. Shakey also carried a detail that anticipates this book’s architecture. When one of its catwhiskers touched an obstacle, the robot stopped by a reflex action beneath the planner.

Nilsson, Nils J. 1984. Shakey the Robot. Technical Note No. 323. Artificial Intelligence Center, SRI International.

↪ The sense-plan-act paradigm, and the failure taxonomy it organizes, is developed in The Sense–Plan–Act Paradigm.

Brooks, Rodney. 1986. “A Robust Layered Control System for a Mobile Robot.” IEEE Journal on Robotics and Automation 2 (1): 14–23.

¹ Moravec’s Paradox: Moravec (1988) observed that computers could be made to perform at an adult level on intelligence tests or at checkers far more easily than they could be given a one-year-old’s perception and mobility, and he traced the gap to evolutionary age, since sensorimotor skill was refined over a far longer history than abstract reasoning. For a physical machine the hard part is therefore the part that touches the causal boundary, perceiving a changing world and acting on it before a deadline, which is where learning and this book both concentrate.

Moravec, Hans. 1988. Mind Children: The Future of Robot and Human Intelligence. Harvard University Press.

Brooks inverted that organization. His subsumption architecture built control from layers of simple, asynchronous behaviors, each coupling sensing to action. Higher layers could suppress the outputs of lower ones, but the lower layers continued to run as higher ones were added (Brooks 1986). A fast layer that keeps working while a slow one is busy or wrong is an ancestor of the permission path this book develops. Neither organization, however, made perception or dexterous motion easy to write by hand, and the planners and behavior layers alike stalled on exactly the skills a toddler has. That difficulty has a name. Moravec’s paradox¹ observes that sensorimotor skills people perform with little conscious effort can be difficult to reproduce computationally, which is why learning became attractive for physical tasks.

Learning had already entered the loop. In 1989 ALVINN, a small neural network, mapped camera and laser range images to the direction a vehicle should steer, and followed real roads under some field conditions (Pomerleau 1989). Through the 1990s robotics also made uncertainty explicit. Probabilistic robotics represents what a machine believes about itself and its surroundings as probability distributions, updated as actions and measurements arrive (Thrun et al. 2005). The DARPA Grand Challenge tested these ideas at scale. In the first race, in 2004, no vehicle covered more than a small fraction of the desert course. In 2005 five vehicles finished, and the winner, Stanford’s Stanley, relied predominantly on machine learning and probabilistic reasoning (Thrun et al. 2006). In systems of this kind, learning mostly estimated what the world was like, and hand-written planners and controllers decided what to do. A decade later, the DARPA Robotics Challenge Finals asked robots to work at a simulated disaster site, driving a vehicle, opening a door, turning a valve, and climbing stairs. The organizers deliberately degraded the communications link to the remote human operators, so that each robot had to carry out parts of every task under supervised autonomy rather than direct teleoperation (Krotkov et al. 2017). Even at that stage, a person remained close to the loop.

Pomerleau, Dean A. 1989. “ALVINN: An Autonomous Land Vehicle in a Neural Network.” Advances in Neural Information Processing Systems (NeurIPS) 1: 305–13.

Thrun, Sebastian, Wolfram Burgard, and Dieter Fox. 2005. Probabilistic Robotics. MIT Press.

Thrun, Sebastian, Mike Montemerlo, Hendrik Dahlkamp, David Stavens, Andrei Aron, James Diebel, Philip Fong, et al. 2006. “Stanley: The Robot That Won the DARPA Grand Challenge.” Journal of Field Robotics 23 (9): 661–92. https://doi.org/10.1002/rob.20147.

Krotkov, Eric, Douglas Hackett, Larry Jackel, Michael Perschbacher, James Pippine, Jesse Strauss, Gill Pratt, and Christopher Orlowski. 2017. “The DARPA Robotics Challenge Finals: Results and Perspectives.” Journal of Field Robotics 34 (2): 229–40. https://doi.org/10.1002/rob.21683.

↪ ALVINN’s missing recovery examples, and the compounding error they expose, are developed in Recovery and Compounding Error.

Krizhevsky, Alex, Ilya Sutskever, and Geoffrey E. Hinton. 2012. “ImageNet Classification with Deep Convolutional Neural Networks.” Advances in Neural Information Processing Systems (NeurIPS) 25.

Deep learning then changed how much a machine could perceive. In 2012 a deep convolutional network trained on GPUs classified ImageNet photographs with error well below the previous state of the art (Krizhevsky et al. 2012), and learned detectors spread into the perception systems of robots and automated vehicles. The architecture around them usually stayed modular, with a learned percept feeding a planner written as rules or as an optimization. The learned outputs were still predictions, yet no person stood between them and the brakes. The Tempe vehicle examined in section 1.8 was a machine of this kind.

Learning reached the actuator itself in the following years. Levine and colleagues trained convolutional networks that mapped raw camera images directly to the torques at a robot’s motors, for tasks such as screwing a cap onto a bottle (Levine et al. 2016). Grasping systems learned from hundreds of thousands of their own attempts, gathered by groups of robot arms working in parallel (Levine et al. 2018; Kalashnikov et al. 2018), and deep reinforcement learning (training a network by trial, error, and a numerical reward) in simulation produced controllers for a legged robot and a dexterous hand that transferred to hardware (Hwangbo et al. 2019; OpenAI et al. 2019). With that step the data changed character. A robot that learns from its own attempts chooses the states it will see, so its training data depends on its own behavior.

Levine, Sergey, Chelsea Finn, Trevor Darrell, and Pieter Abbeel. 2016. “End-to-End Training of Deep Visuomotor Policies.” Journal of Machine Learning Research 17 (39): 1–40.

Levine, Sergey, Peter Pastor, Alex Krizhevsky, Julian Ibarz, and Deirdre Quillen. 2018. “Learning Hand-Eye Coordination for Robotic Grasping with Deep Learning and Large-Scale Data Collection.” The International Journal of Robotics Research 37 (4-5): 421–36.

Kalashnikov, Dmitry, Alex Irpan, Peter Pastor, Julian Ibarz, Alexander Herzog, Eric Jang, Deirdre Quillen, et al. 2018. “Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulation.” Proceedings of the 2nd Conference on Robot Learning, Proceedings of machine learning research, vol. 87: 651–73.

Hwangbo, Jemin, Joonho Lee, Alexey Dosovitskiy, Dario Bellicoso, Vassilios Tsounis, Vladlen Koltun, and Marco Hutter. 2019. “Learning Agile and Dynamic Motor Skills for Legged Robots.” Science Robotics 4 (26): eaau5872. https://doi.org/10.1126/scirobotics.aau5872.

OpenAI, Ilge Akkaya, Marcin Andrychowicz, Maciek Chociej, Mateusz Litwin, Bob McGrew, Arthur Petron, et al. 2019. “Solving Rubik’s Cube with a Robot Hand.” arXiv Preprint arXiv:1910.07113.

Zhao, Tony Z., Vikash Kumar, Sergey Levine, and Chelsea Finn. 2023. “Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware.” Robotics: Science and Systems (RSS). https://doi.org/10.15607/RSS.2023.XIX.016.

Imitation learning, which trains a policy to reproduce recorded demonstrations, made human demonstration a practical route to new manipulation skills, and two ideas shaped the policies now in use. Action chunking predicts a short sequence of future actions from each observation, a design its authors motivated partly by the way a cloned policy’s errors compound over time (Zhao et al. 2023). Diffusion policies generate action sequences by iterative denoising, which lets one policy represent the several different ways a demonstrator might complete the same task (Chi et al. 2025). The learned output had become a short plan of future motion, executed by controllers beneath it.

The latest step changed where learned proposals come from. In digital software systems, foundation models matured as agentic reasoning engines capable of decomposing abstract goals, generating plans, and invoking external tools. Bringing that agentic reasoning into physical embodiment required grounding multimodal representations in spatial geometry, continuous dynamics, and motor coordination. Large multitask datasets collected on many robots prepared the ground (Brohan, Brown, Carbajal, Chebotar, Dabis, et al. 2023; Open X-Embodiment Collaboration et al. 2024), and between 2023 and 2025 several groups showed that a model pretrained on web-scale images and text can be fine-tuned to emit robot actions directly. These vision-language-action (VLA) models share one mechanism. A pretrained vision-language backbone encodes the camera images and an instruction, and an action head converts the backbone’s output into motor commands. The head either decodes actions as discrete tokens, the way a language model decodes words (Brohan, Brown, Carbajal, Chebotar, Chen, et al. 2023; Kim et al. 2024), or produces a continuous action chunk, a short sequence of future setpoints from one inference (Black et al. 2024). A backbone with billions of parameters cannot run at a controller’s rate, so recent designs split the work across rates: a slow model interprets the scene and a fast action model or on-robot decoder emits chunks that controllers execute (Bjorck et al. 2025; Gemini Robotics Team 2025). These systems are dated instances of one step, in which a pretrained general-purpose backbone, rather than a policy trained for one task on one robot, proposes the motion; the sizes and rates their authors report are collected later in State of practice in learned proposers, as of 2026. The running machine adopts the same hierarchy with two proposers: an intent model, the slow vision-language model that reads the scene and chooses the goal, and a chunk policy, the fast learned policy that emits action chunks toward that goal. Their rates are fixed on its machine card (section 1.4.2). Because a general-purpose learned model can now propose motion for a general-purpose machine, deciding which component may permit that motion becomes a design question for every machine that adopts one.

Brohan, Anthony, Noah Brown, Justice Carbajal, Yevgen Chebotar, Joseph Dabis, Chelsea Finn, Keerthana Gopalakrishnan, et al. 2023. “RT-1: Robotics Transformer for Real-World Control at Scale.” Robotics: Science and Systems (RSS). https://doi.org/10.15607/RSS.2023.XIX.025.

Open X-Embodiment Collaboration, Abby O’Neill, Abdul Rehman, Abhinav Gupta, Abhiram Maddukuri, Abhishek Gupta, et al. 2024. “Open X-Embodiment: Robotic Learning Datasets and RT-X Models.” IEEE International Conference on Robotics and Automation (ICRA), 6892–903. https://doi.org/10.1109/ICRA57147.2024.10611477.

Brohan, Anthony, Noah Brown, Justice Carbajal, Yevgen Chebotar, Xi Chen, Krzysztof Choromanski, Tianli Ding, et al. 2023. “RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control.” arXiv Preprint arXiv:2307.15818.

Kim, Moo Jin, Karl Pertsch, Siddharth Karamcheti, Ted Xiao, Ashwin Balakrishna, Suraj Nair, Rafael Rafailov, et al. 2024. “OpenVLA: An Open-Source Vision-Language-Action Model.” arXiv Preprint arXiv:2406.09246.

Black, Kevin, Noah Brown, Danny Driess, Adnan Esmail, Michael Equi, Chelsea Finn, Niccolo Fusai, et al. 2024. “\(\pi_0\): A Vision-Language-Action Flow Model for General Robot Control.” arXiv Preprint arXiv:2410.24164.

Bjorck, Johan, Fernando Castañeda, Nikita Cherniadev, Xingye Da, Runyu Ding, Linxi Fan, Yu Fang, et al. 2025. “GR00T N1: An Open Foundation Model for Generalist Humanoid Robots.” arXiv Preprint arXiv:2503.14734.

Gemini Robotics Team. 2025. “Gemini Robotics: Bringing AI into the Physical World.” arXiv Preprint arXiv:2503.20020.

↪ The memory-bound cost of splitting the work between a slow intent model and a fast chunk policy on the running machine is developed in Two proposers and the evidence horizon.

Figure 2: Robot models train on a sliver of frontier compute: Estimated training compute, on a logarithmic axis, of the models the Epoch AI database tags as language models (gray) and as robotics models (green), from 2018. Robotics models that generate video of the world rather than actions are drawn as open squares and labeled by family. The open robot action policies sit orders of magnitude below the largest language models of the same year; on this record, the scaling that drove language models has not been applied to learned action. Compute values are Epoch’s estimates (Epoch AI 2026).

The scale of that transfer from web-scale pretraining is easy to overstate: in the Epoch AI database, the open robot action policies trained on hundreds to thousands of times less compute than the largest language models of their year (figure 2) (Epoch AI 2026). The gap is about 350 times less for a 2024 vision-language-action model and about 7,000 times less for a 2025 open robot foundation model, and only 25 robotics models in the database carry a compute estimate at all. Physical AI is therefore not a smaller copy of the language-model scaling story. Its learned proposers are bounded by what can be collected from machines acting in the world and by what can run on the machine itself, two constraints that Parts I and II of this book treat as first-class.

Epoch AI. 2026. Data on AI Models.

NVIDIA. 2026. What Is Physical AI? NVIDIA Glossary.

Brooks, Rodney A. 1991. “Intelligence Without Representation.” Artificial Intelligence 47 (1–3): 139–59.

Lee, Edward A., and Sanjit A. Seshia. 2017. Introduction to Embedded Systems: A Cyber-Physical Systems Approach. 2nd ed. MIT Press.

The name physical AI is recent, and industry usage since 2024 applies it broadly to almost any AI that perceives and acts in the physical world (NVIDIA 2026). The history above runs through two research traditions that cover much of the same ground. Embodied AI descends from Brooks’s argument that intelligent behavior arises from a body situated in and interacting with its environment rather than from abstract representation alone (Brooks 1991). Cyber-physical systems name the integration of computation and networking with physical processes (Lee 2008), and they carry the timing and verification discipline of embedded engineering (Lee and Seshia 2017). Embodied AI supplies much of the learning this book uses, and cyber-physical systems supply much of its timing and verification practice. The book’s subject is where the two meet at an actuator, and the definition given below is deliberately narrower than industry usage, narrow enough that the scope test in section 1.6 can apply it to a given machine.

Read as one story, this history moves the learned model step by step toward the actuator, as the band beneath the time axis of figure 1 records. At first nothing was learned. Then learned percepts fed hand-written planners, learned policies emitted motor commands, and finally pretrained general-purpose models began to propose motion for many kinds of machine. At each step, a guarantee that separation or human inspection used to supply had to move inside the machine. The familiar concerns of machine learning systems engineering come along, and each acquires physical stakes: data stops being a fixed corpus because the machine’s own actions choose what it will observe next; compute stops being a matter of throughput and cost alone because a late answer is paid for in stopping distance; evaluation stops being a score on held-out examples because the competence that matters appears only in the closed loop; and deployment stops being a routine version rollout because a release grants a machine authority over motion near people. When a learned output becomes an irreversible action, none of these concerns can be settled inside the model. They belong to the whole machine, and they meet at the boundary where its commands become physical work.

1.3 The Question This Book Answers

That convergence is why the problem now faces the whole field rather than a few robotics specialists. The central question of this book is why a better prediction can still produce a worse physical action, and what the rest of the machine must supply so that a learned proposal becomes useful work within justified limits. The book’s answer is an argument in four linked steps: because the body’s physical limits bound what any proposal may ask for, and because the learned proposer’s competence holds only over the states that its data and closed-loop evaluation covered, an independent permission path—rather than the model itself—must decide what may execute at each tick, while the evidence gathered about the whole machine bounds the operating conditions under which that permission is granted. The argument first requires a shared vocabulary that covers the causal boundary and the cost of delay at it, the scope of physical AI, a five-level model of the machine that separates proposal from permission, and the four laws that organize the rest of the book.

One machine carries the argument through every chapter. A warehouse mobile manipulator must collect an object from a conveyor, open a spring-latched cage door, and carry its load through an aisle shared with people. These tasks join abilities often studied separately. The machine must recognize an unfamiliar object, choose a grasp, coordinate its arm and base, and respond when contact or the scene differs from expectation. Learning makes varied behavior possible; engineering must turn that behavior into useful, repeatable physical work.

This book develops the ability to design, teach, integrate, evaluate, and maintain such a machine within justified operating limits. Its recurring warehouse machine is an illustrative engineering design, not a report of a qualified deployment. The same task connects physical budgets, learning experiments, runtime decisions, and evidence for service. When the evidence is incomplete, the engineer must identify what measurement, new experience, or design change would support the next capability.

Consider the base approaching the end of a storage rack while its model processes a camera image. A worker steps into the aisle before the next proposal returns. The computer can still discard that proposal, but the robot has continued moving throughout the calculation, so choosing an action requires reasoning about both the prediction and the physical state in which it will execute. Three things must therefore hold together: the model must recognize a useful action, the execution mechanism must judge it while the observation remains relevant, and the body must be able to deliver the admitted motion or a feasible alternative. A stronger prediction alone cannot recover clearance already consumed during computation, which is why the engineering problem leads from physical limits to the architecture that checks each proposed action.

Building such a machine requires contributions from four disciplines that normally work apart: machine learning supplies perception and action proposals, but computer systems determine how quickly those proposals compute and move across memory (Williams et al. 2009), while mechanics and control establish which motions the plant can physically sustain and embedded systems enforce the real-time scheduling and bus rules that connect a proposal to an actuator. All four responsibilities meet at a single runtime question: whether the proposed action may execute under the machine’s current conditions. This book develops the reasoning needed to make and justify that decision.

Williams, Samuel, Andrew Waterman, and David Patterson. 2009. “Roofline: An Insightful Visual Performance Model for Multicore Architectures.” Communications of the ACM 52 (4): 65–76.

Definition 1.1: Physical artificial intelligence

Physical artificial intelligence is the engineering of closed-loop machines in which learned models influence, without synchronous human approval, physical actions that change subsequent observations.

1. Significance: Model performance becomes a property of an operating machine. Its actions must fit the plant’s physical limits and the time available to respond.
2. Distinction: The scope test later in this chapter combines a learned model, consequential physical feedback, and delegated authority. Neither a model used only for advice nor an automated machine without a learned component meets all three criteria.
3. Common pitfall: Treating the body as a peripheral device for a model. A useful proposal still needs an independent permission decision based on the current state, physical limits, and available fallback.

The argument starts where the definition does, with the difference between a proposed action and its physical effect. A short stopping calculation then puts a number on what delay costs at the causal boundary, and a documented collision later in the chapter shows these responsibilities failing together before four laws distill them.

1.4 Defining the Causal Boundary

An advisory model that renders an inference on a dashboard or returns a JSON payload across a network has no direct electrical or mechanical coupling to physical actuators. An operator or downstream software filter can inspect, validate, or discard its output before any physical energy is expended. Digital errors are not benign (an operator who acts on flawed guidance can cause serious damage), but advisory software cannot directly alter physical state. Physical AI begins when an automated system acquires the authority to turn a computed proposal directly into physical motion. Because that authority couples software execution to mechanical work, safety arbitration cannot remain an external afterthought. It must live inside the machine architecture, at the causal boundary between digital intent and physical consequence.

1.4.1 From a proposed action to motion

In modern computational frameworks, executing action_chunk = policy(observation) produces a floating-point tensor. That tensor represents an uncommitted proposal; converting it into physical motion requires an entire chain of intermediate controllers, bus interfaces, and power electronics. The causal boundary is the exact interface through which an admitted digital command begins driving physical change in the plant.

A learned policy typically proposes a trajectory of future joint positions, target velocities, or motor torques. Those numerical arrays remain purely informational descriptions until a low-level controller validates, accepts, and begins executing them. The internal representation and predictive horizon of such action sequences are developed in Action chunking; here the systems distinction is between proposing an action and authorizing its irreversible consequences.

That authorization is governed by the permission path, an independent mechanism defined in section 1.7 that alone holds the authority to pass commands to motor drives. The permission path verifies whether a proposed trajectory satisfies physical kinematic constraints, obeys velocity and acceleration bounds, and can complete safely within current time allowances. Once admitted, trajectory setpoints feed into dedicated motor controllers. A position loop compares each setpoint with the measured joint position and computes a correction, and a current controller sets the motor current that produces the torque the correction needs.

↪ The bus timing and jitter bounds on command transport from the permission path to the drives are developed in Real-Time Command Transport and Bus Determinism.

Consider an electric motor driven by pulse-width modulation. A software write updates a compare register in a microcontroller timer, which sets how long each power switch stays on in every switching period. The resulting on-off pattern drives the inverter’s transistor switches, which turn the DC supply into current in the stator windings (the stationary coils). That current sets up a magnetic field that pulls on the magnets of the rotor (the spinning part) to produce torque, which accelerates the rotor shaft, transmission gears, and mechanical payload. The progression is asymmetric. Rewriting a planned setpoint in memory requires negligible energy and can be undone before the next clock cycle, but modifying a command that has already energized a winding requires physical work performed against mechanical inertia over finite time. A subsequent command can apply brakes or reverse drive current, but that corrective maneuver requires its own physical distance and must obey strict actuator limits.

Definition 1.2: Causal boundary

Causal boundary is the interface through which a digital command produces physical forces, currents, or flows.

1. Significance: A proposal can be rejected before it changes the plant. Once executed, it can alter the environment, dissipate energy, and change subsequent observations.
2. Distinction: Restoring a command in memory does not restore the machine’s prior physical state. Recovery requires another physical action, with its own limits and consequences.
3. Common pitfall: Assuming that canceling a software task stops the motion it initiated. The machine needs a feasible stopping or holding response even when its proposer stops computing.

Offline supervised learning evaluates models against static datasets where predictions never influence future test inputs. An embodied robot cannot rely on this decoupling. When the mobile base steers its chassis, its onboard cameras immediately capture a different visual field; when its manipulator nudges a carton, the object’s spatial coordinates and contact forces change; and when the vehicle cruises along an aisle, its closing distance to personnel shifts in real time. The closed-loop feedback introduced in section 1.2 applies directly to the learned model, because an acting machine reshapes the very environment that produces its future training and inference inputs.

In any embodied system, the environment evolves through closed-loop dynamics: the subsequent physical state \(\PhysicalWorldState_{t+1}\) and incoming sensory observation \(\MultimodalObservation_{t+1}\) depend directly on the action \(\CognitivePolicyAction_{t}\) issued by the policy inference: \[\PhysicalWorldState_{t+1} = \DiscretePlantDynamics(\PhysicalWorldState_{t}, \CognitivePolicyAction_{t}), \quad \MultimodalObservation_{t+1} \sim \ProbabilityFunction(\MultimodalObservation \mid \PhysicalWorldState_{t+1})\] Here \(\DiscretePlantDynamics\) represents the physical plant and environmental dynamics (formulated without process noise for this introductory overview), and \(\ProbabilityFunction\) denotes the observation model transforming underlying physical states into discrete sensor measurements. Because each state transition feeds the next observation, a flawed action proposal can steer the machine into configurations its training data never covered. In those out-of-distribution states the model’s outputs degrade further, and the machine departs faster from its nominal trajectory. Advisory output stops at an informational interface. In physical AI the loop closes through the permission path, the motor actuators, and the plant, so the model’s own actions irreversibly reshape the observations \(\MultimodalObservation_{t}\) it receives next (figure 3).² Imitation learning formalizes this compounding divergence as covariate shift. Because the policy encounters states induced by its own previous errors rather than expert demonstrations, the expected cost of mistakes can grow quadratically with the task horizon \(\BehaviorCloningHorizon\) (the number of control steps the task lasts), \(\CovariateDriftBound\), rather than linearly (Ross et al. 2011). Mechanics of Endogenous Covariate Shift develops the mechanics of this endogenous covariate shift. Contractive feedback controllers or well-damped passive mechanics can absorb minor trajectory errors, but verifying behavior over long horizons requires closed-loop evaluation rather than static offline benchmark scoring.

² Recovery Data (ALVINN): The neural network that steered Carnegie Mellon’s Navlab autonomous van directly from camera imagery confronted this challenge in early trials: because human test drivers kept the van centered in the driving lane, training logs contained no examples of recovering from the road shoulder, causing the system to veer off whenever it drifted off center. To synthesize recovery demonstrations without hazardous trials, Pomerleau (1991) transformed recorded road images using synthetic lateral shifts and calculated corresponding steering corrections back toward the lane center. While synthetic shifts work for simple geometric offsets, a physical policy must still be trained over the states its own actions visit.

Pomerleau, Dean A. 1991. “Efficient Training of Artificial Neural Networks for Autonomous Navigation.” Neural Computation 3 (1): 88–97. https://doi.org/10.1162/neco.1991.3.1.88.

↪ Corrective aggregation and the intervention data that defend a policy against covariate shift are developed in Intervention and Recovery Data.

Figure 3: The closed causal loop: Advisory output (top) stops at an informational interface. In physical AI (bottom), permitted model proposals reach hardware registers, where electrical current produces force and motion. Those actions alter the environment and subsequent sensor observations.

1.4.2 The running machine

Tracing how a proposed action crosses the causal boundary on the warehouse mobile manipulator of section 1.3 requires one set of numbers that every later chapter can share, so that the stop The Physical Body derives, the timing The Nervous System budgets, and the evidence Part IV demands all describe the same robot. Table 2 fixes the fifteen parameters of the machine that later derivations use most, each with the evidence label the reader guide defines (measured, derived, datasheet, or illustrative, or chosen for a design decision). The stopping budget that follows uses five of them, the drive limit, the credible deceleration, the clear distance, the pre-brake delay, and the chunk lease inside that delay. This chapter needs only those five and, in one pitfall, the in-loop takeover time; the other rows use ideas (the jerk-limited stop, the intent model’s prefill and decode, human takeover studies) that the section named in each row’s Basis column develops, and they appear here so that every chapter reads the same values. Subsequent worked examples cite the values by parameter name. For complete hardware specifications, the machine data sheet in The machine data sheet catalogs every other input alongside its programmatic registry key.

Table 2: The running machine’s card: The fifteen parameters of the warehouse mobile manipulator that later calculations use most. A derived value built on illustrative inputs is illustrative evidence. Values print from the MLSysIM registry when the book is built.

Parameter	Value	Category	Basis
Loaded mass \(\Mass\)	368 kg	Derived	Base, arm, and a full tote rack; an upper bound (The Five Physical Budgets)
Drive limit \(\SpeedBound\)	1.5 m/s	Illustrative	Class profile of a warehouse base
Aisle speed \(\AisleTravelSpeed\)	1.3 m/s	Chosen	Set below the dry-floor speed ceiling (Sizing the composite stopping budget)
Credible deceleration \(\CredibleBrakingDeceleration\)	2 m/s²	Illustrative	Loaded, on a dry floor whose grip supplies a friction coefficient \(\CoulombFrictionCoefficient \ge\) 0.204 if all the weight bears on the driven wheels, or 0.250 at the share the drive motors allow; tip-over is unchecked until the center of gravity is measured (Kinematic stopping distance and pre-brake delay)
Stop jerk bound \(\JerkLimit\)	7.11 m/s³	Derived	Stipulated for the jerk-limited stop, whose deceleration bound is \(\ProfileAccelerationBound = \CredibleBrakingDeceleration\) (The C^2 stopping suffix and jerk-limited braking)
Rack-end clear distance \(\ClearDistance\)	1.10 m	Illustrative	Clearance the stop must fit inside
Pre-brake delay \(\PrebrakeDelay\)	133.6 ms	Derived	Observation age, chunk lease, tick, bus cycle, and brake onset (The latency waterfall); excludes the dispatch-to-admission interval, which the spare pays for (The board-link budget and the dispatch-to-admission interval)
Permission tick \(\PermissionLoopPeriod\)	1 ms	Derived	Period of the chosen permission-loop rate (Physical time constants and sampling phase lag)
Chunk lease \(\ChunkLease\)	60 ms	Chosen	Longest an admitted chunk may drive without renewal (Chunk Lease Sizing from the Stopping Budget)
Setpoint step \(\FiniteChange \TimeInstant\)	25 ms	Chosen	Spacing of the chunk policy’s setpoints (Action chunking)
Chunk replan period	50 ms	Derived	A whole number of setpoint steps
Intent-model latency	336 ms	Derived	Prefill plus decode on the application processor (The memory wall)
Intent-model rate	2 Hz	Chosen	Largest rate whose period covers the latency
In-loop takeover \(\TakeoverResponseTime\)	605 ms	Derived	Slow-bound human reaction plus console video and handshake (Human reaction time and travel before braking)
Out-of-loop takeover \(\DisengagedTakeoverResponseTime\)	4.47 s	Derived	Take-over time of the 90th-percentile study condition plus video and handshake

With these parameters fixed, every later design problem on this platform starts from the same physical foundation. It asks which margin a commanded action approaches and whether the machine can execute an assured protective response before that margin is exhausted. The first margin to price is stopping clearance, and four rows of the card, the drive limit, the credible deceleration, the clear distance, and the pre-brake delay, are enough to price it.

1.4.3 Delay consumes stopping clearance

When the loaded warehouse base cruises down an aisle at speed \(\SpeedAtDetection\) and a worker steps out from behind a rack, stopping requires clearance across two distinct phases: the distance the base travels before braking forces engage, and the distance it travels while mechanical braking brings its momentum to zero. Assuming steady initial velocity \(\SpeedAtDetection\), uniform deceleration \(\CredibleBrakingDeceleration\) once brakes engage, and a stationary obstacle, the elementary stopping distance satisfies the kinematic budget: \[\StoppingDistance = \underbrace{\SpeedAtDetection \PrebrakeDelay}_{\text{travel during response delay}} + \underbrace{\frac{\SpeedAtDetection^2}{2 \CredibleBrakingDeceleration}}_{\text{mechanical dissipation}} \le \ClearDistance \tag{1}\] Here \(\SpeedAtDetection\) denotes initial forward speed, \(\PrebrakeDelay\) represents total pre-brake response latency, \(\CredibleBrakingDeceleration\) is the sustained braking deceleration, and \(\ClearDistance\) is the usable line-of-sight clearance. The pre-brake delay \(\PrebrakeDelay\) spans the time from when an obstacle first appears in a sensor exposure to the physical onset of retarding force. Specifically, it sums the age of the sensor observation upon handoff, the unexpired lease duration of any currently executing motion chunk (the lease is the longest an admitted chunk may drive before the permission path must act again), one discrete permission-path evaluation tick (the period at which the permission path re-checks the motion it has admitted), one bus communication cycle (the time to carry a command to the motor drives), and mechanical brake onset latency.

The lease belongs in the worst case even though the permission path re-checks every tick, because of what that check can see. Each tick tests the admitted chunk against limits the permission path already holds (the chunk’s freshness, its lease, and the kinematic bounds, measured against the wheel and joint encoders); the permission path has no camera pipeline of its own. On the detection path this budget models, evidence that a worker has stepped out therefore reaches the brakes only through the learned perception path and a replacement proposal. If the proposer stalls or fails to answer the worker, the most recently admitted chunk keeps driving until its lease expires, and only then does the permission path command the stop. The term deliberately excludes one interval, the dispatch-to-admission interval (the time while the learned policy computes a replacement proposal and the permission path admits it). Its length depends on where the proposer and the permission path run and how they communicate, decisions this chapter has not yet made, so The board-link budget and the dispatch-to-admission interval allocates it and the finished budget pays for it from the spare this elementary budget leaves.

The budget has two terms that grow differently with speed: travel during the delay grows linearly (\(\SpeedAtDetection \PrebrakeDelay\)), and travel while shedding kinetic energy grows with the square of speed (\(\SpeedAtDetection^2 / (2 \CredibleBrakingDeceleration)\)). Equating them gives the crossover speed \(\StoppingCrossoverSpeed = 2 \CredibleBrakingDeceleration \PrebrakeDelay\). Below \(\StoppingCrossoverSpeed\), delay travel is the larger term; above it, braking travel dominates. Shortening sensing, computation, and signaling reduces the delay term, while lowering speed reduces both terms: halving speed halves the delay travel and quarters the braking travel. Which change saves more clearance depends on the reductions available, not on term dominance alone. Solving equation 1 for \(\SpeedAtDetection\) gives the fastest speed whose stop fits inside a verified clear distance, a ceiling Dynamic velocity ceilings and clearance budgeting derives with the allowances this elementary form omits.

Every input to the budget comes from the running machine. Its speed, deceleration, clear distance, and pre-brake delay are rows of the machine card (table 2), and the five terms of that delay come from the same MLSysIM registry, so the notebook’s givens are the ones every later chapter refines.

The learned backbone is 22 ms of the 133.6 ms delay; the lease costs more.

Napkin Math 1.1: Does the base stop inside its clearance?

Problem: The loaded base cruises toward a rack end at its drive limit, and a worker may step out at the end of the rack. The permission path must stop the base inside the rack-end clear distance on the machine card. Does the elementary stopping budget of equation 1 fit, which term dominates, and how much of the delay belongs to the learned model?

Variables:

• Speed \(\SpeedAtDetection =\) 1.5 m/s, the drive limit (illustrative).
• Credible deceleration \(\CredibleBrakingDeceleration =\) 2 m/s², loaded on a dry floor (illustrative).
• Clear distance \(\ClearDistance =\) 1.10 m (illustrative).
• Pre-brake delay \(\PrebrakeDelay =\) 133.6 ms (derived). It is the sum of five registry terms: the observation age of the camera frame that first shows the worker, 51.6 ms, which includes the 22 ms perception backbone (the rest is half the exposure, sensor readout, transfer, image processing, and the handoff to the proposer, itemized in The latency waterfall); the chunk lease of 60 ms, the longest an admitted command may drive before the permission path must act; one permission tick of 1 ms; one bus cycle of 1 ms; and a brake onset of 20 ms. It excludes the dispatch-to-admission interval, for the reason given with equation 1.

Math:

\[\tau_{\text{delay}} = 51.6 + 60 + 1 + 1 + 20 = 133.6\ \text{ms}\] \[v\,\tau_{\text{delay}} = 1.5\ \text{m/s} \times 0.1336\ \text{s} = 200.4\ \text{mm}\] \[\frac{v^2}{2 a_{\text{brake}}} = \frac{(1.5\ \text{m/s})^2}{2 \times 2\ \text{m/s}^2} = 562.5\ \text{mm}\] \[d_{\text{stop}} = 200.4 + 562.5 = 762.9\ \text{mm} \le D_{\text{clear}} = 1100\ \text{mm}, \quad D_{\text{clear}} - d_{\text{stop}} = 337.1\ \text{mm}\] \[v^{*} = 2 a_{\text{brake}} \tau_{\text{delay}} = 2 \times 2\ \text{m/s}^2 \times 0.1336\ \text{s} \approx 0.53\ \text{m/s}\]

Result: The elementary budget fits, with 337.1 mm to spare. The drive limit sits above the crossover speed, so braking dominates: it needs 2.8× the travel of the delay, and the delay adds 35.6 percent to the braking distance. At half speed, 0.75 m/s, delay travel halves to 100.2 mm while braking quarters to 140.6 mm, for a stop of 240.8 mm.

Systems insight: The learned model’s backbone accounts for 22 ms of the delay, or 33 mm of travel; the chunk lease alone costs 90 mm. Each added millisecond of delay anywhere in the chain costs 1.5 mm at this speed, so a slower but more accurate model is a distance cost the budget must absorb. The spare is not yet a safety margin. Sizing the composite stopping budget adds the localization bound, a protective margin, tracking error, and the longer stop of a jerk-limited brake (one that limits jerk, the rate of change of deceleration, by ramping its braking up instead of stepping it); together they exceed the clear distance at the drive limit, which is why the card sets the aisle speed at 1.3 m/s.

Figure 4: Delay travel and braking travel against speed: The elementary stopping budget of the loaded base, split into travel during the 133.6 ms pre-brake delay, which grows linearly with speed, and braking travel at 2 m/s², which grows with its square. The two terms are equal at the crossover speed \(\StoppingCrossoverSpeed\); above it braking dominates. At the 1.5 m/s drive limit the elementary stop fits inside the 1.10 m clear distance, before the allowances that later chapters add.

The running machine’s drive limit lies well above the crossover speed the notebook computed (figure 4), so braking, not delay, sets most of its stop, and the sum still clears the dashed clear-distance line before any later allowance is added. These values are representative parameters and a derived latency profile, not measured stopping distances for a real chassis, but they already show the two levers a designer holds. Shortening response latency contracts the linear term, whereas lowering cruising speed contracts both, the braking term quadratically. Selecting a model purely on offline accuracy overlooks the first lever, because added inference latency is added travel.

The budget just computed models one particular detection path, and naming it shows which learned computation it charges. The worker is seen by the learned perception stack, so the perception backbone sits inside the observation age. The chunk policy’s own compute time does not appear, because the lease, not the policy, bounds how long an admitted chunk may keep driving once the proposer falls silent. A machine can also carry a second, independent protective path with its own sensing, timing, power, and fallback logic, such as a certified safety scanner wired to the drives. A stop that path triggers bypasses perception dispatch and the lease entirely. The backbone and lease terms leave its delay, and its own sensing and switching latency take their place, while the braking travel and the allowances stay. Under that decoupled architecture, the protective path’s latency, rather than the execution time of the neural network, governs the protective stop. Industrial automated guided vehicles (AGVs) implement this separation today, with safety standards mandating certified safety scanners that halt the machine before human contact, independent of fleet management or guidance computers (International Organization for Standardization 2023). This book layers learned sensorimotor proposers atop deterministic safety kernels and analyzes the architectural contracts required between them.

International Organization for Standardization. 2023. ISO 3691-4:2023: Industrial Trucks, Safety Requirements and Verification, Part 4: Driverless Industrial Trucks and Their Systems. International Organization for Standardization.

↪ The scanner path’s own timing budget, and how the software stopping budget maps onto certified safety functions, are developed in Certified safety functions beneath the permission path.

International Organization for Standardization. 2024. ISO 13855:2024: Safety of Machinery, Positioning of Safeguards with Respect to the Approach of the Human Body. International Organization for Standardization.

Deploying this budget in real-world facilities requires several additional physical allowances. Beyond the allowances the notebook’s systems insight names (localization, a protective margin, tracking error, and the longer jerk-limited stop), real-world controllers must also account for unmodeled initial acceleration and degraded tire-floor traction. Dynamic obstacles complicate the clearance budget even further: if a pedestrian walks toward the mobile robot at speed \(\HumanWalkingSpeedBound\), the relative closing distance contracts during both the pre-brake delay and the mechanical deceleration phase. For this reason, functional safety standards mandate adding a pedestrian approach-speed allowance to the vehicle’s own braking distance (International Organization for Standardization 2024). Later chapters formalize these dynamic constraints and allocate delay budgets across compute hardware. Even before they do, a learned policy’s behavior is acceptable only inside the real-time envelope that plant momentum sets.

1.4.4 Irreversibility and recovery

The stopping budget prices one consequence of irreversibility. The other is what recovery becomes. A database transaction that aborts restores the state it started from, but a gripper that has permanently deformed a workholding fixture, or a winding whose insulation has degraded from overheating, has no earlier state to return to. Recovery on a physical machine is therefore a new trajectory, not a reversal.

Corrective commands can brake the motors, release a grasp, or plan a diversion, but each is itself motion, limited by the torque the actuators can deliver, by friction, and by the speeds and accelerations the drives can reach. Which limit governs depends on what the machine is doing. For the mobile base it is stopping clearance under momentum. When the arm closes its jaws on a fragile component or braces against an assembly fixture, it is contact force, the flex of the gears and belts between motor and gripper (transmission compliance), and heat in the windings. Because recovery is constrained motion, the permission path must check before it admits an action that a feasible recovery from the resulting state will still exist; checking afterward is too late.

↪ The contact loads a transmission delivers to a gripped part or a latch, the arm’s limiting quantity here, are developed in Transmission compliance, contact impedance, and tooth fracture.

1.5 Three Machine Classes

As the warehouse mobile manipulator travels down an aisle, its primary engineering challenge is kinetic: preserving enough clearance to stop safely before an obstacle. When the same robot presses against a door latch, the governing constraint shifts to contact mechanics, regulating interaction forces to avoid damaging the mechanism or the workpiece. When holding a suspended payload stationary, the machine performs zero mechanical work, yet its actuators must sustain continuous stall torque without exhausting thermal headroom or browning out the shared electrical supply. Because each phase of the task spends a different physical margin, the book names three machine classes by the margin each one spends (figure 5).

Mass-dominated mobility (Class 1) spends stopping clearance. A mobile base, delivery vehicle, or aerial drone carries significant momentum and cannot halt instantaneously. Forward velocity, control and sensing latency, and available braking deceleration or reverse thrust jointly dictate the spatial clearance required to arrest or redirect its motion.

Contact-dominated manipulation (Class 2) spends force and deformation margin. When an articulated arm presses against a stiff fixture, minute position displacements can generate destructive contact forces over fractions of a millimeter. Physical contact is both the means of task success (seating a part or opening a latch requires deliberate force) and the immediate hazard. The engineering task is to supply the wrenches (the combined forces and torques) needed to complete the assembly while guaranteeing that contact forces never exceed the mechanical yield limits of the payload or the manipulator.

Energy- and thermal-dominated operation (Class 3) spends heat and electrical supply margins. Supporting a heavy load or resisting gravity demands sustained motor current, driving continuous Joule heating in actuator windings even when joints remain stationary. Concurrent peak current draws from drive inverters, compute accelerators, and sensor payloads can induce voltage sag across a shared DC bus. The system must sustain these thermal and electrical loads across the entire operational profile while reserving sufficient energy and thermal headroom to execute a safe protective abort.

Figure 5: Three machine classes: Each class is named for the constraint that dominates it and the margin it spends: (a) mass-dominated mobility (Class 1) spends stopping clearance, set by momentum and braking; (b) contact-dominated manipulation (Class 2) spends force and deformation margin, set by contact stiffness and the strength of the object and mechanism; and (c) energy- and thermal-dominated operation (Class 3) spends heat and supply margin, set by Joule heating, thermal capacitance, and power delivery.

These three classes identify dominant physical constraints rather than mutually exclusive hardware categories. A practical robot transitions across all three regimes throughout a single task. The running machine integrates a wheeled base, an articulated arm, navigation lidar, and wrist cameras. An end-to-end learned policy can coordinate simultaneous base trajectories and arm proposals (Fu et al. 2024), but the integrated machine must enforce the safety envelopes of every regime concurrently. A fixed-base arm still faces thermal and bus-voltage limits during prolonged holding, just as a wheeled base encounters contact constraints when docking against a workstation.

Fu, Zipeng, Tony Z. Zhao, and Chelsea Finn. 2024. “Mobile ALOHA: Learning Bimanual Mobile Manipulation with Low-Cost Whole-Body Teleoperation.” Proceedings of the 8th Conference on Robot Learning (CoRL), Proceedings of machine learning research, vol. 270.

Systems Perspective 1.1: The humanoid crucible

A humanoid robot exposes the tight coupling across all three regimes. Catching a shifted load perturbs dynamic balance, requiring an immediate recovery step that demands peak joint torque. That torque surge draws heavy current from the shared DC bus and injects rapid thermal energy into the joint windings. If the power distribution network sags or thermal protection limits current, the recovery step fails and the machine falls. Shielding the onboard computing stack from reset does not by itself supply the transient mechanical energy needed to stabilize the body. The part-ending humanoid examples throughout this book test how each systems principle survives these tightly coupled physical demands.

↪ The multi-contact balance mechanics a humanoid’s recovery step depends on are developed in Centroidal Dynamics and Floating-Base Control.

Identifying whether a task spends stopping clearance, contact force, or thermal headroom tells the engineer which physical budget binds, regardless of how the controller is implemented. Deciding whether the machine also requires the proposal-and-permission architecture of physical AI—rather than a classical feedback loop or an offline tool—requires a separate test of how its decisions are generated and authorized.

1.6 The Physical AI Scope Test

A robotic arm in a fulfillment center can follow a fixed motion profile under a proportional-integral-derivative (PID) controller (a classical hand-tuned feedback law that corrects in proportion to the present error, its accumulated sum, and its rate of change), execute an offline trajectory pre-screened and loaded by a human technician, or stream joint torques directly from a learned policy running on real-time sensor frames. While all three regimes actuate identical electric motors and gear trains, they impose different architectural obligations on the computing system. The scope test establishes the operational boundaries that define physical AI throughout this book.

The scope of physical AI is defined by the simultaneous intersection of three structural criteria:

1. Learned Statistical Model: The system relies on a policy, perception encoder, or world model whose runtime behavior emerges from training data rather than an inspectable closed-form specification, as seen in deep neural networks or diffusion policies (Chi et al. 2025). Correctness cannot be formally proven across all state configurations from source code alone; engineers must establish operational validity empirically across a defined operating domain. By contrast, a linear plant model fitted through classical system identification,³ where parameter uncertainty yields provable stability margins, falls outside this category. A high-capacity neural policy falls within it.
2. Consequential Physical Feedback: The system’s actions alter the material state of the surrounding physical world, and those state changes directly govern what its sensors measure in subsequent time steps under classical mechanics, thermodynamics, or electromagnetism. The environment cannot be paused or rolled back, and energy dissipated into it cannot be recalled.
3. Delegated Authority: The system’s outputs reach execution without synchronous human approval for each control cycle. In a machine that also meets the second criterion, that execution is actuation, so the authority is physical. A learned component holds this authority whenever its output (a raw percept, a semantic state estimate, or a proposed motion trajectory) can propagate to live actuators without such approval. The learned network need not write directly to motor driver registers; the system possesses delegated authority whenever learned signals actively shape downstream actuation commands. Adding downstream safety checks or deterministic rate limiters does not vacate this authority, provided accepted proposals still drive hardware motion.

Chi, Cheng, Zhenjia Xu, Siyuan Feng, Eric Cousineau, Yilun Du, Benjamin Burchfiel, Russ Tedrake, and Shuran Song. 2025. “Diffusion Policy: Visuomotor Policy Learning via Action Diffusion.” The International Journal of Robotics Research 44 (10-11): 1684–704. https://doi.org/10.1177/02783649241273668.

³ System Identification: The control-engineering practice of fitting low-order dynamic models to measured input-output time series. When this fitting establishes bounded parameter uncertainty under stated modeling assumptions, control engineers can use those bounds to prove stability guarantees before deploying the controller. Modern high-dimensional neural networks lack tractable closed-form uncertainty bounds and therefore require empirical verification across explicitly specified operating domains.

Together, these three criteria delineate the domain of physical AI. The three-set Venn diagram in figure 6 formalizes this boundary, locating physical AI at the central intersection where a learned model, consequential physical feedback, and delegated authority converge. Each pairwise overlap holds systems that meet only two of the three criteria: advisory systems place a learned model inside a physical process, as when a technician inspects learned toolpaths before loading them onto a mill, but a person still approves each action; digital autonomy executes learned outputs without human approval, as in an automated trading engine, whereas its actions land in software rather than in physics; and classical control closes a physical feedback loop automatically, as in a PID speed governor, but relies on an analytical control law rather than a learned model. Excluding these pairwise overlaps restricts the design problem to systems where statistical uncertainty collides directly with physical irreversibility.

Figure 6: The physical AI scope test: Physical AI combines a learned component, consequential physical feedback, and delegated authority to let learned outputs reach actuation without human approval. An independent veto belongs inside this intersection because accepted proposals can still move the plant. The pairwise overlaps locate advisory systems, digital autonomy, and classical control.

A learned model alone does not make a machine a physical AI system. Consider an offline trajectory generator that optimizes smooth paths using a learned diffusion model. If a manufacturing engineer must inspect, validate, and load those toolpaths onto a CNC mill prior to cutting metal, the statistical model operates only as an advisory tool. The human retains ultimate physical authority. Similarly, a vision model that highlights defects on an assembly line display assists human inspection without closing the physical loop.

Conversely, delegated authority does not require end-to-end neural control. Most production autonomous systems adopt modular architectures where deep neural networks handle perception while deterministic trajectory planners compute actual actuator setpoints. These systems satisfy the delegated-authority test through perception: if an unvetted object detection or bounding-box estimate triggers an automated swerve or emergency braking maneuver, the learned representation directly dictates plant dynamics. The autonomous test vehicle in the Tempe collision examined in section 1.8 is such a machine: changing classifications in its modular perception fed the path predictions that decided whether it would brake.

Shared autonomy falls within physical AI whenever a learned sub-system modifies live motion before a human operator can intervene. In a robotic surgical console equipped with learned haptic virtual fixtures, for example, the assistive policy exerts real forces on the master tool manipulators to guide the surgeon around delicate anatomical structures. Even if the surgeon holds administrative responsibility for the procedure, high-bandwidth force cues alter physical tissue contact well within normal human neurological reaction latency (typically 150 ms to 250 ms). Because the machine enforces learned spatial guidance dynamically without requiring the surgeon to review and acknowledge each micro-correction, it exercises delegated authority over physical motion.

When classifying an ambiguous machine, engineers must evaluate each of the three criteria independently. Implementing an independent deterministic veto, such as an analog brake relay triggered by a certified safety lidar, does not strip a machine of delegated authority, because learned proposals still govern all nominal motion up to the intervention threshold. In contrast, requiring a human operator to screen and authorize every trajectory command before transmission reliably removes that authority. Once a design satisfies all three criteria, it qualifies as physical AI, and its computing architecture must respect the hard timing and causal boundaries established in section 1.4.

Checkpoint 1.1: Binding budgets and the scope test

• Delay vs. braking: Explain why response delay and active braking consume separate portions of stopping clearance, and identify which one dominates above the crossover speed.
• Stopping budget estimate: Using the machine card, estimate how far the elementary stop at the 1.5 m/s drive limit grows if the 60 ms chunk lease doubles, and determine whether it still fits inside the 1.10 m clear distance.
• Binding margins: Identify the relevant physical margin when the warehouse machine travels, presses a latch, and holds a load.
• Scope criteria: Apply all three scope criteria to a new machine, including one whose learned output passes through an independent veto.

Satisfying the scope test binds statistical uncertainty directly to hardware consequences. To design systems that operate safely under these demands, computer systems engineers must map how timing, state estimation, and control authority distribute along the execution path from raw sensor frames to physical actuator motion.

1.7 The Machine in Five Levels

When the running machine’s chunk policy stalls mid-aisle, something other than the policy must still be able to stop the base. When the policy emits a chunk that would carry the arm through a coworker’s hand, something other than the policy must refuse it. An embodied machine therefore needs an architecture that prevents learned models from writing directly to the actuators they could damage, while still letting those models guide physical behavior.

The Body establishes what motion, force, and mechanical power the hardware can physically sustain. The Brain generates learned proposals intended to operate within those capabilities. Mediating between them, the Nervous System coordinates communication and control, preserving both the temporal margin and the hardware authority required to intercept and refuse an inadmissible command. Adding the interface where commands become current and the environment the machine acts on gives a five-level model that runs from learned computation down to the physical world:

1. The Brain (Level 1): The learned models that interpret sensory observations and propose candidate goals, trajectories, or torques. Generating a proposed trajectory or sequence of setpoints confers no intrinsic authority to execute it.
2. The Nervous System (Level 2): The deterministic communication buses, real-time schedulers, and control loops that bridge proposals to physical execution. Its permission path evaluates, filters, or rejects proposals against fresh sensory evidence and invariant physical limits, and it executes a fallback shown feasible from the current state under the stated sensing, plant, and response-time assumptions when the proposer falters.
3. The Body (Level 3): The physical plant, including motor drives, transmissions, electromagnetic windings, structural linkages, and energy storage, that delivers mechanical action.
4. The Boundary (Level 4): The hard physical and electrical interface where digital registers translate into actuator voltages or currents, and where physical phenomena register across sensor transducers as raw measurements.
5. The World (Level 5): The external physical environment (surfaces, payloads, obstacles, ambient lighting, and human collaborators), which evolves continuously under the laws of physics regardless of whether the machine measures it.

The list orders the levels by abstraction, from learned computation down to physics, not along the path a command travels. A command leaves the Nervous System, crosses the Boundary at a register write, and only then drives current through the Body’s windings. The Boundary is listed after the Body because it gathers the transducers at both edges of the machine, the register writes that energize the windings and the sensor captures that turn light, force, and motion into numbers, and those are the places where the machine exchanges energy and information with the World. The actuation half of this level is the causal boundary defined earlier in the chapter. The proposal boundary, defined below, is not a level at all. It is the line between the Brain and the Nervous System that separates proposing from permitting.

A concrete physical AI implementation often maps this hierarchy across distinct computational tiers: learned proposers run on high-throughput accelerator hosts (such as an application processor paired with a GPU or neural processing unit), the permission path executes on an independent real-time microcontroller or dedicated real-time operating system (RTOS) core, and the current loops that regulate motor-phase current run directly on dedicated motor drives. This architectural isolation is rooted in strict timing guarantees and fault-containment boundaries rather than specific silicon families or nominal operating system labels.

↪ The loaded test of the two-processor split that realizes this isolation is developed in Empirical Verification of Silicon Isolation.

Governance does not constitute a sixth level in this real-time control hierarchy. Instead, it serves as the operational and lifecycle envelope enclosing all five. Governance determines which entity holds valid command authority, what verification evidence licenses the machine to operate, and when that license lapses and must be renewed. Governance transitions unfold across the human and operational timescales of safety interventions, policy rollouts, and maintenance releases, rather than the millisecond rhythms of real-time control loops.

↪ The authority state machine that hands each actuator channel to exactly one owner during a transfer is developed in Authority Transitions.

Definition 1.3: Proposal boundary

Proposal boundary is the line between a learned model, which may only propose an action, and the permission path, an independent mechanism with bounded latency that the model cannot delay or corrupt and that alone may permit the action to reach an actuator. The permission path validates every proposal against fresh sensory evidence, the stopping envelope (the state space from which the machine can safely decelerate to a halt prior to obstacle collision), and a fallback trajectory shown feasible from the current state. On the running machine, the algorithmic admission checks remain an unrated performance layer (one not certified to a functional-safety rating) above separate certified protective functions; Certified safety functions beneath the permission path explains that distinction. If any condition is violated, the permission path rejects the proposal within the established response bound.

1. Significance: Prevents learned model outputs from directly manipulating actuators. This architectural protection remains dependable only as long as the permission path’s independent sensing, dynamic models, and fallback maneuvers remain physically valid.
2. Distinction: Even a deep reinforcement learning policy trained to emit joint torques possesses zero raw actuator-write authority; its output remains a mere proposal until explicitly admitted by the permission path.
3. Common pitfall: Treating the permission path as a cosmetic post-processing filter or output clamp, rather than an autonomous supervisory mechanism capable of seizing control when the learned proposer hangs, violates deadlines, emits pathological values, or falls silent.

The proposal boundary determines who holds the authority to approve an action, whereas the causal boundary marks the physical interface where an admitted command alters the state of the plant. The Nervous System bridges this divide, verifying proposals against hard constraints before energizing the drives.

Readers steeped in classical control theory will recognize this design as a supervisory architecture. Specifically, the proposal boundary implements runtime assurance. In classic Simplex architectures, a complex, high-performance controller (here, the learned policy) is paired with a certified baseline controller, coordinated by decision logic that determines which subsystem drives the plant (Seto et al. 1998; Sha 2001). The lineage table in section 1.10 (table 3) maps this book’s terms to their established names once the four laws are stated.

↪ Runtime assurance for physical AI, which pairs the learned proposer with a resident protective controller and an output selector, is developed in The Per-Tick Gatekeeper.

Hardware fault isolation alone does not guarantee safety, because architectural independence also constrains the permission path’s sensor inputs. The learned detection path used in the stopping budget prices response delay, but its lease cannot protect against a perception error that produces timely, unsafe proposals. If an enforcement monitor validates a proposed trajectory using the very same learned perception pipeline that synthesized it, a single perception error will corrupt both components simultaneously. The sensing pipelines that grant execution authority must therefore maintain distinct failure modes from the learned models they supervise.

Across the five stacked levels in figure 7, characteristic operating rates span four orders of magnitude, from the continuous physical World at the bottom, through the event-timed Boundary, the 10–25 kHz current loops of the mechanical Body, and the 1 kHz permission loop of the Nervous System, up to the hierarchical Brain (operating from 40 Hz setpoints to 2 Hz intent models at the top). The prominent gap between Brain and Nervous System marks the foundational proposal boundary: components above this line may only formulate proposals, while only deterministic subsystems below it possess the authority to command physical actuators. The operating frequencies charted across the levels reflect the reference architecture analyzed throughout this book, rather than immutable physical constants for every robotic platform. Subsequent chapters derive these timing budgets directly from task dynamics and mechanical plant characteristics. What remains fundamental across all physical AI systems is the proposal boundary that severs learned computation from unchecked execution authority.

Figure 7: The five-level machine: Each band carries its pinned rate and the chapters that work on it. The wider gap between Brain and Nervous System marks the proposal boundary, above which a component may only propose. A chapter-opening stack with no level highlighted marks a governance chapter. The permission-loop rate shown is one implementation.

Deploying capable physical AI requires balancing expressive policy proposals with deterministic execution guarantees. This separation of concerns does not demand rejecting every imperfect proposal out of hand; real-time intervention mechanisms can project a trajectory onto safe constraints or smoothly blend corrections, preserving task progress while respecting physical invariants. Systems evaluation must therefore disentangle what the learned proposer achieves in isolation, how effectively the integrated machine executes under routine supervisory filtering, and whether the fallback mechanisms themselves introduce secondary hazards.

↪ The minimal-intervention optimization filters that project a proposal onto its constraints are developed in The Minimal Intervention Principle and Optimization Filters.

Distributing responsibilities across these five levels gives the rest of the book its structure. It also gives a way to diagnose real failures by asking which level failed, rather than reducing every physical malfunction to a deficiency in neural network weights.

1.8 When the Responsibilities Fail Together

A warehouse model that recognizes an obstacle too late cannot recover the clearance already consumed. In a road vehicle, the same timing problem can combine with tracking errors, delayed intervention, and missing redundancy. The 2018 Tempe collision provides a documented example of these interacting failures; it does not establish that one prescribed hardware architecture would have prevented them. The National Transportation Safety Board (NTSB) attributed the crash primarily to the vehicle operator’s distraction. The design choices traced below explain why the automated system had made that single, unmonitored person the last line of defense.

On a March night in 2018, a modified Volvo XC90 operating Uber Advanced Technologies Group’s developmental automated driving system (ADS) struck and fatally injured a pedestrian pushing a bicycle across a road in Tempe, Arizona (National Transportation Safety Board 2019).⁴ The vehicle carried a roof-mounted lidar, forward and side radars, surround cameras, and GPS antennas, all routed to ADS compute and data-storage racks in the cargo bay (figure 8). The ADS first detected the pedestrian 5.6 s before impact, but detection alone did not provide a correct path prediction (National Transportation Safety Board 2019). The failure ran through perception and path prediction, a response rule that withheld braking, and a human operator the design relied on as its last countermeasure.

⁴ Reading the Tempe Record: Times in this section count backward from impact at \(\Time\), as the event table of NTSB/HAR-19/03 does, and speeds are those the ADS itself recorded (National Transportation Safety Board 2019). The report also cautions that communication delay leaves unclear exactly when the planned slowdown began. Reading each time as an interval before impact, rather than as a clock time, is what lets it be converted into distance traveled at the recorded speed, the conversion this section applies.

Figure 8: Tempe test vehicle sensor locations: The NTSB diagram identifies the developmental ADS vehicle’s lidar, camera, radar, computing, and telecommunications locations. (Source: NTSB, public domain).

1.8.1 Tracking history loss under classification churn

Despite continuing to track the detected object, the ADS did not brake in response to it because its classification kept shifting among vehicle, bicycle, and other before \(\ImpactInstant-1.2\text{ s}\) (National Transportation Safety Board 2019). Whenever the classification changed, the ADS path predictor did not use the object’s earlier locations, and for an “other” object it could assume a static path.⁵ The loss of earlier locations matters because successive positions provide evidence of motion across the roadway; a changing semantic label must not silently discard that evidence. Because each relabeling shortened the track, the resulting path predictions did not place the pedestrian in the SUV’s lane until \(\ImpactInstant-1.2\text{ s}\) (National Transportation Safety Board 2019).

⁵ Tracking-History Loss: The NTSB record states only that a changed classification caused the predictor to omit the object’s earlier locations; it does not describe a Kalman-filter reset or a zero-velocity initialization, so neither should be read into it (National Transportation Safety Board 2019). The consequence follows from the omission alone. Each relabeling shortened the evidence window from which motion is estimated, so an object crossing the lane began again, after each change, as one with no recorded history of crossing.

National Transportation Safety Board. 2019. Collision Between Vehicle Controlled by Developmental Automated Driving System and Pedestrian, Tempe, Arizona, March 18, 2018. HAR-19/03. National Transportation Safety Board.

1.8.2 Action suppression and lost stopping distance

The ADS included an action-suppression interval intended to reduce responses to false alarms while the system checked the situation and allowed operator intervention (National Transportation Safety Board 2019). That software choice directly consumed time available for a physical response: at \(\ImpactInstant-1.2\text{ s}\), with the vehicle traveling at 43.2 mph, the ADS recognized an emergency and entered its one-second action-suppression period with braking withheld. At \(\ImpactInstant-0.2\text{ s}\), suppression ended with the vehicle at 40.5 mph. The ADS then initiated a plan for gradual slowdown and sounded an alert; it did not request emergency braking. The NTSB notes that communication delay leaves unclear whether that slowdown began before the operator disengaged the ADS at \(\ImpactInstant-0.02\text{ s}\) (National Transportation Safety Board 2019).

Recognition came seconds after detection; suppression then spent 18.7 m with the brake withheld.

⁶ Jerk Limit: Jerk, met in the stopping notebook as the rate of change of deceleration, is measured in m/s³. A jerk limit makes a brake ramp its deceleration up instead of stepping it, which spares occupants, loads, and transmissions a shock but costs stopping distance, because full deceleration arrives later. The warehouse machine’s card carries a jerk bound for the same reason, and the machine’s finished stopping budget pays for the ramp.

The first term of the warehouse stopping budget applies directly. Averaging the two recorded speeds over the 1 s of suppression, the vehicle traveled about 18.7 m while emergency braking was withheld, distance that no later action could recover. The speed still fell during that second because a slowdown for the planned right turn at the next intersection was already under way; what the suppression withheld was the emergency response (National Transportation Safety Board 2019). The emergency itself was defined by a chosen limit. The ADS treated a situation as an emergency when avoiding a collision required more than 7 m/s² (0.71 g) of deceleration or a jerk⁶ beyond its limit, and it capped its own automated braking at that deceleration; police determined that the vehicle could decelerate at an average of 0.92 g, about 9.0 m/s², on that road surface (National Transportation Safety Board 2019). The difference between a chosen permission limit and a physical one is the distinction the book’s numbers keep throughout. The calculation stops short of a counterfactual stop, because the report does not establish when a different design would have begun braking. Detection, correct path prediction, hazard recognition, and physical braking are distinct events; reporting the first detection time alone conceals the failures between them.

↪ The stopping envelope, which charges every interval of delayed braking against the clear distance, is developed in Stopping Envelopes.

War Story 1.1: The disabled safety redundancy (2018)

Context: On March 18, 2018, a developmental ADS on a 2017 Volvo XC90 struck and killed a pedestrian crossing a road in Tempe, Arizona (National Transportation Safety Board 2019).

Mechanism: As traced above, classification changes discarded the object’s tracking history, and the one-second suppression withheld braking until only a gradual slowdown was planned. The design did not use maximum braking solely to mitigate an unavoidable collision (National Transportation Safety Board 2019).

Impact: The vehicle struck and fatally injured the pedestrian. The late gradual-slowdown plan could not recover the lost stopping opportunity.

Response: The NTSB determined that the probable cause was the vehicle operator’s failure to monitor the driving environment and the ADS because she was visually distracted throughout the trip by her personal cell phone. It found three contributing factors at Uber ATG, all a consequence of an inadequate safety culture: inadequate safety risk assessment procedures, ineffective oversight of vehicle operators, and a lack of adequate mechanisms for addressing operators’ automation complacency. The impaired pedestrian’s crossing outside a crosswalk and the Arizona Department of Transportation’s insufficient oversight of automated vehicle testing also contributed. The NTSB further found that deactivating the Volvo collision warning and automatic emergency braking, which Uber ATG had done over possible radar interference, removed a layer of safety redundancy, and it recommended stronger safety risk management for testing automated vehicles (National Transportation Safety Board 2019).

Systems lesson: Classification uncertainty, a delayed response rule, and reliance on an inattentive operator can combine. The design made the operator the primary countermeasure in an emergency, yet no mechanism monitored her attention in real time: authority rested on evidence nobody collected. A design needs timely hazard assessment and a response that can mitigate impact when avoidance is no longer possible; the investigation does not prescribe a particular hardware architecture.

⁷ Safety-Critical Standards: Three standards divide the ground. ISO 26262 addresses hazards caused by faults in a vehicle’s electrical and electronic systems (International Organization for Standardization 2018); ISO 21448, safety of the intended functionality (SOTIF), addresses hazards that arise when a function works as designed but performs insufficiently, as perception can (International Organization for Standardization 2022); and UL 4600 sets out the safety case for autonomous products with no human driver to fall back on (UL Standards & Engagement 2023). Tempe falls on the SOTIF side, since the recorded failures were in classification and path prediction rather than in a faulted component, and none of the three prescribes a processor architecture. The standards map for the running machine maps the standards that govern the running machine.

International Organization for Standardization. 2018. ISO 26262:2018: Road Vehicles, Functional Safety. International Organization for Standardization.

International Organization for Standardization. 2022. ISO 21448:2022: Road Vehicles, Safety of the Intended Functionality. International Organization for Standardization.

The investigation supports a broader architectural requirement. A developmental ADS needs safety redundancy and a timely mitigation path even when perception or path prediction fails, and simulation tests by Volvo that the NTSB cites suggest that the factory emergency braking might have prevented or mitigated this collision. The report does not establish when a separate permission path would have triggered or where it would have stopped the vehicle.⁷

Tempe illustrates failure before impact, where classification changes and a software suppression delay prevented timely braking. The 2023 Cruise robotaxi incident marks the opposite pole after impact. A hit-and-run driver’s car threw a pedestrian into the path of the driverless vehicle, which braked hard but struck her and stopped. It then began a pullover toward a minimal-risk condition⁸ and, because its sensing did not detect a person beneath the vehicle, moved about 20 ft (6.1 m) at up to 7.7 mph while she was pinned beneath it. This account comes from a review commissioned by the company’s boards, whose focus was the company’s disclosures to regulators (Quinn Emanuel Urquhart and Sullivan, LLP 2024).

⁸ Minimal-Risk Condition: In automated driving, the stable, usually stopped, state a system brings a vehicle to when a trip cannot or should not continue. Reaching it can itself require motion, as a pullover to the curb does, and in the Cruise case that motion was the harm. A fallback trajectory therefore needs the same permission check as any other proposal, judged against fresh evidence about the space it will move through.

↪ The Cruise pullover, and the fallback rule that forbids renewed motion while the space beneath the vehicle is unverified, are developed in The four rungs of the fallback ladder.

Neither failure reduces to an inaccurate model. The first spent stopping distance before braking began, and no later action could recover it. The second chose motion after the impact without fresh evidence about the space it moved through. Together they show what must hold before the machine acts and what a fallback may do once harm has begun. Both also had to be reconstructed afterward from records of what the machine perceived and when, so accountability, like safety, depends on the path that authorizes motion.

↪ Who answered for the Cruise collision, and the decisions behind an admitted action that no per-tick log records, are developed in Who Answered for the Cruise Collision.

Checkpoint 1.2: Forensics of boundary and authority failures

Verify how the Tempe mechanisms connect to the architecture:

• Classification changes and path prediction: Explain how classification changes caused the ADS path predictor to omit earlier object locations, despite continued tracking.
• Software suppression vs. kinematic clearance: Distinguish the \(\ImpactInstant-1.2\text{ s}\) recognition time from the one-second suppression period, and explain why the gradual-slowdown plan at \(\ImpactInstant-0.2\text{ s}\) could not restore the time already spent.
• Safety redundancy: Explain why disabling factory collision warning and emergency braking removed a layer of protection from a developmental ADS.

1.9 Where Safety, Security, and Accountability Meet

Tempe and Cruise both turned on what the machine knew when it moved or failed to brake, and both had to be reconstructed afterward from what it had recorded. The same three questions (whether the evidence behind a motion was sound, whether anything could refuse it, and whether the decision can be traced) also describe an attack and an audit. In enterprise software and distributed systems, safety, security, and accountability usually live in separate tiers. Security defends network boundaries with cryptographic tokens, reliability frameworks mitigate component failures through retries and circuit breakers, and accountability records transactions to append-only databases. In physical AI, all three converge on the causal boundary, the interface where software commands meet physical actuators. Consider the running machine crossing a warehouse aisle. It needs a valid stopping decision whether its forward cameras are obscured by dirt, an attacker has injected spoofed packets onto its internal Controller Area Network (CAN) bus, or a vision-language-action policy has hallucinated an unobstructed path. In each case the hazard is kinetic, a collision with racking or a worker. Because momentum cannot be rolled back like a failed database transaction, admission control cannot simply confirm who dispatched a command. It must determine whether executing that command now would violate the machine’s physical limits.

An ordinary environmental change challenges this boundary as directly as an attack. The running machine’s credible deceleration of 2 m/s² rests on the premise, stated on its card, that the floor supplies a friction coefficient of at least 0.204. When the floor cannot supply the friction the brakes ask for, traction rather than the brakes caps deceleration, and braking distance grows in inverse proportion to the available friction. Spilled hydraulic fluid can drop tire-floor friction from about \(0.8\) on dry concrete to \(0.2\) (an illustrative pair). A vehicle that brakes at the traction limit would see its braking distance quadruple. The running machine asks far less of a dry floor than it supplies, but \(0.2\) falls below its premise, so its credible deceleration no longer holds, and every clearance envelope and speed profile computed from that deceleration a moment earlier is invalid. The learned planner, processing camera streams that do not reveal the transparent slick, may continue to output an aggressive velocity command with high nominal confidence. Safety verification cannot treat the model’s internal confidence as proof of physical feasibility. Restricting the machine requires independent, real-time evidence from the plant and its surroundings, such as wheel-slip telemetry, disagreement between commanded and inertial-measurement-unit (IMU) acceleration, and motor current transients, to detect the divergence and restrict motion using a response feasible under the updated traction estimate. If the machine has already consumed the clearance that response needs, detection can support harm mitigation but cannot establish collision avoidance.

Security threats target this same permission machinery by corrupting the evidence used to authorize motion. An adversary does not need to crack cryptographic keys or modify model weights to destabilize a physical agent; distorting the sensory inputs that inform admission decisions is sufficient. Projecting structured light patterns onto camera lenses, spoofing ultra-wideband (UWB) localization beacons, or injecting rogue packets into sensor buses can deceive the controller into approving destructive commands, even while the neural network executes its inference pipeline without numerical fault. Isolating the proposer from the permission path offers little defense if both subsystems ingest the same corrupted telemetry. Securing the causal boundary therefore requires validating the evidence itself, by cross-checking heterogeneous sensor streams (for example, corroborating camera-based visual odometry against wheel encoders and inertial sensors), enforcing cryptographic message authentication across bus transceivers, and maintaining fail-operational plausibility checks that reject sensor readings deviating from physical conservation laws.

↪ A well-formed packet that carries a false measurement becomes a fault trial in the injection campaign developed in Physical sensor spoofing and proposer injection.

In digital systems, a cryptographic capability token or access control list entry grants authority to modify a file regardless of processor load or ambient conditions. In physical AI, physical authority is state-dependent. Cryptographic credentials establish identity and organizational rights to request an action, but physical admissibility depends entirely on the instantaneous operational state of the machine and its workspace. A joint velocity command of 2 rad/s may represent a routine transit when an industrial manipulator operates in open free space, yet that identical command becomes unacceptable if the arm is within 2 mm of a mechanical hard stop or a human coworker. Evaluating physical authority requires testing the proposed action against instantaneous kinematics, payload inertia, motor thermal headroom, and state estimation uncertainty. Admission control must also remain coupled to an executable fallback. If the permission gate rejects a proposed command because the machine approaches an operational boundary, the controller must immediately dispatch a dynamically feasible contingency maneuver, such as controlled regenerative braking or a compliant hold that yields to contact, that halts or redirects the plant without exceeding mechanical stress limits.

Accountability demands an audit trail that reconstructs the causal chain connecting sensory evidence to physical outcomes. When an embodied machine causes a collision or near-miss, post-incident analysis requires timestamp and clock-conversion uncertainty small enough to distinguish the relevant sensing, admission, dispatch, and physical-response events across multiple data streams: the raw observations ingested by the planner, the action proposed by the learned policy, the state estimates and dynamic constraints evaluated by the admission gate, the filtered command dispatched to the motor drives, and the measured response from joint encoders and force-torque sensors. Command logs record the intended trajectory, while sensor telemetry measures the plant’s actual mechanical response. Recording dispatched setpoints alone does not confirm that an actuator executed them; an open-circuit fault, a tripped thermal breaker, or mechanical gearbox jamming will cause the physical plant to diverge immediately from commanded behavior. Both streams must be logged against synchronized hardware clocks (such as the IEEE 1588 Precision Time Protocol) so that investigators can distinguish between a planning failure, an admission gate error, and an electromechanical actuator breakdown. Reconstruction is necessary but does not by itself say who answers for an action, because the decisions that set the machine’s envelope, its staffing, and its permission to operate are made before any tick and have to be recorded on purpose (Decisions Behind an Admitted Action).

↪ Protocol: The IEEE 1588 Precision Time Protocol exchange that aligns these clocks is developed in Clock Synchronization and Network Timestamps.

Safety, security, and accountability ultimately converge on three unified demands at the permission path guarding the causal boundary:

1. Fresh, authenticated physical state: The permission gate must operate on telemetry fresh enough to bound physical uncertainty and robust enough against tampering or degradation to justify admitting an action.
2. Deterministic authority to intervene: The admission mechanism needs uncompromised physical authority to veto or clamp infeasible proposals and inject safe contingency maneuvers, regardless of the learned model’s internal confidence or identity.
3. Synchronized causal telemetry: The system must keep an immutable record of what was proposed, what state was evaluated, what commands were dispatched, and how the machine responded.

The incidents above map onto these demands. Tempe’s predictor discarded evidence it already had, and the factory emergency braking that could have intervened independently had been disabled; Cruise moved without fresh evidence about the space beneath it. Both investigations then depended on the third demand, the recorded telemetry from which the events were reconstructed. The permission path is therefore where a physical machine’s safety, security, and accountability obligations are discharged, and anchoring probabilistic proposals to checks against physical limits is what lets learned capability operate within justified bounds. What those checks must respect, and why, is the subject of four laws.

↪ The operational log that survives physical failure, from which an investigator reconstructs each admitted action, is developed in Trustworthy Operational Logs.

1.10 The Four Bedrock Laws

Manipulating an unfamiliar door latch or transporting an unmodeled payload requires far more than a sequence of statistically accurate model predictions. An embodied machine must respect the mechanical limits of its plant, adapt to dynamic environments altered by its own motion, and arbitrate between speculative model outputs and physically safe actuation. Four bedrock laws organize these engineering responsibilities, each anchoring a major part of this book:

The first law: irreversibility. Software cannot roll back physical consequences. In pure digital systems, an erroneous transaction can be aborted and state restored from a checkpoint. In physical space, however, work performed on a plant is thermodynamic and permanent. A motor command that collides with a fixture, tears a cable, or fractures a fragile payload cannot be retracted. Every admissibility check must therefore complete before the machine consumes its remaining clearance margin. As the stopping distance budget demonstrates, processing latency directly expends physical clearance, forcing the mechanical actuators to brake over an ever-shrinking distance. Reversing a drive voltage may eventually return an arm to its starting coordinates, but it cannot mend cracked structural material or recover energy lost to friction and heat. Part I translates this physical reality into kinetic budgets for the robot’s body and deterministic timing constraints for its low-level controllers.

The second law: endogenous data. In classic offline machine learning, data samples are drawn independently from a stationary distribution. In embodied systems, data generation is endogenous. Every control action alters the physical state and determines what sensors will observe next. If an arm nudges an object during an approach, the object shifts; if a mobile base drifts off its nominal path, the subsequent visual frames deviate from the training demonstrations. Single-step prediction accuracy on an offline benchmark cannot guarantee closed-loop behavior, because minor execution errors compound across the task horizon, the covariate shift met with the closed causal loop (section 1.4.1). Part II frames data collection, model training, and policy evaluation as a continuous improvement cycle: diagnose behavioral failures, curate targeted recovery demonstrations, retrain candidate policies, and systematically evaluate both performance gains and unintended regressions. Empirical evidence from past rollouts bounds what can be claimed about present capability; only qualified new experience can expand that envelope.

The third law: proposal is not permission. A learned model operates only as a proposer, never as an unconstrained executor. An architecturally independent permission path (a runtime safety filter or an assurance monitor, for example) must arbitrate every proposed control command before that command reaches hardware actuators. Granting permission requires three prerequisites: fresh state estimates from verified sensors, an explicit check that the command respects every physical safety envelope, and an immediately executable fallback maneuver if the proposal is delayed, missing, or unsafe. This separation of authority must remain robust against system faults, so a runaway or stalled model thread must never be allowed to delay, starve, or corrupt the deterministic safety logic guarding the plant. Part III constructs the real-time execution runtime around this division between speculative proposal and independent, bounded permission.

The fourth law: permission must be earned. An embodied machine earns permission to operate only as far as empirical evidence supports its performance under stated conditions. Flawless execution across a handful of laboratory trials does not justify autonomy across diverse surface friction, variable payloads, degraded lighting, or sensor degradation. Every deployment safety case must explicitly demarcate the system’s operational design domain (ODD),⁹ articulate the environmental assumptions underlying those boundaries, and specify fail-safe degradation policies whenever runtime conditions drift beyond verified bounds. The 2018 Tempe collision showed the cost of ignoring this boundary. The system architecture relied on an in-vehicle safety operator as the primary emergency countermeasure, yet lacked any mechanism to monitor whether the operator was actually attending to the roadway (National Transportation Safety Board 2019). Part IV formalizes command authority, fault injection testing, and release criteria so that epistemic and physical uncertainty bound what the machine is permitted to do.

⁹ Operational Design Domain (ODD): The automated-driving term for the stated environmental and operating conditions (roadway types, speed ranges, weather, lighting) under which a system is designed to work, and the starting scope of a UL 4600 safety case (UL Standards & Engagement 2023). This book applies the concept to the payloads, surface friction, operating speeds, and disturbances covered by the machine’s empirical testing. Operating outside that tested domain has no evidence behind it, which is why Choosing a tractable coverage table organizes the operating domain as an explicit scenario table against which collected training data are audited.

UL Standards & Engagement. 2023. ANSI/UL 4600 Ed. 3-2023: Standard for Evaluation of Autonomous Products. UL Standards & Engagement.

Of these four, only the first is a physical law. Irreversibility follows from Newtonian mechanics and thermodynamics. Work performed on a plant transfers energy and raises entropy, and no software architecture can undo that transition. The remaining three are engineering design principles, distilled from decades of systems engineering, industrial robotics, and aerospace failures. A team could build an architecture that violates them, for example by piping unvalidated network outputs straight to motor amplifiers, but it would incur severe and predictable risks. The book elevates all four to bedrock laws because it treats them as mandatory constraints for any production embodied system.

None of these three architectural principles is entirely novel. Each has a rich foundation in computer systems, control theory, and safety-critical engineering. Table 3 maps this book’s terminology to the established literature, alongside the canonical sources and documented real-world failures each principle distills, including the Tempe and Cruise incidents. Where the book’s terms introduce tighter operational precision, as when tying state freshness directly to the dynamic stopping budget, the book retains specialized systems phrasing; when searching broader engineering literature, the classical terms serve as the standard index.

Table 3: Lineage of the four laws: Each law and key term of this book against the field’s established name, its canonical source, and a documented failure it distills. Only the first law is physical; the other three are design principles, which is why each has an independent literature. Runtime assurance enters with the proposal boundary, and section 1.14 traces its lineage together with the safety filter and the assurance case.

Book’s law or term	Kind	Established name in the field	Canonical source	Documented failure it distills
First law: irreversibility (the causal boundary)	Physical	Irreversible work on the plant; the reaction-plus-braking stopping budget	Mechanics; Kinetic Momentum	Tempe (2018): the one-second action suppression let the vehicle travel about 18.7 m with emergency braking withheld (National Transportation Safety Board 2019)
Second law: endogenous data	Design principle	Covariate shift; compounding error in imitation learning	Ross, Gordon, and Bagnell (Ross et al. 2011)	A learning result rather than an accident: a cloned policy’s errors can compound with the task horizon (Mechanics of Endogenous Covariate Shift)
Third law: proposal is not permission (proposal boundary, permission path)	Design principle	Runtime assurance and its Simplex architecture; safety filter	Sha (Sha 2001); Hsu, Hu, and Fisac (Hsu et al. 2024)	Therac-25: software took over protection that hardware interlocks had provided (Leveson and Turner 1993); Mars Pathfinder (1997): through priority inversion, lower-priority work delayed a deadline-critical bus task until the scheduler’s completion check reset the computer (Reeves 1997)
Fresh evidence (an input to every permission check)	Design principle	Age of information	Kaul, Yates, and Gruteser (Kaul et al. 2012)	While not a pure staleness failure, Tempe (2018) shows the neighboring hazard of discarded evidence: each object reclassification dropped historical tracks, forcing path prediction to compute trajectories on truncated sensor history (National Transportation Safety Board 2019)
Fourth law: permission must be earned	Design principle	Assurance case; operational design domain (ODD)	UL 4600 (UL Standards & Engagement 2023)	Tempe (2018): the design made the operator the primary countermeasure in an emergency, yet nothing monitored her attention in real time (National Transportation Safety Board 2019); Cruise (2023): a post-collision pullover moved the vehicle without evidence that the space beneath it was clear, because its detection system did not identify the pedestrian there (Quinn Emanuel Urquhart and Sullivan, LLP 2024)

Ross, Stéphane, Geoffrey Gordon, and J. Andrew Bagnell. 2011. “A Reduction of Imitation Learning and Structured Prediction to No-Regret Online Learning.” Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics (AISTATS) 15: 627–35.

Reeves, Glenn E. 1997. What Really Happened on Mars? Electronic mail message, Jet Propulsion Laboratory.

Kaul, Sanjit, Roy D. Yates, and Marco Gruteser. 2012. “Real-Time Status: How Often Should One Update?” Proceedings of the IEEE Conference on Computer Communications (INFOCOM), 2731–35. https://doi.org/10.1109/INFCOM.2012.6195689.

Quinn Emanuel Urquhart and Sullivan, LLP. 2024. Report to the Boards of Directors of Cruise LLC, GM Cruise Holdings LLC, and General Motors Holdings LLC Regarding the October 2, 2023 Accident in San Francisco. Quinn Emanuel Urquhart; Sullivan, LLP.

The proposal boundary is where all four laws meet in one architecture. Because irreversibility requires that every check complete before a command alters the plant, and because endogenous data prevents offline training metrics from certifying a proposer in the closed loop, the third law (proposal is not permission) places that check on an independent permission path that the model cannot delay or corrupt, while the fourth law (permission must be earned) confines admitted proposals to the operating conditions that empirical evidence has verified. Scaling up model capacity does not relax these constraints; on the contrary, it intensifies physical pressure on the system. Although general machine learning methods deliver superior performance as compute expands (Sutton 2019), the running mobile machine must supply its onboard compute within a finite electrical and thermal budget. Every additional neural layer, attention head, or diffusion step draws against fixed physical allocations of memory bandwidth, electrical power, and thermal dissipation. Inference latency on the stopping path is travel that braking cannot recover. On the running machine at its drive limit, each millisecond costs 1.5 mm of clearance, as the stopping notebook showed. This compute, thermal, and temporal ceiling forms the physical wall confronting every learned proposer.

Sutton, Richard S. 2019. “The Bitter Lesson.” Incompleteideas.net 43.

↪ The memory-bandwidth and latency bottlenecks confronting onboard learned proposers are developed in The Cost of a Fresh Evaluation.

Finally, rejecting an inadmissible command must have an explicit physical realization. In traditional software, an unhandled exception or failed assertion can simply terminate a thread. On a moving robot, halting a software process leaves physical momentum untouched. A refusal is meaningful only if the execution runtime immediately dispatches an admissible fallback (a dynamic braking maneuver, a compliant hold, or a controlled stop) that the physical body can safely execute within its remaining stopping clearance. Translating the four laws and their runtime mechanisms into an engineered system takes the four parts of this book, each producing records that later parts rely on.

1.11 The Architecture of This Book

Moving a learned proposal, such as the running machine’s trajectory to open a warehouse cage door, from an abstract task requirement to safe, justified physical operation requires a disciplined four-stage progression. The engineer must first bound what the hardware plant can physically achieve, characterize what the learned proposer has demonstrated under rigorous testing, build an isolated runtime monitor to intercept live proposals, and establish the operational conditions under which continued execution remains justified. Each stage corresponds to one of the four laws. The system of records illustrated in figure 9 formalizes this progression, capturing operating assumptions and empirical evidence so that downstream runtime decisions can inspect and rely on them. Conceptually, these records trace two converging pipelines. One strand establishes offline physical limits and verified competence; the second processes online sensor observations and real-time candidate actions. Both pipelines converge at the permission decision, where the system admits or rejects a proposed trajectory. In From a Better Prediction to Useful Service, this architecture synthesizes both strands to resolve the book’s central challenge of engineering dependable, high-performance physical systems out of unverified learned models and unforgiving mechanical hardware.

A record is a machine-checkable artifact that downstream safety and runtime decisions can verify and trust. The initial stopping calculation (notebook 1.1) already sketches the first record in the chain: a limit record, with the six elements the reader guide introduces (nominal value, operating conditions, uncertainty, margin, runtime detector, and provenance). Its nominal value is the derived stopping distance under stated operating conditions of loaded payload mass, dry concrete, a stationary obstacle, and constant approach speed. Because those inputs are illustrative scalars rather than calibrated distributions, however, its uncertainty remains unquantified, which means the apparent buffer between the obstacle and the machine is only an arithmetic difference between provisional figures rather than a verified safety margin. The calculation also lacks a runtime detector—no sensor yet monitors whether floor traction drops or cargo shifts outside the assumed envelope (The six elements of a limit record)—and its provenance rests on baseline assumptions rather than calibrated hardware logs. An ungrounded calculation becomes operational engineering evidence only when all six fields are populated and backed by empirical data. Every numbered box across the four part bands in figure 9 establishes a machine-checkable record of this rigor for a distinct architectural decision.

Figure 9: The spine of the book: Each part is led by one law, and each chapter of Parts I to IV except The Cognitive Brain (Chapter 3) produces a record that a later one consumes. What the plant can do and what the proposer has shown it can do run as one strand, what the proposer proposes now as another, and the two meet at the permission decision.

Part I, The Machine Anatomy, translates physical irreversibility into rigid architectural limits: The Physical Body profiles the mechanical plant to quantify its physical budgets (measurement freshness, stopping distance, deliverable torque, winding heat, and supply voltage); The Cognitive Brain measures the throughput and inference latency of the learned proposer to determine what it can deliver before deadlines expire; and The Nervous System links compute to the actuators across deterministic buses, structuring a permission path whose timing guarantees derive from the plant’s stopping requirements. Together, the limit record and handoff record make every physical and temporal assumption explicit. Part II, Teaching the Machine, confronts the reality that physical actions continuously alter future sensor observations. Physical Data, Policy Training, and Closed-Loop Evaluation build the evidentiary chain required to collect demonstrations, train control policies, and bound policy performance to a declared operational design domain. Across this pipeline, the provenance record, policy manifest, and evaluation record document both the empirical competence demonstrated by the model and the operational boundaries where its statistical guarantees break down.

Part III, Running the Machine, enforces at runtime the separation between proposing an action and permitting it to execute. From Sensor Perception through Workload Placement, candidate commands are traced through the online runtime: qualifying sensor measurements in observation contracts, maintaining state estimates in a belief record, bounding a goal with an intent lease (a time-limited grant of a target, never of a path, and distinct from the chunk lease that bounds one motion segment), and checking candidate trajectories against real-time physical constraints before admission and dispatch, then testing whether the hardware keeps the permission path independent. Part IV, Governing the Machine, enforces the principle that permission must be earned: Supervisory Intervention, Adversarial Verification, and Deployment Release govern how authority transfers between the machine and a human operator, how fault injection tests whether protective responses hold, and how deployment decisions are assembled into release manifests; Responsible Physical AI then names who answers for each decision behind an admitted action, tests whether each human role the design relies on is one a person can perform, and counts the people who bear the machine’s risk and appear in its records. Finally, Engineering Physical AI brings the framework to bear on an unfamiliar machine, synthesizing what an engineering team can construct and formally justify while identifying the residual risks that require further mitigation. Across these parts, the warehouse mobile manipulator introduced here provides a single concrete machine on which to ground each architectural decision. Its records render design assumptions fully inspectable, though an engineer must remember that assembling a record documents an explicit claim rather than guaranteeing its physical truth. The system record register in The record register catalogs all sixteen records, noting their producing chapters and the specific invariants they bind, while the machine card in section 1.4.2 specifies the foundational physical parameters of the robot itself. Together, this four-stage route forms a repeatable discipline for making learned behavior dependable in physical space, guarding against the common misconceptions that arise when digital assumptions are carried across the causal boundary.

1.12 Fallacies and Pitfalls

The misconceptions that follow arise when assumptions from pure software engineering, statistical learning, or classical control theory are imported into embodied machines. In digital systems, an erroneous state can typically be trapped, rolled back, or restarted. Across the causal boundary, however, software outputs become physical forces that alter the world irrevocably. Preventing physical harm requires hard deterministic guarantees that learning objectives, human operators, and asynchronous recovery routines cannot provide on their own.

Fallacy: Safety constraints can be trained into the policy.

Engineering teams often attempt to enforce physical safety by adding penalty terms, barrier penalties, or negative rewards to the training objective. While penalty shaping suppresses undesirable actions over the training distribution, it provides no mathematical guarantee on the actions emitted during deployment. Deep neural networks remain uncalibrated function approximators when exposed to out-of-distribution inputs, sensor degradation, or novel environmental dynamics. A statistical loss function optimizes expected performance, but safety requires hard invariants that hold on every control cycle. For this reason, deterministic safety enforcement must sit downstream of the policy, operating as an independent permission path that filters proposed commands before they reach actuator driver registers, exactly where the third law places the boundary gate.

Pitfall: Treating human supervisory intervention as instantaneous in shared-autonomy systems.

In shared-autonomy designs, such as a robotic surgical tool providing haptic guidance from a learned anatomical model, system designers sometimes treat the human operator as a failsafe catch-all. This assumption ignores the physiological limits of human reaction time, which ranges between 150 ms and 250 ms for simple tactile or visual reflexes and grows under cognitive load. Once console video and a handshake are added, the running machine’s card budgets an in-loop takeover at 605 ms. Over that interval, an actuator running on a 1 kHz control loop can execute hundreds of unchecked cycles, generating enough force to tear soft tissue or damage a mechanical workpiece before the operator can intervene. Delegating authority to a human supervisor does not eliminate the machine’s capacity for harm during the supervisory lag. The permission path must enforce local force, velocity, and displacement envelopes autonomously, preserving the established physical envelopes under their operating assumptions without relying on timely human overrides.

Pitfall: Assuming software latency merely degrades performance gracefully.

In purely digital systems, inference latency or scheduler jitter increases response time or degrades service quality. In physical systems, timing is a functional correctness requirement. If a learned policy stalls, whether from accelerator kernel contention, memory allocation pauses, or network serialization delays, allowing its previous command to persist across subsequent control cycles turns stale intent into physical danger. For example, a mobile manipulator that repeats its last forward-velocity command during a 100 ms inference hiccup can collide with an unmapped obstacle. Actuation commands must therefore carry bounded temporal validity, enforced by hardware or real-time software watchdogs. When that validity window (the chunk lease of the stopping budget) expires without a verified update, the low-level controller must begin a fallback within its established response budget. The selected stop or hold must remain feasible for the current mechanism, state, and surviving resources.

Fallacy: Assuming physical actuation is reversible through subsequent software compensation.

Transactional software architectures rely on rollbacks and compensation routines to undo erroneous operations. In embodied systems, thermodynamics and materials science break this equivalence. If a robotic arm applies excessive torque to a workpiece, issuing a reverse motor command cannot unbend plastically deformed metal, repair fractured composite, or recover the electrical energy dissipated as heat in the motor windings (\(\MotorPhaseRmsCurrent^2 \PhaseResistance\)). Anomaly detection routines that react after physical contact can only truncate ongoing damage; they cannot undo the mechanical work already transferred across the causal boundary. Preventive verification, such as checking proposed trajectories against workspace boundaries, torque limits, and kinetic energy caps before dispatch, must serve as the primary defense. Reactive recovery strategies remain necessary for unexpected disturbances, but they function as a secondary backstop rather than a substitute for preventive enforcement.

1.13 Summary

Physical AI converges two historically distinct engineering disciplines: classical robotics and modern machine learning. As learned models move progressively closer to direct actuator control, the boundary between digital computation and physical reality sharpens at the causal boundary, the interface where an admitted command becomes an irreversible physical effect. In purely software systems, state rollbacks and retries can recover from algorithmic faults; in the physical world, software cannot undo kinetic energy, momentum, or impact. Safe recovery requires an executable, physically feasible countermeasure.

Physical dynamics also do not pause while neural networks compute. Because a machine continues to travel during sensing and processing delays, safety requires deterministic admission checks upstream of any consequential actuation. As the warehouse mobile manipulator’s stopping budget and the Tempe collision show, prediction quality offers no protection once response delay has spent the clearance. The architecture resolves this tension by separating proposal from permission and giving an independent, bounded-latency permission path the authority to refuse. Four laws, one physical and three design principles, organize the framework the rest of the book develops, while the elementary stopping budget and its crossover speed form this chapter’s quantitative core (table 4).

Table 4: Key equations of the causal boundary: The elementary stopping budget and its crossover speed, with the place each is derived.

Equation	What it bounds or decides	Derived
\(\StoppingDistance = \SpeedAtDetection\PrebrakeDelay + \dfrac{\SpeedAtDetection^2}{2\CredibleBrakingDeceleration} \le \ClearDistance\)	Whether a moving machine can stop within its observed clearance: distance traveled during response delay plus braking distance, prior to allowances added in later chapters	equation 1
\(\StoppingCrossoverSpeed = 2\CredibleBrakingDeceleration\PrebrakeDelay\)	The crossover speed at which delay distance equals braking distance; below \(\StoppingCrossoverSpeed\), delay travel is larger, while above it, braking travel is larger	From equation 1, section 1.4.3

Key Takeaways: Engineering across the causal boundary

• Useful behavior requires an integrated machine: Physical AI combines learned decision-making, physical dynamics, and delegated mechanical authority. Delivering reliable autonomous operation requires co-designing task competence with physical body limits, real-time compute runtimes, and the operational evidence supporting deployment.
• Response delay consumes physical margin: The elementary stopping budget \(\StoppingDistance = \SpeedAtDetection\PrebrakeDelay + \SpeedAtDetection^2/(2\CredibleBrakingDeceleration) \le \ClearDistance\) partitions stopping distance into latency-induced travel and mechanical braking. Below the crossover speed \(\StoppingCrossoverSpeed = 2\CredibleBrakingDeceleration\PrebrakeDelay\), response delay dominates stopping distance; above \(\StoppingCrossoverSpeed\), braking deceleration dominates. For the warehouse mobile manipulator at its drive limit, pre-braking travel adds 35.6 percent to braking distance. The decisive delay belongs to whichever path triggers the protective response, which can operate independently of the learned proposer’s inference time.
• Proposal and permission serve distinct architectural roles: The learned proposer (the Brain) generates goal-directed actions, while an independent permission path admits, modifies, or rejects them before they cross the causal boundary. Safety enforcement relies on validated sensor inputs, physical kinematic envelopes, strict real-time deadlines, and verifiable fallback maneuvers.
• Evidence governs safe operation and systematic refinement: Four laws structure this book. Irreversibility is a physical law; the other three (endogenous data, proposal is not permission, and permission must be earned) are design principles that translate covariate shift, runtime assurance, and the assurance case into systems practice. Together they link physical hardware, embodied learning, real-time execution, and operational governance, establishing verified service within defined operational envelopes while identifying the targeted evaluations needed to expand them.

Before judging whether a proposed action is right, an engineer should establish what physical response would remain if the proposal were wrong, stale, or never arrived, and whether the machine still holds the clearance, the time, and the authority to carry that response out. A design that cannot answer from its own measured limits has handed safety to prediction quality, and the stopping budget showed that prediction quality cannot buy back clearance once delay has spent it. The answer therefore starts from what the plant can do under stated conditions, never from what the model believes.

1.14 Historical Perspective and Further Reading

In physical artificial intelligence, moving learned models directly into the actuation loop creates severe timing and safety hazards. The engineering principles that resolve these challenges originate in embedded systems, real-time control, and safety engineering. General-purpose computing treats execution time as a mere performance optimization, assuming that correctness depends solely on logical input-output relationships. Cyber-physical systems theory overturned this assumption, holding that physical dynamics and computational processes must be modeled jointly because a command dispatched at the wrong time is physically incorrect (Lee 2008). This insight forms the foundation of the causal boundary. Similarly, separating command proposal from execution permission stems from runtime assurance architectures. The Simplex architecture demonstrated that an industrial control system could safely run an unverified, high-performance controller if paired with a certified baseline controller and deterministic switching logic capable of taking back the plant before safety boundaries were violated (Seto et al. 1998; Sha 2001). Modern safety filters in embodied machine learning generalize this design, validating neural network proposals against mathematical certificates of safe states (control barrier functions or reachability envelopes, which Safety Enforcement develops) before granting actuator permission (Hsu et al. 2024).

Seto, Danbing, Bruce Krogh, Lui Sha, and Alongkrit Chutinan. 1998. “The Simplex Architecture for Safe Online Control System Upgrades.” Proceedings of the 1998 American Control Conference (ACC) 6: 3504–8. https://doi.org/10.1109/ACC.1998.703255.

Hsu, Kai-Chieh, Haimin Hu, and Jaime F. Fisac. 2024. “The Safety Filter: A Unified View of Safety-Critical Control in Autonomous Systems.” Annual Review of Control, Robotics, and Autonomous Systems 7: 47–72. https://doi.org/10.1146/annurev-control-071723-102940.

Kelly, Timothy Patrick. 1998. “Arguing Safety: A Systematic Approach to Managing Safety Cases.” PhD thesis, Department of Computer Science, University of York.

Many foundational safety principles in physical systems emerged from forensic investigations of catastrophic failures. The Therac-25 radiation therapy disasters demonstrated the fatal peril of replacing mechanical interlocks with software routines (Leveson and Turner 1993). When race conditions bypassed software checks, the machine administered lethal radiation overdoses, proving that software reliability does not imply system safety. Systems-theoretic safety subsequently recast accidents not as isolated component breakdowns, but as failures of the overall control structure to enforce safety constraints on the interactions among components, feedback loops, and human operators (Leveson 2011). This shift led directly to the assurance case, a structured argument demonstrating that concrete evidence satisfies defined safety claims within explicit operating environments (Kelly 1998). The UL 4600 standard adopted this structure as its central framework for autonomous products (UL Standards & Engagement 2023), abandoning arbitrary road-testing mileage in favor of verifiable, claim-based arguments. Public accident records, particularly the National Transportation Safety Board investigation of the 2018 Tempe autonomous vehicle collision (National Transportation Safety Board 2019), remain the field’s most complete documentation of how safety arguments unravel when classification changes, a response delay chosen in software, and reliance on a human fallback interact in the field.

• Cyber-physical systems (Lee 2008). Lee articulates why classical computing abstractions fail when computation interacts with physical processes. General-purpose computer architectures deliberately abstract away time, treating execution speed as a performance consideration rather than a correctness specification. Lee demonstrates that for physical interactions, the timing of an actuation command is inseparable from its physical meaning, providing the conceptual foundation for treating the causal boundary as an explicit systems interface.
• Runtime assurance (Sha 2001). Sha’s paper on the Simplex architecture is the place to see how the switching logic decides when to take the plant back from the unverified controller, and why that logic must be simpler than what it supervises. It is the pattern behind this book’s separation of proposal from permission.
• Therac-25 (Leveson and Turner 1993). Leveson and Turner’s forensic analysis traces the specific race conditions and the development and reporting practices that let the overdoses recur, detail the summary above omits. It remains the essential case study in why protective functions need justification independent of the software they supervise.
• Systems-theoretic safety (Leveson 2011). Leveson introduces the STAMP accident model, reframing safety as a continuous control problem rather than a collection of component failure probabilities. Accidents occur when the control structure fails to enforce safety constraints across component interactions, sensor delays, and feedback loops. This text provides the theoretical basis for enforcing safety invariants as dynamic runtime constraints rather than static design assumptions.
• The Tempe investigation (National Transportation Safety Board 2019). The National Transportation Safety Board report on the fatal 2018 Tempe autonomous vehicle crash offers the most thorough empirical record of an embodied AI safety failure. The report details how classification changes that discarded tracking history, a one-second action suppression meant to limit responses to false alarms, the separate deactivation of the vehicle’s factory emergency braking over possible radar interference, and reliance on an operator whose attention nothing monitored combined in the collision. The event timeline and sensor telemetry provide vital reading for anyone building real-time perception-action loops.

Lee, Edward A. 2008. “Cyber Physical Systems: Design Challenges.” Proceedings of the 11th IEEE International Symposium on Object and Component-Oriented Real-Time Distributed Computing (ISORC), 363–69. https://doi.org/10.1109/ISORC.2008.25.

Sha, Lui. 2001. “Using Simplicity to Control Complexity.” IEEE Software 18 (4): 20–28.

Leveson, Nancy G., and Clark S. Turner. 1993. “An Investigation of the Therac-25 Accidents.” Computer 26 (7): 18–41.

Leveson, Nancy. 2011. Engineering a Safer World: Systems Thinking Applied to Safety. MIT Press.

What’s Next: From the causal boundary to physical limits

A permission decision needs a physical limit against which to judge the proposed action. The Physical Body begins Part I with the five budgets the permission path must check before it commits torque (measurement freshness, stopping distance, deliverable torque, winding heat, and supply voltage) and asks which binds first for a given maneuver, under what stress, and on what evidence. Each budget ends as a limit record, the six-element form this chapter sketched from its stopping calculation. The Cognitive Brain then examines the learned proposer’s capabilities and timing, and The Nervous System uses both sets of constraints to design the handoff and independent permission path. The progression starts with what the machine can sustain and ends with how computation may safely request it.