Physical AI Notes
KO

Edge server: lessons

A workstation on the robot's network that handles heavy model inference. VLAs such as GR00T or π0 run as policy servers that the robot calls.

Around the robot

No lessons are published yet. Here are the planned topics.

Overview

How to decide where inference runs (dividing work among the onboard computer, edge server and data center), why memory bandwidth and power come before TOPS

  1. Criteria for where to run inference: dividing the work among the onboard computer, edge servers and the data centerComing soon

    Cycle time, tail latency, link dropouts and data volume as the criteria for where inference runs, and the loops each location takes on.

  2. Numbers to check before TOPS: memory bandwidth and powerComing soon

    The performance ceiling for batch=1 inference and how to read a spec sheet.

Basics

Setting up a VLA policy server and calling it from the robot, measuring round-trip latency

  1. Setting up a VLA policy server and calling it from the robot: request–response and measuring round-trip latencyComing soon

    Running the publicly available openpi policy server and GR00T inference examples, calling them from a simulator client, and measuring the round-trip latency distribution.

Advanced

Analyzing VLA inference with the roofline model, comparing FP16 and INT8 engines in closed loop, async execution and Real-Time Chunking

  1. Memory bandwidth determines the inference speed of generative models: analyzing VLA inference with the roofline modelComing soon

    How the bottleneck differs between token-based and flow-matching VLAs, and computing the bandwidth-bound ceiling.

  2. The performance cost of quantization shows up in closed loop: comparing FP16 and INT8 enginesComing soon

    Cases where FP16 and INT8 engines rank one way on open-loop error and the opposite way on closed-loop success rate.

  3. The arm must keep moving during inference: asynchronous execution and Real-Time ChunkingComing soon

    How inference latency relates to the execution horizon, and a comparison of four execution schemes.

In practice

Criteria for moving a loop off the robot, tail latency and handling link dropouts

  1. Which loops can move off the robot: tail latency and handling link dropoutsComing soon

    The four criteria of cycle time, tail latency, link-dropout handling and data volume, and tc netem experiments limited to the control, policy and perception loops.