SageMaker's Training Sessions Let Your Own Python Drive a Live GPU-Backed Session, Billed by the Token
AWS presents SageMaker AI's training session APIs as the right starting point when you need fast iteration and algorithmic freedom—a custom loss, a new reinforcement-learning reward signal, or a multi-stage pipeline—without provisioning or operating GPU infrastructure yourself. You drive a live, GPU-backed session from your own Python process on your local machine; the service runs the GPU-bound sampling, gradient and checkpoint operations while you keep the data, loop control and reward.
Usage is billed per token processed, so you are charged only for the compute your loop consumes. AWS's example: if a reward function takes ten seconds to score a batch on your machine, no GPU time is billed while it runs. That puts the slow local parts of a training loop off the GPU meter.
AI-generated discussion · Different perspectives
The billing is the more interesting half: per-token charges put the meter on the model's work, not on the ten seconds your Python spends scoring a batch.
The pitch sells algorithmic freedom, but freedom to design your own loop is also freedom to design a bad one. Removing GPU provisioning does not tell you whether your custom reward is teaching the model anything sensible.