Run a Classic RL experiment¶

Train a Classic RL agent on a built-in Gym environment using the experiment wizard. About 15–20 minutes if compute is available.

Prerequisites¶

Arena account with permission to create projects.
Classic RL project (not Advanced Training). You cannot switch modes after creation.

For an interactive version of this flow, click the Setting up an experiment card on Home.

1. Open Experiments¶

After sign-in you land on Home. In the left sidebar, click Experiments.

Home page with Experiments highlighted in the sidebar — The left sidebar with **Experiments** selected.¶

2. Projects list¶

The Experiments page lists your projects as tiles (default) or a table. Click New project, or Create Project if the list is empty.

Experiments page showing project tiles — The projects list on **Experiments**.¶

3. Create a project¶

Click New project (or Create Project if the list is empty). Enter a Name and Description. Leave Classic RL selected on the Training switcher. Confirm with Create Project.

New project dialog with Classic RL training mode — The new project dialog with **Classic RL** selected.¶

4. Project page¶

Arena opens the new project. The Experiments table is empty until you add a run.

Empty project page with Create Experiment button — The project page ready for a first experiment.¶

5. New experiment¶

A new project has no experiments, so click Create Experiment. If the project already has runs, use New experiment in the toolbar. Enter a name and confirm with Create Experiment. The setup wizard opens: Resources, Environment, Agent, Training, HPO, Summary.

New experiment name dialog — Naming a new experiment before the wizard opens.¶

6. Resources¶

On Training cluster, pick Number of nodes, GPU class, GPU size, and Number of GPUs. Click Next.

Experiment wizard Resources step — The **Training cluster** step with node and GPU pills selected.¶

7. Environment¶

Choose a built-in Gym environment. Review validation summary or charts if shown. Click Next.

Experiment wizard Environment step — The **Environment** step with a gym selected.¶

8. Agent¶

Select an algorithm such as PPO. Defaults are enough for this walkthrough. Click Next, then continue through Training and HPO with defaults.

Experiment wizard Agent step with PPO — The **Agent** step with an algorithm chosen.¶

9. Train¶

On Summary, review settings and click Train. The experiment row shows a live status in the project table.

Experiment wizard Summary step with Train button — The **Summary** step before training starts.¶

10. Results and checkpoints¶

Open the Results tab to inspect metrics and logs when the run finishes. To view checkpoints, switch to the Experiments tab and expand the experiment row.

Results tab with experiment metrics — The **Results** tab after a run is available.¶

Status labels are defined in Experiment statuses.

From the terminal¶

Create a Classic RL project. Save a default spec from Python as run-spec.json, then submit. Copy a resource id from arena resources list. Built-in gym names match the grid on Environment.

Install the CLI and sign in (Authentication):

pip install "agilerl[arena]"
arena login
# or: export ARENA_API_KEY=arena_pat_…

import json
from pathlib import Path

from agilerl.arena import ArenaClient

with ArenaClient() as client:
    spec = client.get_default_run_spec(algorithm="PPO", gym_env="CartPole")
    Path("run-spec.json").write_text(json.dumps(spec, indent=2))

arena projects create classic-demo --description "Classic RL"

arena resources list

arena experiments submit ./run-spec.json --project classic-demo --experiment-name ppo-cartpole --resource-id a2-highgpu-1g --num-nodes 2

Python¶

from agilerl.arena import ArenaClient

with ArenaClient() as client:
    client.create_project("classic-demo", "Classic RL", llm_based=False)
    spec = client.get_default_run_spec(algorithm="PPO", gym_env="CartPole")
    client.submit_experiment(
        manifest=spec,
        project="classic-demo",
        resource_id="a2-highgpu-1g",
        experiment_name="ppo-cartpole",
        num_nodes=2,
    )
    print(client.list_experiments(project="classic-demo"))