Snowflake: from sign-up to a deployed model¶
Train an LLM on a table in your Snowflake account, then serve it from the same account. Training and serving run on Snowpark Container Services inside your account. Your data, checkpoints, metrics, and weights stay there.
Two people do the work. A Snowflake account admin runs steps 1 to 5 once. A data scientist runs steps 6 to 9 for each model. Step 10 is for whoever owns the cleanup. One person can do both.
Each step lists the arena snowflake commands for it and says what happens. All commands at the end lists them in one table. The Arena Snowflake CLI reference has every flag and limit.
Step 1: Get a Snowflake account¶
What you do
Sign up at signup.snowflake.com. Pick the cloud and region you want to train in. Then:
Add a payment method, so the account is not a trial.
Sign in as a user that has the
ACCOUNTADMINrole. The user who created the account has it.Check that the region has GPU families. Run this in a Snowsight worksheet:
SHOW COMPUTE POOL INSTANCE FAMILIES;
Look for a family in the GPU table, such as
GPU_NV_Son AWS orGPU_GCP_NV_L4_1_24Gon Google Cloud.Note the account identifier in
ORG-ACCOUNTform. Every command below asks for it:SELECT CURRENT_ORGANIZATION_NAME() || '-' || CURRENT_ACCOUNT_NAME();
What happens
Training needs compute pools and external access integrations. Trial accounts cannot create external access integrations, so setup in step 4 stops on them with Snowflake error 509009 and creates nothing. GPU families differ by cloud, and not every region has every family. See Requirements.
Step 2: Get Arena credentials from AgileRL¶
What you do
Contact AgileRL and ask for:
Snowflake training turned on for your organization.
An org key scoped to Snowflake runs only. It starts with
arena_org_.The external user ID that goes with the key.
What happens
Setup stores the key and the user ID as secrets in your Snowflake account. Only jobs that run as the arena_trainer role can read them. The key works for Snowflake runs and nothing else, so anyone who reads the secret cannot use it elsewhere in Arena. See Get Arena credentials.
Step 3: Install the CLI on the admin’s machine¶
What you do
You need Python 3.10 to 3.13, Docker running, and about 30 GB of free disk.
pip install -U "agilerl-arena[snowflake]"
Put the Arena credentials from step 2 in the shell you run the CLI from:
export ARENA_ORG_KEY=arena_org_...
export ARENA_EXTERNAL_USER_ID=your-user-id
What happens
The trainer image is about 13 GB compressed. Setup pulls it to this machine and pushes it into your account, which is why it needs Docker and the disk space. During setup this machine talks to three hosts: your Snowflake account, Arena, and public.ecr.aws. Your Snowflake account never pulls from a public registry.
arena snowflake commands on your machine use ARENA_ORG_KEY and ARENA_EXTERNAL_USER_ID only when both are set. Without them, setup asks for the key and user ID, and submit checks the licence with your stored arena login, which may belong to a different organization. Set the variables before you run setup or submit.
Step 4: Run setup¶
What you do
To read the SQL first, print it. This connects to nothing and creates nothing:
arena snowflake setup --print-sql
Then run setup:
arena snowflake setup
Setup asks, in order:
The first time only, your account identifier and Snowflake login name. It then opens a browser. Sign in as a user with
ACCOUNTADMIN.The Arena org key and external user ID from step 2, unless
ARENA_ORG_KEYandARENA_EXTERNAL_USER_IDare set or you pass--org-keyand--external-user-id.The warehouse credit cap per month (default 100) and the compute pool credit budget per month (default 500).
An email for budget alerts. Press Enter to skip.
It prints everything it will create and asks once. Answer yes. Setup is done when it prints The account is ready.
Setup creates one GPU pool per GPU family. Without --gpu-family it keeps the families already set up, or picks the smallest GPU family in the region the first time. To choose families, repeat the flag:
arena snowflake setup --gpu-family GPU_NV_S --gpu-family GPU_NV_M --gpu-family GPU_NV_L
--gpu-family replaces the saved family list. Pass every family you want each time, not only the one you are adding. --max-gpus N sets how many GPUs each GPU pool can hold. The default is the saved cap, or two nodes.
Then check the account as the arena_trainer role:
arena snowflake check
To list the pools later, or change one family’s GPU cap without running setup again (this also creates the pool for a new family):
arena snowflake pools
arena snowflake pools set GPU_NV_M --max-gpus 8
What happens
Before it creates anything, setup checks that:
your user has
ACCOUNTADMIN;Docker is running and the trainer image tag exists on public ECR;
the account is not a trial (it creates and drops a disabled integration);
the key starts with
arena_org_, Arena accepts it, and Snowflake training is on for your organization.
Any failure stops setup with nothing created. After you answer yes, setup:
creates the
arena_trainerrole, thearena_job_userservice user, thearena_whwarehouse with thearena_monitorresource monitor, the compute pools, and a budget for the pools;creates the
arena.trainingschema with the image repository, therunsstage, thearena_metricstable, and the three Arena secrets;creates two external access integrations:
arena_hf_eai(Hugging Face, for the base model) andarena_api_eai(the Arena host only);writes
@arena.training.runs/_arena/setup.json, which every laterarena snowflakecommand reads;pulls the trainer image that matches this CLI release from public ECR by digest and pushes it into
arena.training.images;runs
nvidia-smiin that image on the GPU pool. The driver must be 580.65.06 or newer.
The push takes as long as your upload speed needs for 13 GB.
check confirms that arena_trainer can use the Arena objects and pools, records the newest trainer image in setup.json, and runs the driver check again. If something is missing it tells you to re-run setup.
If the account has no active event table, setup and check warn that arena snowflake logs cannot read finished jobs. To keep those logs, run:
ALTER ACCOUNT SET EVENT_TABLE = SNOWFLAKE.TELEMETRY.EVENTS;
See Admin: set up the account for every flag and the full list of objects.
Step 5: Add data scientists and data¶
What you do
Create each data scientist’s Snowflake user in Snowsight first. Then grant them the Arena role:
arena snowflake invite JANE
Let Arena jobs read each training table. The name must be fully qualified. Repeat --table for more tables:
arena snowflake grant --table ANALYTICS.PUBLIC.SFT_EXAMPLES
If training calls an RL environment outside the account, allow its host too:
arena snowflake grant --host env.example.com
What happens
invite grants arena_trainer to the user and prints the join command to send them. It does not create users. grant --table gives arena_trainer USAGE on the database and schema and SELECT on the table. grant --host creates a third integration, arena_env_eai, for that host on port 443 and adds it to setup.json, so every later run gets it. See Invite data scientists and Let jobs read a table or reach a host.
Step 6: Join the account (data scientist, once)¶
What you do
You need a key pair registered on your Snowflake user, or a programmatic access token (PAT). Passwords are rejected. To make a key pair:
openssl genrsa 2048 | openssl pkcs8 -topk8 -inform PEM -out rsa_key.p8 -nocrypt
openssl rsa -in rsa_key.p8 -pubout -out rsa_key.pub
Send rsa_key.pub to the admin, who puts its body on your user without the BEGIN and END lines:
ALTER USER jane SET RSA_PUBLIC_KEY = 'MIIBIjANBgkqh...';
Then install the CLI and join:
pip install -U "agilerl-arena[snowflake]"
arena snowflake join --account MYORG-MYACCOUNT --user JANE --key rsa_key.p8
Use --token PAT in place of --key to sign in with a token. Without either, join asks token or key. It prints Joined MYORG-MYACCOUNT as JANE. when it works. Add --yes to replace an existing [arena] connection without asking.
What happens
join writes an [arena] connection in ~/.snowflake/connections.toml with role arena_trainer, warehouse arena_wh, and schema arena.training. It then checks that you can use the role and read setup.json. You do not need an Arena login. Every later command reads setup.json from the account, so admin changes reach you without joining again. A PAT has the same network policy rule as in step 9. See Join the account.
Step 7: Train¶
What you do
Write a manifest. This one runs SFT on a table with PROMPT and RESPONSE columns (see A sample table to create one):
algorithm:
name: SFT
environment:
env_type: dataset
objective: sft
columns:
prompt: PROMPT
response: RESPONSE
network:
pretrained_model_name_or_path: Qwen/Qwen3.5-0.8B
max_context_length: 2048
training:
max_steps: 200
Submit saves merged weights at the end of the run, which step 8 needs. Submit it:
arena snowflake submit sft.yaml --gpus 1 \
--source-table ANALYTICS.PUBLIC.SFT_EXAMPLES
submit prints the run ID, such as arena-20261111093000-a1b2c3. Options you are likely to use:
--gpu-family GPU_NV_Mpicks the GPU family. It must be one the admin set up.--source-query "SELECT ..."reads a query in place of a table.--reward-file reward.pyis required for GRPO and other RL over table rows (see GRPO manifest).--resume RUN_IDcontinues from that run’s newest finished checkpoint (see Resume a run).--checkpoint-steps Nsaves a checkpoint every N steps. The default ismax_steps // 10.
Watch the run, read its logs, or stop it:
arena snowflake status RUN_ID
arena snowflake logs RUN_ID
arena snowflake logs RUN_ID --role worker --instance 0
arena snowflake cancel RUN_ID
What happens
submitstops before starting anything if the CLI and the account’s trainer image are from different releases. A newer CLI tells you to re-runarena snowflake setup, which copies the matching image. A newer image tells you to runpip install -U "agilerl-arena[snowflake]".Unless you pass
--gpu-family,submitpicks the family with the least GPU memory that can hold the manifest at that--gpuscount. It stops before starting anything if no family fits, or if the pool cannot hold that many GPUs.submitstarts three job services in your account: a message broker, a head on a CPU pool, and one GPU worker per GPU node.Once every GPU node has joined, the head registers the run with Arena, copies your table to Parquet in the run’s directory on the
runsstage, and starts the trainer.While the run trains, the head checks in with Arena every 5 minutes. If it cannot reach Arena for about 30 minutes, the trainer stops.
Checkpoints go to
@arena.training.runs/<run id>/. Metrics go to thearena.training.arena_metricstable (see Metrics).statusprints the state of the broker, head, and workers and of the run.logsprints the head’s log unless you pass--role brokeror--role worker.When training ends, the broker service is dropped and the head and workers exit.
canceldrops all three at once. The pools suspend after 5 idle minutes.
Apart from downloading the base model from Hugging Face, the only calls that leave the account go to Arena: when the run starts, the check-ins while it trains, and when it ends. They carry the run ID, account name, GPU count, GPU family, and start and end times. No table rows, prompts, metrics, or weights are sent. See What leaves the account.
Snowflake can cancel jobs during its weekend node maintenance windows. Plan long runs around them, or resume. See Maintenance windows.
Step 8: Deploy the model¶
What you do
arena snowflake publish RUN_ID --model-name support_bot --serve
arena snowflake endpoint support_bot
publish waits for the job. That can take 20 minutes or more while the GPU service starts. It then prints the model, version, service, and endpoint. endpoint prints the service’s host name again later.
Pass --step N to publish a specific checkpoint, --version NAME to name the registry version, and --gpu-count N to serve on more than one GPU. Leave out --serve to log the model to the registry without starting a service.
What happens
A job in your account loads the run’s newest complete merged checkpoint and logs it to the Snowflake Model Registry in arena.training. With --serve it then starts a vLLM inference service with a public endpoint. The version defaults to the checkpoint directory, such as checkpoint_200, and the service to support_bot_checkpoint_200. The weights never leave Snowflake. If training is still using the GPU pool, the publish job needs a spare node. See Publish and serve.
Step 9: Call the model¶
What you do
Create a PAT restricted to arena_trainer, which owns the service. Snowflake shows the token once, so copy it:
ALTER USER jane ADD PROGRAMMATIC ACCESS TOKEN support_bot_client
ROLE_RESTRICTION = 'ARENA_TRAINER';
The endpoint speaks the OpenAI chat API:
from openai import OpenAI
client = OpenAI(
base_url="https://<endpoint>/v1",
api_key="unused",
default_headers={"Authorization": 'Snowflake Token="<PAT>"'},
)
reply = client.chat.completions.create(
model="unused",
messages=[{"role": "user", "content": "Hello"}],
)
print(reply.choices[0].message.content)
What happens
Snowflake checks the token, its role, and where the request comes from. Things to know:
By default Snowflake accepts a PAT only from a user that has a network policy, and the policy has to allow the address you call from. See Snowflake’s PAT docs.
A missing or rejected token, a role without access, or a blocked address returns 404, not 401 or 403. Check those before you suspect the endpoint.
The service suspends after about 30 minutes without requests. The next request waits while it starts again.
To let another role call the model, see Call the model.
Step 10: Stop and clean up¶
What you do
Stop serving when you no longer need the endpoint:
arena snowflake unpublish support_bot
To remove Arena from the account, the admin runs:
arena snowflake teardown
What happens
unpublish drops the inference service. Add --version checkpoint_200 --drop-version to drop the registry version too.
teardown lists what it will drop and asks once. It drops the Arena compute pools and their services, the integrations, the arena database, arena_job_user, arena_trainer, arena_wh, and arena_monitor, then removes the local Arena connections and key. The arena database holds the checkpoints, metrics, and published models, so copy out anything you want to keep first. See Tear down.
Keep the CLI up to date¶
When AgileRL ships a newer agilerl-arena, commands that talk to Arena print one line on stderr, once per run of the CLI:
agilerl-arena 1.10.0 is available (you have 1.9.4). Run: pip install -U "agilerl-arena[snowflake]"
The command still works. When your version is older than the oldest one Arena supports, Arena refuses the command and the CLI prints:
agilerl-arena 1.8.0 is no longer supported; the minimum is 1.9.0. Run: pip install -U agilerl-arena
Run pip install -U "agilerl-arena[snowflake]" to keep the Snowflake extras, then re-run arena snowflake setup so the account’s trainer image matches the new CLI. A run that starts from an image Arena no longer supports fails with a message that ends then run arena snowflake setup to update the trainer image. Runs already training keep checking in and finish normally.
Billing¶
AgileRL bills GPU hours: the number of GPUs in the run times the time from the first GPU worker start to the end of the run. If a run stops without reporting its end, billing stops at its last check-in. Snowflake bills your account for the compute pools, the warehouse, and storage as usual. See Billing.
If a command fails, Common errors lists the usual messages.
All commands¶
Who |
Command |
What it does |
|---|---|---|
Admin |
|
Install or upgrade the CLI |
Admin |
|
Create the Arena objects and copy the trainer image |
Admin |
|
Check the objects as |
Admin |
|
List pools, or change one family’s GPU cap |
Admin |
|
Grant |
Admin |
|
Let jobs read a table or reach a host |
Data scientist |
|
Write the |
Data scientist |
|
Start a training run |
Data scientist |
|
Show the run and its services |
Data scientist |
|
Print a service’s log |
Data scientist |
|
Stop the run |
Data scientist |
|
Log a checkpoint to the Model Registry and serve it |
Data scientist |
|
Print the inference endpoint |
Data scientist |
|
Drop the inference service |
Admin |
|
Drop every Arena object in the account |