Module Fehu_algorithmsSource

Available Algorithms

Policy Gradient Methods

• Reinforce: Monte Carlo Policy Gradient (REINFORCE)

Value-Based Methods

• Dqn: Deep Q-Network (DQN)

Usage Pattern

All algorithms follow this pattern:

  open Fehu

  (* 1. Create policy network *)
  let policy_net = Kaun.Layer.sequential [...] in

  (* 2. Initialize algorithm *)
  let agent = Algorithm.create
    ~policy_network:policy_net
    ~n_actions:n
    ~rng:(Rune.Rng.key 42)
    Algorithm.default_config
  in

  (* 3. Train *)
  let agent = Algorithm.learn agent ~env ~total_timesteps:100_000 () in

  (* 4. Use trained policy *)
  let action = Algorithm.predict agent obs ~training:false |> fst

Choosing an Algorithm

• REINFORCE: Simple policy gradient, works for small discrete action spaces, requires complete episodes. Good for learning but sample inefficient.
• DQN: Off-policy value-based method with experience replay, good for discrete actions, more sample efficient than REINFORCE.

Future algorithms:

• PPO: More sample efficient, supports continuous actions, industry standard
• SAC: Off-policy actor-critic, excellent for continuous control