> ## Documentation Index
> Fetch the complete documentation index at: https://wb-21fd5541-update-serverless-training-api-9.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# Sweep configuration options

> Reference for all sweep configuration keys including method, metric, parameters, early termination, and command.

A sweep configuration consists of nested key-value pairs. Use top-level keys within your sweep configuration to define qualities of your sweep search, such as the parameters to search through ([`parameters`](#parameters) key) and the methodology to search the parameter space ([`method`](#method) key).

The following table lists top-level sweep configuration keys and a brief description. For more information about each key, see the respective sections.

| Top-level keys                        | Description                                                                           |
| ------------------------------------- | ------------------------------------------------------------------------------------- |
| `program`                             | (required) Training script to run                                                     |
| `entity`                              | The entity for this sweep                                                             |
| `project`                             | The project for this sweep                                                            |
| `description`                         | Text description of the sweep                                                         |
| `name`                                | The name of the sweep, displayed in the W\&B UI.                                      |
| [`method`](#method)                   | (required) The search strategy                                                        |
| [`metric`](#metric)                   | The metric to optimize (only used by certain search strategies and stopping criteria) |
| [`parameters`](#parameters)           | (required) Parameter bounds to search                                                 |
| [`early_terminate`](#early_terminate) | Any early stopping criteria                                                           |
| [`command`](#command)                 | Command structure for invoking and passing arguments to the training script           |
| `run_cap`                             | Maximum number of runs for this sweep                                                 |

For more information, see [Define sweep configuration](/models/sweeps/define-sweep-configuration).

## `metric`

Use the `metric` top-level sweep configuration key to specify the name, the goal, and the target metric to optimize.

| Key      | Description                                                                                                                                                                                                                                                                         |
| -------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `name`   | Name of the metric to optimize.                                                                                                                                                                                                                                                     |
| `goal`   | Either `minimize` or `maximize` (default is `minimize`).                                                                                                                                                                                                                            |
| `target` | Goal value for the metric you're optimizing. The sweep doesn't create new runs when a run reaches a target value that you specify. Active agents that have a run executing (when the run reaches the target) wait until the run completes before the agent stops creating new runs. |

## `parameters`

In your YAML file or Python script, specify `parameters` as a top level key. Within the `parameters` key, provide the name of a hyperparameter you want to optimize. Common hyperparameters include learning rate, batch size, epochs, and optimizers. For each hyperparameter you define in your sweep configuration, specify one or more search constraints.

The following table shows supported hyperparameter search constraints. Based on your hyperparameter and use case, use one of the following search constraints to tell your sweep agent where (in the case of a distribution) or what (such as `value` or `values`) to search or use.

| Search constraint | Description                                                                                                                                                      |
| ----------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `values`          | Specifies all valid values for this hyperparameter. Compatible with `grid`.                                                                                      |
| `value`           | Specifies the single valid value for this hyperparameter. Compatible with `grid`.                                                                                |
| `distribution`    | Specify a probability [distribution](#distribution-options-for-random-and-bayesian-search). See the note following this table for information on default values. |
| `probabilities`   | Specify the probability of selecting each element of `values` when using `random`.                                                                               |
| `min`, `max`      | (`int` or `float`) Maximum and minimum values. If `int`, for `int_uniform`-distributed hyperparameters. If `float`, for `uniform`-distributed hyperparameters.   |
| `mu`              | (`float`) Mean parameter for `normal`- or `lognormal`-distributed hyperparameters.                                                                               |
| `sigma`           | (`float`) Standard deviation parameter for `normal`- or `lognormal`-distributed hyperparameters.                                                                 |
| `q`               | (`float`) Quantization step size for quantized hyperparameters.                                                                                                  |
| `parameters`      | Nest other parameters inside a root level parameter.                                                                                                             |

<Note>
  If you don't specify a [distribution](#distribution-options-for-random-and-bayesian-search), W\&B sets the following distributions based on these conditions:

  * `categorical` if you specify `values`
  * `int_uniform` if you specify `max` and `min` as integers
  * `uniform` if you specify `max` and `min` as floats
  * `constant` if you specify `value`
</Note>

## `method`

Specify the hyperparameter search strategy with the `method` key. Choose from three hyperparameter search strategies: grid, random, and Bayesian search.

### Grid search

Grid search iterates over every combination of hyperparameter values. Grid search makes uninformed decisions on the set of hyperparameter values to use on each iteration. Grid search can be computationally costly.

Grid search executes forever if it searches within a continuous search space.

### Random search

Random search chooses a random, uninformed set of hyperparameter values on each iteration based on a distribution. Random search runs forever unless you stop the process from the command line, within your Python script, or from [the W\&B App](/models/sweeps/visualize-sweep-results).

If you choose random search (`method: random`), specify the distribution for each hyperparameter in the `parameters` key.

### Bayesian search

In contrast to [random](#random-search) and [grid](#grid-search) search, Bayesian models make informed decisions. Bayesian optimization uses a probabilistic model to decide which values to use. The process iteratively tests values on a surrogate function before evaluating the objective function. Bayesian search works well for small numbers of continuous parameters but scales poorly. For more information about Bayesian search, see the [Bayesian Optimization Primer paper](https://web.archive.org/web/20240209053347/https://static.sigopt.com/b/20a144d208ef255d3b981ce419667ec25d8412e2/static/pdf/SigOpt_Bayesian_Optimization_Primer.pdf).

Bayesian search runs forever unless you stop the process from the command line, within your Python script, or from [the W\&B App](/models/sweeps/visualize-sweep-results).

### Distribution options for random and Bayesian search

Within the `parameters` key, nest the name of the hyperparameter. Next, specify the `distribution` key and specify a distribution for the value.

The following table lists distributions W\&B supports.

| Value for `distribution` key | Description                                                                                                                                                                              |
| ---------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `constant`                   | Constant distribution. Must specify the constant value (`value`) to use.                                                                                                                 |
| `categorical`                | Categorical distribution. Must specify all valid values (`values`) for this hyperparameter.                                                                                              |
| `int_uniform`                | Discrete uniform distribution on integers. Must specify `max` and `min` as integers.                                                                                                     |
| `uniform`                    | Continuous uniform distribution. Must specify `max` and `min` as floats.                                                                                                                 |
| `q_uniform`                  | Quantized uniform distribution. Returns `round(X / q) * q` where `X` is uniform. `q` defaults to `1`.                                                                                    |
| `log_uniform`                | Log-uniform distribution. Returns a value `X` between `exp(min)` and `exp(max)` such that the natural logarithm is uniformly distributed between `min` and `max`.                        |
| `log_uniform_values`         | Log-uniform distribution. Returns a value `X` between `min` and `max` such that `log(X)` is uniformly distributed between `log(min)` and `log(max)`.                                     |
| `q_log_uniform`              | Quantized log uniform. Returns `round(X / q) * q` where `X` is `log_uniform`. `q` defaults to `1`.                                                                                       |
| `q_log_uniform_values`       | Quantized log uniform. Returns `round(X / q) * q` where `X` is `log_uniform_values`. `q` defaults to `1`.                                                                                |
| `inv_log_uniform`            | Inverse log uniform distribution. Returns `X`, where `log(1/X)` is uniformly distributed between `min` and `max`.                                                                        |
| `inv_log_uniform_values`     | Inverse log uniform distribution. Returns `X`, where `log(1/X)` is uniformly distributed between `log(1/max)` and `log(1/min)`.                                                          |
| `normal`                     | Normal distribution. Return value is normally distributed with mean `mu` (default `0`) and standard deviation `sigma` (default `1`).                                                     |
| `q_normal`                   | Quantized normal distribution. Returns `round(X / q) * q` where `X` is `normal`. `q` defaults to `1`.                                                                                    |
| `log_normal`                 | Log normal distribution. Returns a value `X` such that the natural logarithm `log(X)` is normally distributed with mean `mu` (default `0`) and standard deviation `sigma` (default `1`). |
| `q_log_normal`               | Quantized log normal distribution. Returns `round(X / q) * q` where `X` is `log_normal`. `q` defaults to `1`.                                                                            |

## `early_terminate`

Use early termination (`early_terminate`) to stop poorly performing runs. If early termination occurs, W\&B stops the current run before it creates a new run with a new set of hyperparameter values.

<Note>
  You must specify a stopping algorithm if you use `early_terminate`. Nest the `type` key within `early_terminate` within your sweep configuration.
</Note>

### Stopping algorithm

<Note>
  W\&B supports the [Hyperband](https://arxiv.org/abs/1603.06560) stopping algorithm.
</Note>

[Hyperband](https://arxiv.org/abs/1603.06560) hyperparameter optimization evaluates whether a program should stop or continue at one or more pre-set iteration counts, called *brackets*.

When a W\&B run reaches a bracket, the sweep compares that run's metric to all previously reported metric values. The sweep terminates the run if the run's metric value is too high (when the goal is minimization) or if the run's metric is too low (when the goal is maximization).

Brackets are based on the number of logged iterations. The number of brackets corresponds to the number of times you log the metric you're optimizing. The iterations can correspond to steps, epochs, or something in between. Bracket calculations don't use the numerical value of the step counter.

<Note>
  Specify either `min_iter` or `max_iter` to create a bracket schedule.
</Note>

| Key        | Description                                                                                                                   |
| ---------- | ----------------------------------------------------------------------------------------------------------------------------- |
| `min_iter` | Specify the iteration for the first bracket                                                                                   |
| `max_iter` | Specify the maximum number of iterations.                                                                                     |
| `s`        | Specify the total number of brackets (required for `max_iter`)                                                                |
| `eta`      | Specify the bracket multiplier schedule (default: `3`).                                                                       |
| `strict`   | Enable `strict` mode that prunes runs aggressively, more closely following the original Hyperband paper. Defaults to `false`. |

<Note>
  Hyperband checks which [runs](/models/ref/python/experiments/run) to end once every few minutes. The end run timestamp might differ from the specified brackets if your run or iteration is short.
</Note>

## `command`

Use the `command` top-level key to control how the sweep agent invokes your training script and passes arguments to it. Modify the format and contents with nested values within the `command` key. You can directly include fixed components such as filenames.

<Note>
  On Unix systems, `/usr/bin/env` ensures that the OS chooses the correct Python interpreter based on the environment.
</Note>

W\&B supports the following macros for variable components of the command:

| Command macro              | Description                                                                                                                                                           |
| -------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `${env}`                   | `/usr/bin/env` on Unix systems, omitted on Windows.                                                                                                                   |
| `${interpreter}`           | Expands to `python`.                                                                                                                                                  |
| `${program}`               | Training script filename specified by the sweep configuration `program` key.                                                                                          |
| `${args}`                  | Hyperparameters and their values in the form `--param1=value1 --param2=value2`.                                                                                       |
| `${args_no_boolean_flags}` | Hyperparameters and their values in the form `--param1=value1` except boolean parameters are in the form `--boolean_flag_param` when `True` and omitted when `False`. |
| `${args_no_hyphens}`       | Hyperparameters and their values in the form `param1=value1 param2=value2`.                                                                                           |
| `${args_json}`             | Hyperparameters and their values encoded as JSON.                                                                                                                     |
| `${args_json_file}`        | The path to a file containing the hyperparameters and their values encoded as JSON.                                                                                   |
| `${envvar}`                | A way to pass environment variables. `${envvar:MYENVVAR}` expands to the value of the `MYENVVAR` environment variable.                                                |