Hyperparameter Optimization Parameters Generator - Enumerate Grid Search Configs

Question

## Round 1 - Coding

## Problem

Implement a hyperparameter grid search generator. Given a dictionary mapping parameter names to lists of possible values, generate all possible combinations in a deterministic order.

```python
def generate_configs(
    param_grid: dict[str, list]
) -> list[dict]:
    # Returns all combinations of hyperparameter values
    # Order: iterate last key fastest (lexicographic of keys)
    ...
```

```
Example:
param_grid = {
    "lr": [0.01, 0.1],
    "batch_size": [32, 64],
    "dropout": [0.1]
}
generate_configs(param_grid) -> [
    {"lr": 0.01, "batch_size": 32, "dropout": 0.1},
    {"lr": 0.01, "batch_size": 64, "dropout": 0.1},
    {"lr": 0.1,  "batch_size": 32, "dropout": 0.1},
    {"lr": 0.1,  "batch_size": 64, "dropout": 0.1},
]
```

**Extension:** Implement early stopping - accept a `budget: int` and return only the first `budget` configs from a random shuffle.

```python
def generate_random_configs(
    param_grid: dict[str, list],
    budget: int,
    seed: int = 42
) -> list[dict]: ...
```

## Follow-ups

1. How would you implement this as a generator (lazy evaluation) to handle huge grids?
2. What is the total config count given the example above? How do you compute it without generating all configs?
3. How does random search compare to grid search statistically?
4. How would you add support for conditional parameters (e.g. `num_layers` only applies if `model_type == "deep"`)?

Hyperparameter Optimization Parameters Generator - Enumerate Grid Search Configs

Question Details

Full Details

Topics

More from Pinterest