optfunc

optfunc provides differentiable PyTorch optfuncs, CVXPY-backed convex benchmarks, and pytest-oriented helpers for evaluating first-order and second-order optimizers.

The package is published as optfuncs and imported as optfunc.

Detailed user guides live under docs/.

For CPU-only use:

pip install optfuncs

For convex-only CVXPY benchmarks:

pip install "optfuncs[convex]"

For CPU-only use plus convex benchmarks:

pip install "optfuncs[torch-cpu,convex]"

With uv in this repository:

uv sync --group torch-cpu

To enable the CVXPY convex benchmark suite:

uv sync --extra convex

Supported extras:

Torch hardware variants are also available as local uv dependency groups. Only choose one Torch group at a time:

The CUDA 13 convex development environment is:

uv sync --group torch-cu130 --group cupy-cu13 --extra convex --dev

Detailed CPU/CUDA wheel behavior is documented in docs/native-wheel-variants.md.

When another uv project wants to expose multiple optional dependency sets, add the convex/base benchmark support and the Torch backend support separately. This keeps the target project's extras explicit and lets users choose the Torch variant independently. optfuncs[convex] does not install or require Torch; add a torch-* extra only when the target project also uses unconstrained PyTorch optfuncs.

Recommended target-project extra layout:

Example:

uv add --optional test "optfuncs[convex]>=0.0.7"
uv add --optional torch "optfuncs[torch-cu130]>=0.0.7"

For a CPU project, use:

uv add --optional test "optfuncs[convex]>=0.0.7"
uv add --optional torch "optfuncs[torch-cpu]>=0.0.7"

Then install the combination you need from the target project:

uv sync --extra test --extra torch

Keep the target project's Torch source/index selection aligned with the selected optfuncs[...] hardware extra.

Helper scripts:

# bash/zsh on Linux or macOS, and Git Bash/MSYS/Cygwin on Windows
./scripts/sync_torch_variant.sh cpu
./scripts/sync_torch_variant.sh cu130
./scripts/sync_torch_variant.sh rocm
./scripts/sync_torch_variant.sh xpu

# Windows PowerShell or PowerShell 7 on Windows/Linux/macOS
.\scripts\sync_torch_variant.ps1 cpu
.\scripts\sync_torch_variant.ps1 cu130
.\scripts\sync_torch_variant.ps1 rocm
.\scripts\sync_torch_variant.ps1 xpu

If Windows blocks local PowerShell scripts, run:

powershell -ExecutionPolicy Bypass -File .\scripts\sync_torch_variant.ps1 cu130

Platform notes:

• CPU works on Linux, macOS, and Windows.
• CUDA extras work on Linux and Windows.
• ROCm works on Linux.
• XPU works on Linux and Windows.
• macOS should use torch-cpu in this project configuration.

The backend indexes follow the official uv PyTorch guide and PyTorch install selector:

• https://docs.astral.sh/uv/guides/integration/pytorch/
• https://pytorch.org/get-started/locally/

The repository provides these built-in optfuncs:

Use a class directly when you know which function you want:

import torch
from optfunc import Sphere

opt_func = Sphere(dim=8, dtype=torch.float64)
x = torch.zeros(8, dtype=torch.float64)

value = opt_func(x)
grad = opt_func.grad(x)
hessian = opt_func.hessian(x)
hvp = opt_func.hvp(x, torch.ones_like(x))
x_star = opt_func.global_minimizer()
distance = opt_func.distance_to_optimum(x)

Use OptFuncRegistry when a test should select an optfunc by name:

from optfunc import OptFuncRegistry

opt_func = OptFuncRegistry.create("rosenbrock", dim=4)
print(OptFuncRegistry.available())

Use BenchmarkRegistry when the caller may switch between unconstrained functions and one concrete convex benchmark instance:

from optfunc import BenchmarkRegistry

convex_problem = BenchmarkRegistry.create(
    "gw_maxcut",
    constraints="convex",
    dim=8,
    seed=19,
)

optimal_value = convex_problem.meta.optimal_value
X_star = convex_problem.known_solution("X")
cpp_data = convex_problem.problem_data
cvxpy_problem = convex_problem.problem  # lazily constructed when requested

Use ConvexFamily when you want a family of parameterised convex programs:

from optfunc import ConvexFamily

family = ConvexFamily.create("gw_maxcut", dim=8)

theta = family.sample_parameters(seed=0)
problem = family.build_instance(theta)

theta_sequence = family.sample_parameters(5, seed=0)
problem_sequence = family.build_instance(theta_sequence)
same_problem_sequence = family.generate_sequence(5, seed=0)

Each optfunc uses the same conventions:

• input x is a 1-D PyTorch tensor with shape (dim,);
• output is a scalar tensor;
• grad, hessian, and hvp use PyTorch autograd unless a subclass provides a better implementation;
• global_minimizer() returns a known theoretical minimizer when available;
• distance_to_optimum(x) defaults to Euclidean distance to global_minimizer();
• project_to_bounds(x) clamps a point into the optfunc's documented search box.

The repository also provides CVXPY convex benchmarks behind constraints="convex":

Each convex benchmark family is generated from ConvexFamily and exposes ProblemFamily methods:

• sample_parameters(seed=0): sample one theta dictionary;
• sample_parameters(n, seed=0): sample a uniform theta sequence;
• sample_theta_*: family-specific sampling methods with the same single-or-many convention;
• build_instance(theta_or_theta_sequence): turn theta into one cpp-native problem instance or a list of instances;
• perturb(theta, n=None, magnitude=..., seed=0): produce one or more nearby theta dictionaries;
• perturb_*: family-specific perturbation methods that match the corresponding sample_theta_* naming;
• generate_sequence(n, seed=0): build a cpp-native problem sequence from the default seed theta sampler.

Each concrete CppNativeProblem exposes:

• problem.to_cvxpy_problem() / problem.problem: the CVXPY problem converted from the native description;
• problem.data: deterministic random instance data used to build the problem;
• problem.problem_data: Clarabel-style canonical data P, q, A, b, and ordered cone blocks;
• problem.cpp_native / problem.to_problem_data(): the native conic data used by cpp solver integrations;
• problem.solve(...): solve with CVXPY, defaulting to MOSEK and falling back to SCS if MOSEK is unavailable in the current environment;
• problem.known_solution() and problem.distance_to_optimum() when the theoretical solution is encoded in the benchmark definition.

The internal conic data follows Clarabel's A x + s = b, s in K convention. Cone blocks are stored in the order zero, nonnegative, second_order, psd, then exponential. Use optfunc.cvxs.sort_cones, add_cone_block, delete_cone_block, and to_ptf_text when building or exporting solver-specific formats.

The gw_maxcut benchmark is a seeded Goemans-Williamson Max-Cut SDP relaxation on a complete weighted bipartite graph. The seed fixes both the planted cut and the positive edge weights. Its SDP relaxation optimum is known without calling a solver: problem.meta.optimal_value is the total planted cut weight and problem.known_solution("X") is the rank-one planted cut matrix.

If a convex benchmark does not have a closed-form theoretical solution yet, you can leave that solution unspecified and use problem.solve() as the reference value/solution path. The default reference solver is MOSEK.

The sequence API uses numpy.random.Generator for reproducible parameter sampling, so ConvexFamily native problem generation and CVXPY conversion work with the convex extra. Torch is required only for constraints="none" PyTorch optfuncs.

Detailed Clarabel and gw_maxcut benchmark examples are available in docs/convex-benchmark-usage.md.

External conic solvers can use the cone descriptor and operator packages without using CVXPY benchmark families. The minimum runtime path is NumPy plus the compiled optfunc.cvxs.cones.cpp extension; convex, SciPy, CVXPY, Torch, and CuPy are only needed when the downstream project explicitly uses those features.

Recommended imports:

import numpy as np

from optfunc.cvxs.cones import NonnegativeCone, PsdCone, make_cone_operator
from optfunc.cvxs.cones.cpp import NonnegativeOperatorCpp

cone = NonnegativeCone(3, name="ineq")
operator = make_cone_operator(cone, backend="cpp")

x = np.asarray([-1.0, 2.0, -3.0], dtype=np.float64)
projected = np.empty_like(x)
operator.project_into(x, projected)

assert projected.tolist() == [0.0, 2.0, 0.0]
assert operator.to_descriptor(name=cone.name) == cone
assert NonnegativeOperatorCpp(3).rows == cone.rows

The descriptor classes ZeroCone, NonnegativeCone, SecondOrderCone, PsdCone, and ExponentialCone describe Clarabel-style cone metadata only: kind, dim, rows, and optional name. The C++ operator classes implement operations on cone vectors: project, project_into, contains, violation, basis, unit_vector_into, project_batch_into, and workspace/fused helpers.

Downstream solvers may also pass their own descriptor objects to make_cone_operator when they expose compatible attributes:

from dataclasses import dataclass


@dataclass(frozen=True)
class SolverCone:
    kind: str
    dim: int
    rows: int
    name: str | None = None


operator = make_cone_operator(
    SolverCone(kind="psd", dim=2, rows=3, name="sdp_block"),
    backend="cpp",
)

Use optfunc.cvxs.native.ConicProblemData and the P, q, A, b builder helpers only when you want optfuncs to assemble Clarabel-style problem data. Those utilities require SciPy sparse matrices. CVXPY-backed benchmark families and translators require the convex extra.

Detailed installed-package guidance for external solver integration is available in docs/cpp-cone-operators.md.

The optimizer evaluation API lives in optfunc.testing. A standard test file has three parts:

1. Define or import an optimizer function.
2. Configure one or more OptimizerCase objects.
3. Assign make_optimizer_tests(...) to a pytest-visible name such as test_my_optimizer.

Run the file with:

uv run pytest tests/test_my_optimizer.py -q --tb=short --optfunc-report

Each OptimizerCase becomes an independent pytest item. If one case fails or raises an exception, pytest continues running the remaining cases. --optfunc-report prints a final serial summary with function gap, distance to the theoretical optimum, gradient norm, Hessian information, step count, and error messages. Do not combine --optfunc-report with pytest-xdist -n in v1.

OptimizerBudget describes the budget passed to the optimizer.

from optfunc.testing import OptimizerBudget

budget = OptimizerBudget(max_steps=500, lr=0.05)

Fields:

• max_steps: positive integer iteration limit.
• lr: positive learning-rate-like scalar. The test harness does not enforce how the optimizer uses it; your optimizer reads it from problem.budget.lr.

For torch.optim.Adam, a typical use is:

optimizer = torch.optim.Adam([x], lr=problem.budget.lr)
for step in range(problem.budget.max_steps):
    ...

ConvergenceTolerances decides whether a finished optimizer run passes.

from optfunc.testing import ConvergenceTolerances

tolerances = ConvergenceTolerances(
    value_gap=1e-8,
    x_distance=1e-4,
    grad_norm=1e-4,
    hessian_min_eig=0.0,
)

Fields:

• value_gap: maximum allowed absolute gap between final value and the known theoretical optimum value. Set to None to skip this check.
• x_distance: maximum allowed distance from final x to the known theoretical minimizer. Set to None to skip this check.
• grad_norm: maximum allowed Euclidean norm of the final gradient. Set to None to skip this check.
• hessian_min_eig: optional lower bound on the final Hessian's smallest eigenvalue. Set to None to skip this check.

The report still computes available metrics even when a tolerance is None; None only disables that pass/fail check.

OptimizerCase describes one pytest item.

from optfunc.testing import OptimizerCase
from optfunc.testing import OptimizerBudget, ConvergenceTolerances

case = OptimizerCase(
  opt_func="sphere",
  constraints="none",
  dim=8,
  budget=OptimizerBudget(max_steps=350, lr=0.05),
  tolerances=ConvergenceTolerances(value_gap=1e-8, x_distance=1e-4, grad_norm=1e-4),
  start="near_minimizer",
  start_radius=0.5,
  seed=0,
  hessian_max_dim=32,
)

Important fields:

• opt_func: registry name such as "sphere" or an already-created TorchOptFunction instance.
• constraints: "none" for the existing Torch optfunc suite, or "convex" for the CVXPY convex benchmark suite.
• dim: required when opt_func is a string.
• batch_size: for constraints="convex", generate this many same-family problem instances.
• perturb_magnitude: for convex batches, controls successive parameter perturbations through ProblemFamily.perturb(...); when it is zero, the harness uses ProblemFamily.sample_parameters(n, seed=...).
• budget: OptimizerBudget passed to the optimizer through problem.budget.
• tolerances: ConvergenceTolerances used after the optimizer returns.
• case_id: optional pytest id; by default this is like sphere[8].
• x0: optional explicit initial point. If omitted, the harness builds one from start.
• start: "near_minimizer", "random", or "zeros".
• start_radius: offset size used by "near_minimizer".
• seed: random seed used by "random".
• device: optional torch device string, for example "cuda" or "cpu".
• dtype: torch dtype, default torch.float64.
• hessian_max_dim: largest dimension for dense Hessian report metrics. Larger cases skip dense Hessian metrics to avoid slow tests.

For constraints="convex", the optimizer receives a ConvexOptimizationProblem with .instances, .parameters, and .problem for single-instance cases. If the optimizer solves the CVXPY instances in place and returns None, the harness checks objective gaps and known-solution distances when those references are available.

make_optimizer_tests converts an optimizer plus cases into a pytest test function.

from optfunc.testing import make_optimizer_tests

test_my_optimizer = make_optimizer_tests(
    optimizer=my_optimizer,
    cases=[case1, case2],
    name="test_my_optimizer",
)

Rules:

• Assign the returned function to a module-level variable whose name starts with test_, otherwise pytest will not collect it.
• For constraints="none", optimizer must accept one OptimizationProblem argument.
• For constraints="none", the optimizer may return either a final torch.Tensor or an OptimizerResult.
• Every case becomes an independent parametrized pytest item.

For convex cases, the optimizer accepts ConvexOptimizationProblem instead:

from optfunc.testing import (
  ConvergenceTolerances,
  ConvexOptimizationProblem,
  OptimizerCase,
  make_optimizer_tests,
)


def cvxpy_solver(problem: ConvexOptimizationProblem):
  for instance in problem:
    instance.solve()


test_convex_batch = make_optimizer_tests(
  optimizer=cvxpy_solver,
  cases=[
    OptimizerCase(
      opt_func="gw_maxcut",
      constraints="convex",
      dim=8,
      batch_size=4,
      perturb_magnitude=10.0,
      seed=0,
      tolerances=ConvergenceTolerances(value_gap=1e-5, x_distance=1e-3, grad_norm=None),
    )
  ],
)

This example shows the recommended shape for a user-owned optimizer wrapper. The test harness does not hide torch.optim.Adam; the user function decides how to initialize Adam, how to use the budget, and what history to expose.

Create tests/test_torch_adam.py:

import torch

from optfunc.testing import (
  ConvergenceTolerances,
  OptimizerBudget,
  OptimizerCase,
  OptimizerResult,
  make_optimizer_tests,
  optimizer_adapter,
)


def scalar_item(value):
  return float(value.detach().cpu().item())


@optimizer_adapter(name="torch_adam")
def torch_adam(problem):
  x = problem.x0.detach().clone().requires_grad_(True)
  optimizer = torch.optim.Adam([x], lr=problem.budget.lr)
  history = []

  for step in range(1, problem.budget.max_steps + 1):
    optimizer.zero_grad(set_to_none=True)
    loss = problem.value(x)
    loss.backward()
    optimizer.step()

    with torch.no_grad():
      x.copy_(problem.project_to_bounds(x.detach()))

    if step % 25 == 0 or step == problem.budget.max_steps:
      x_now = x.detach()
      grad = problem.grad(x_now)
      history.append(
        {
          "step": step,
          "value": scalar_item(problem.value(x_now)),
          "grad_norm": scalar_item(torch.linalg.vector_norm(grad)),
          "x_norm": scalar_item(torch.linalg.vector_norm(x_now)),
        }
      )

  return OptimizerResult(
    final_x=x.detach().clone(),
    steps=problem.budget.max_steps,
    history=history,
  )


test_torch_adam = make_optimizer_tests(
  optimizer=torch_adam,
  cases=[
    OptimizerCase(
      opt_func="sphere",
      dim=8,
      budget=OptimizerBudget(max_steps=350, lr=0.05),
      tolerances=ConvergenceTolerances(
        value_gap=1e-8,
        x_distance=1e-4,
        grad_norm=1e-4,
      ),
      start="near_minimizer",
      start_radius=0.5,
    ),
    OptimizerCase(
      opt_func="rosenbrock",
      dim=4,
      budget=OptimizerBudget(max_steps=1200, lr=0.02),
      tolerances=ConvergenceTolerances(
        value_gap=5e-4,
        x_distance=5e-2,
        grad_norm=1e-2,
      ),
      start="near_minimizer",
      start_radius=0.25,
    ),
  ],
)

Run:

uv run pytest tests/test_torch_adam.py -q --tb=short --optfunc-report

The OptimizationProblem object passed into torch_adam exposes:

• problem.opt_func: the selected TorchOptFunction;
• problem.x0: the initial point for this case;
• problem.budget: the OptimizerBudget;
• problem.value(x): scalar objective value;
• problem.grad(x): gradient;
• problem.value_and_grad(x): objective and gradient;
• problem.hessian(x): dense Hessian;
• problem.hvp(x, v): Hessian-vector product;
• problem.project_to_bounds(x): clamp into the optfunc's bounds.

For second-order methods, call problem.hessian(x) or problem.hvp(x, v) inside the same optimizer wrapper and return the same OptimizerResult shape.

For smoke tests or examples, optfunc.testing.make_torch_adam() provides the same Adam loop as a convenience:

from optfunc.testing import make_torch_adam, make_optimizer_tests

test_adam = make_optimizer_tests(
    optimizer=make_torch_adam(),
    cases=[...],
)

For production optimizer tests, prefer writing the small wrapper yourself so the learning rate, projection, stopping rule, and history format are explicit in your test file.

Developer workflows, release steps, PyPI verification, project structure, and API design notes live in README.dev.md.

Optfunc definitions are adapted from SFU's optimization benchmark collection:

https://www.sfu.ca/~ssurjano/optimization.html