LLMize Methods

class llmize.methods.OPRO(problem_text=None, obj_func=None, llm_model=None, api_key=None)

Bases: Optimizer

No-index:

OPRO (Optimization by PROmpting) optimizer for numerical optimization using LLMs.

OPRO is the original approach that directly prompts LLMs to generate better solutions based on previous examples. It works by showing the LLM a history of solutions and their scores, then asking it to generate new solutions that improve upon the best ones.

This optimizer is best suited for: - Simple optimization problems with clear patterns - Problems where the relationship between solutions and scores is easily learnable - Quick prototyping and testing - Problems with relatively small search spaces

Example

>>> def sphere(x):
...     return sum(float(i)**2 for i in x)
>>>
>>> opro = OPRO(
...     problem_text="Minimize the sphere function sum(x_i^2)",
...     obj_func=sphere,
...     api_key="your-api-key"
... )
>>> result = opro.minimize(
...     init_samples=[["1", "1"], ["2", "2"]],
...     init_scores=[2, 8],
...     num_steps=10
... )

Note

This class inherits from Optimizer and uses the default configuration parameters unless overridden during initialization or optimization.

__init__(problem_text=None, obj_func=None, llm_model=None, api_key=None)

Initialize the OPRO optimizer.

Parameters:

• problem_text (str, optional) – Natural language description of the optimization problem. This should clearly state what needs to be optimized and any constraints.

• obj_func (callable, optional) – Objective function that takes a solution and returns a numerical score. Higher scores indicate better solutions for maximization, lower scores for minimization.

• llm_model (str, optional) – Name of the LLM model to use. If None, uses the default from configuration file.

• api_key (str, optional) – API key for the LLM service. If None, will attempt to read from environment variables.

meta_prompt(batch_size, example_pairs, optimization_type='maximize')

Generate a prompt for the LLM model to generate new solutions. Parameters: - batch_size (int): Number of new solutions to generate. - example_pairs (str): Example solutions and scores. - optimization_type (str): “maximize” or “minimize” (default: “maximize”).

Returns: - text: A formatted prompt structure.

optimize(init_samples=None, init_scores=None, num_steps=None, batch_size=None, temperature=None, callbacks=None, verbose=1, optimization_type='maximize', parallel_n_jobs=None)

Run the OPRO optimization algorithm.

Parameters: - init_samples (list): A list of initial solutions. - init_scores (list): A list of initial scores corresponding to init_samples. - num_steps (int): The number of optimization steps (default from config). - batch_size (int): The number of new solutions to generate at each step (default from config). - temperature (float): The temperature for the LLM model (default from config). - callbacks (list): A list of callback functions to be triggered at the end of each step. - optimization_type (str): “maximize” or “minimize” (default: “maximize”). - parallel_n_jobs (int): Number of parallel jobs for evaluation (default from config).

Returns: - results (OptimizationResult): An object containing the optimization results.

class llmize.methods.ADOPRO(problem_text=None, obj_func=None, llm_model=None, api_key=None)

Bases: Optimizer

No-index:

ADOPRO (Adaptive Optimization by PROmpting) optimizer for numerical optimization using LLMs.

ADOPRO is an enhanced version of OPRO that dynamically adjusts prompts based on optimization progress. It monitors the optimization trajectory and adapts the prompting strategy to escape local optima and improve convergence.

This optimizer is best suited for: - Complex optimization landscapes with multiple local optima - Problems where OPRO gets stuck or converges prematurely - Adaptive optimization strategies - Problems requiring dynamic exploration-exploitation balance

Example

>>> def rastrigin(x):
...     A = 10
...     n = len(x)
...     return A*n + sum(float(i)**2 - A*math.cos(2*math.pi*float(i)) for i in x)
>>>
>>> adopro = ADOPRO(
...     problem_text="Minimize the Rastrigin function (highly multimodal)",
...     obj_func=rastrigin,
...     api_key="your-api-key"
... )
>>> result = adopro.minimize(
...     init_samples=[["1", "1"], ["2", "2"], ["0", "0"]],
...     init_scores=[...],
...     num_steps=20
... )

Note

This class inherits from Optimizer and uses adaptive prompting strategies to improve upon the basic OPRO approach.

__init__(problem_text=None, obj_func=None, llm_model=None, api_key=None)

Initialize the ADOPRO optimizer.

Parameters:

• problem_text (str, optional) – Natural language description of the optimization problem. Should include information about the complexity and any known challenges (e.g., multimodality).

• obj_func (callable, optional) – Objective function that takes a solution and returns a numerical score.

• llm_model (str, optional) – Name of the LLM model to use. If None, uses the default from configuration file.

• api_key (str, optional) – API key for the LLM service. If None, will attempt to read from environment variables.

meta_prompt(batch_size, example_pairs, optimization_type='maximize')

Generate a prompt for the LLM model to generate new solutions. Parameters: - batch_size (int): Number of new solutions to generate. - example_pairs (str): Example solutions and scores. - optimization_type (str): “maximize” or “minimize” (default: “maximize”).

Returns: - text: A formatted prompt structure.

optimize(init_samples=None, init_scores=None, num_steps=None, batch_size=None, temperature=None, callbacks=None, verbose=1, optimization_type='maximize', parallel_n_jobs=None)

Run the ADOPRO optimization algorithm.

Parameters: - init_samples (list): A list of initial solutions. - init_scores (list): A list of initial scores corresponding to init_samples. - num_steps (int): The number of optimization steps (default: 50). - batch_size (int): The number of new solutions to generate at each step (default: 5). - temperature (float): The temperature for the LLM model (default: 1.0). - callbacks (list): A list of callback functions to be triggered at the end of each step. - optimization_type (str): “maximize” or “minimize” (default: “maximize”).

Returns: - results (OptimizationResult): An object containing the optimization results.

class llmize.methods.HLMEA(problem_text=None, obj_func=None, llm_model=None, api_key=None)

Bases: Optimizer

No-index:

HLMEA (Hyper-heuristic LLM-driven Evolutionary Algorithm) optimizer for numerical optimization.

HLMEA combines evolutionary algorithm principles with LLM guidance to maintain solution diversity and avoid premature convergence. It uses hyper-heuristics to adaptively select evolutionary operators (selection, crossover, mutation) based on the optimization progress.

This optimizer is best suited for: - Combinatorial optimization problems (e.g., TSP, scheduling) - Large search spaces where diversity is crucial - Problems requiring exploration of multiple solution regions - Complex optimization landscapes with many local optima

Example

>>> def tsp_distance(tour):
...     # Calculate total distance for TSP tour
...     return total_distance
>>>
>>> hlmea = HLMEA(
...     problem_text="Solve the Traveling Salesman Problem - find shortest tour",
...     obj_func=tsp_distance,
...     api_key="your-api-key"
... )
>>> result = hlmea.minimize(
...     init_samples=[["A", "B", "C", "D"], ["A", "C", "B", "D"]],
...     init_scores=[100, 120],
...     num_steps=50,
...     batch_size=10
... )

Note

HLMEA uses sophisticated evolutionary strategies guided by LLM prompts to maintain diversity and avoid premature convergence.

__init__(problem_text=None, obj_func=None, llm_model=None, api_key=None)

Initialize the HLMEA optimizer.

Parameters:

• problem_text (str, optional) – Natural language description of the optimization problem. For combinatorial problems, specify the problem type and constraints.

• obj_func (callable, optional) – Objective function that takes a solution and returns a numerical score.

• llm_model (str, optional) – Name of the LLM model to use. If None, uses the default from configuration file.

• api_key (str, optional) – API key for the LLM service. If None, will attempt to read from environment variables.

meta_prompt(batch_size, example_pairs, optimization_type='maximize', hp_text='The solutions below are generated randomly.')

Generate a prompt for the LLM model to generate new solutions. Parameters: - batch_size (int): Number of new solutions to generate. - example_pairs (str): Example solutions and scores. - optimization_type (str): “maximize” or “minimize” (default: “maximize”).

Returns: - text: A formatted prompt structure.

optimize(init_samples=None, init_scores=None, num_steps=None, batch_size=None, temperature=None, callbacks=None, verbose=1, optimization_type='maximize', parallel_n_jobs=None)

Run the HLMEA optimization algorithm.

Parameters: - init_samples (list): A list of initial solutions. - init_scores (list): A list of initial scores corresponding to init_samples. - num_steps (int): The number of optimization steps (default: 50). - batch_size (int): The number of new solutions to generate at each step (default: 5). - temperature (float): The temperature for the LLM model (default: 1.0). - callbacks (list): A list of callback functions to be triggered at the end of each step. - optimization_type (str): “maximize” or “minimize” (default: “maximize”).

Returns: - results (OptimizationResult): An object containing the optimization results.

class llmize.methods.HLMSA(problem_text=None, obj_func=None, llm_model=None, api_key=None)

Bases: Optimizer

No-index:

HLMSA (Hyper-heuristic LLM-driven Simulated Annealing) optimizer for numerical optimization.

HLMSA combines simulated annealing principles with LLM optimization to provide controlled exploration of the solution space. It uses adaptive cooling rates and perturbation strategies to balance exploration and exploitation.

This optimizer is best suited for: - Problems with many local optima requiring careful exploration - Fine-tuning solutions where small improvements matter - Temperature-sensitive optimization problems - Problems where controlled convergence is important

Example

>>> def multimodal(x):
...     # Function with many local optima
...     return math.sin(5*x) + math.cos(3*x) + x**2
>>>
>>> hlmsa = HLMSA(
...     problem_text="Find global minimum of multimodal function",
...     obj_func=multimodal,
...     api_key="your-api-key"
... )
>>> result = hlmsa.minimize(
...     init_samples=[["0"], ["1"], ["-1"]],
...     init_scores=[...],
...     num_steps=30,
...     batch_size=5
... )

Note

HLMSA uses simulated annealing principles guided by LLM prompts to dynamically adjust cooling rates and perturbation strategies.

__init__(problem_text=None, obj_func=None, llm_model=None, api_key=None)

Initialize the HLMSA optimizer.

Parameters:

• problem_text (str, optional) – Natural language description of the optimization problem. For multimodal problems, mention the presence of local optima.

• obj_func (callable, optional) – Objective function that takes a solution and returns a numerical score.

• llm_model (str, optional) – Name of the LLM model to use. If None, uses the default from configuration file.

• api_key (str, optional) – API key for the LLM service. If None, will attempt to read from environment variables.

meta_prompt(batch_size, example_pairs, optimization_type='maximize', hp_text='The solutions below are generated randomly.')

Generate a prompt for the LLM model to generate new solutions using Simulated Annealing. Parameters: - batch_size (int): Number of new solutions to generate. - example_pairs (str): Example solutions and scores. - optimization_type (str): “maximize” or “minimize” (default: “maximize”).

Returns: - text: A formatted prompt structure.

optimize(init_samples=None, init_scores=None, num_steps=None, batch_size=None, temperature=None, callbacks=None, verbose=1, optimization_type='maximize', parallel_n_jobs=None)

Run the HLMSA optimization algorithm.

Parameters: - init_samples (list): A list of initial solutions. - init_scores (list): A list of initial scores corresponding to init_samples. - num_steps (int): The number of optimization steps (default: 50). - batch_size (int): The number of new solutions to generate at each step (default: 5). - temperature (float): The temperature for the LLM model (default: 1.0). - callbacks (list): A list of callback functions to be triggered at the end of each step. - optimization_type (str): “maximize” or “minimize” (default: “maximize”).

Returns: - results (OptimizationResult): An object containing the optimization results.

Submodules

• llmize.methods.opro module
  • OPRO
• llmize.methods.adopro module
  • ADOPRO
• llmize.methods.hlmea module
  • HLMEA
• llmize.methods.hlmsa module
  • HLMSA