Monday, 10 July 2023

Optimizing with non-negative constraints

Consider the following functions

import numpy as np
import scipy.optimize as opt
import math

# Periodic indexation
def pl(list, i):
    return list[i % len(list)]

# Main function (index j)
def RT(list, j, L):
    return sum((math.e**(-sum((k-abs(i))*pl(list, j+i)/v for i in range(-k, k+1)))-math.e**(-sum((k+1-abs(i))*pl(list, j+i)/v for i in range(-k, k+1))))/(sum(pl(list, j+i) for i in range(-k, k+1))+ 1e-10) for k in range(L+1))

# Vector function
def fp(list):
    return [RT(list, j, math.floor(len(list)/st)) for j in range(len(list))]

for 1<=j<=len(list)$, where L = np.floor(len(list)/st) and v = 7. We generally take st = 2. Given data ltest, I would like to find the best approximation to the non-negative periodic array list so that |ltest - fp(list)| is minimized (or ltest ≃ fp(list). A mathematical expression for RT can be written as

enter image description here

where {x} = list. For a general list, gradient descent is relatively efficient. However, I have been struggling to find the best possible approximation provided the elements in list are strictly non-negative. Any ideas?

Here is my attempt. Take ltest to be this data, for example. A first guess can be given by

# Initial guess
x0 = [(math.pi/(4*v))*i**(-2) for i in ltest]

I will then use the L-BFGS-B method available from scipy, as follows

# Function to compute the difference between ltest and the list generated by RT
def fun(params, *args):
    list = params
    ltest0 = args[0]
    mse = np.sum((np.array([RT(list, j, math.floor(len(list)/st)) for j in range(len(list))]) - ltest0) ** 2)
    return mse

# Create non-negative bounds
bounds = opt.Bounds(0, np.inf)  # All parameters should be between 0 and infinity

# Call the constrained optimization algorithm
res = opt.minimize(fun, x0, args=(ltest,), method='L-BFGS-B', bounds=bounds)

# The optimized list is stored in res.x
opt_list = res.x

While this is suppose to converge, I find it extremely slow, and far from the solutions that consider any-valued entries of list. Any ideas on how to improve this? While the number of parameters is len(list) (=1000, in the example), perhaps L could be reduced by increasing st.



from Optimizing with non-negative constraints

No comments:

Post a Comment