[Feature]: Custom loss function implementation in python

[xnerhu]

Feature Request

Hello. I would like to have a feature, where I can pass custom function loss implemented in python instead of in julia. I'm not able to implement it in julia as it part of bigger optimization pipeline. I believe this feature can be implemented by passing python function pointer to julia and executing it from there.

[MilesCranmer]

Maybe you could try PythonCall.jl? https://juliapy.github.io/PythonCall.jl/stable/pythoncall/ It lets you call Python functions from Julia. See https://astroautomata.com/PySR/examples/#7-julia-packages-and-types for an example of using an external Julia package in the loss function.

[xnerhu]

Maybe you could try PythonCall.jl? https://juliapy.github.io/PythonCall.jl/stable/pythoncall/ It lets you call Python functions from Julia. See https://astroautomata.com/PySR/examples/#7-julia-packages-and-types for an example of using an external Julia package in the loss function.

It somewhat worked thanks.

import os


os.environ["PYTHON_JULIACALL_THREADS"] = "1"
from datetime import datetime
import numpy as np
from pysr import PySRRegressor
from pysr import jl

from julia.api import Julia


x = None


def custom_loss(y_true, y_pred):
    # global x
    # print(y_true, y_pred, x)
    # if x is None:
    #     x = np.random.rand()
    return float((y_true - y_pred) ** 2)


jl.seval(
    """
import Pkg
Pkg.add("PythonCall")
"""
)
jl.custom_loss_function = custom_loss

jl.seval(
    """
function custom_loss_wrapper(y_true, y_pred)
    py_obj = PythonCall.pycall(custom_loss_function, y_true, y_pred)
    return PythonCall.pyconvert(Float32, py_obj)
end
"""
)

X = np.random.rand(100, 2)
y = X[:, 0] * X[:, 1] + np.random.rand(100) * 0.1

model = PySRRegressor(
    niterations=40,
    binary_operators=["+", "-", "*", "/"],
    unary_operators=["cos", "exp"],
    elementwise_loss="custom_loss_wrapper",
)

model.fit(X, y)
print(model)
predictions = model.predict(X)
print(predictions)

Though I had to do os.environ["PYTHON_JULIACALL_THREADS"] = "1" from #661

because of

  Resolving package versions...
  No Changes to `C:\Users\xnerhu\.julia\environments\pyjuliapkg\Project.toml`
  No Changes to `C:\Users\xnerhu\.julia\environments\pyjuliapkg\Manifest.toml`
Compiling Julia backend...

Please submit a bug report with steps to reproduce this fault, and any error messages that follow (in their entirety). Thanks.
Exception: EXCEPTION_ACCESS_VIOLATION at 0x7ff85e1587a5 --  at 0x7ff85e1587a5 -- ION with steps to reproduce this fault, and any error messages that follow (in their entirety). Thanks.
Exception: EXCEPTION_ACCESS_VIOLATION at 0x7ff85e163a4d -- PyObject_GC_Malloc at C:\Python310\python310.dll (unknown line)  
nd any error messages that follow (in their entirety). Thanks.
Exception: EXCEPTION_ACCESS_VIOLATION at 0x7ff85e1587a5 --  at 0x7ff85e1587a5 -- IONC:\Python310\python310.dll (unknown lin expression starting at none:0
\python310.dll (unknown line)
in expression starting at none:0
in expression starting at none:0
\python310.dll (unknown line)
ne)
in expression starting at none:0
in expression starting at none:0
\python310.dll (unknown line)
PyEval_EvalFrameDefault at C:\PytPyEval_EvalFrameDefault at C:\Python310\python310.dll (unknown line)
PyFunction_Vectorcall at C:\Python310\python310.dll (unknown line)

PyFunction_Vectorcall at C:\Python310\python310.dll (unknown linyFunction_Vectorcall at C:\Python310\python310.dll (unknown line)
PyVectorcall_Call at C:\Python310\python310.dll (unknown line)
e)
PyVectorcall_Call at C:\Python310\python310.dll (unknown line)
PyVectorcall_Call at C:\Python310\python310.dll (unknown line)
PyObject_Call at efault at C:\Python310\python310.dll (unknown PyObject_Call at C:\Python310\python310.dll (unknown line)   
wn line)
PyObject_Call at C:\Python310\python310.dll (unknown line)
PyFunction_Vectorcall at C:\Python310\python310.dll (unknowPyFunction_Vectorcall at C:\Python310\python310.dll (unknown line)
PyObject_CallObject at C:\Users\xnerhu\.julia\packages\PythonCall\S5MOg\src\C\pointers.jl:297 [inlined]
macro expansion at C:\Users\xnerhu\.julia\packages\PythonCall\S5MOg\src\Core\Py.jl:132 [inlined]
pycallargs at C:\Users\xnerhu\.julia\packages\PythonCall\S5MOg\src\Core\builtins.jl:212
unknown function (ip: 00000138b4d57ddf)
#pycall#21 at C:\Users\xnerhu\.julia\packages\PythonCall\S5MOg\src\Core\builtins.jl:230
#pycall#21 at C:\Users\xnerhu\.julia\packages\PythonCall\S5MOg\#pycall#21 at C:\Users\xnerhu\.julia\packages\PythonCall\S5MOg\src\Core\builtins.jl:230
nlined]
ined]
macro expansion at C:\Users\xnerhu\.julia\packages\PythonCamacro expansion at C:\Users\xnerhu\.julia\packages\PythonCall\S5MOg\src\Core\Py.jl:132 [inlined]
pycallargs at C:\Users\xnerhu\.julia\packages\PythonCall\S5MOg\src\Core\builtins.jl:212
jl_apply at C:/workdir/src\julia.h:1982 [inlined]
do_apply at C:/workdir/src\builtins.c:768
do_apply at C:/workdir/src\builtins.c:76pycall at C:\Users\xnerhu\.julia\packages\PythonCall\S5MOg\src\Core\builtins.jl:220 
pycall#21 at C:\Users\xnerhu\.julia\packages\PythonCall\S5MOg\src\Core\builtins.jl:230
:297 [inlined]
macro expansion at C:\Users\xnerhu\.julia\packages\PythonCall\S5MOg\src\Core\Py.jl:132 [inlined]
pycallargs at C:\Users\xnerhu\.julia\packages\PythonCall\S5MOg\src\Core\builtins.jl:212
unknown function (ip: 00000138b4d57ddf)
ages\PythonCall\S5MOg\src\Core\builtins.jl:212
unknown function (ip: 00000138b4unknown function (ip: 00000138b4d57ddf)
l at C:\Users\xnerhu\.julia\packages\SymbolicRegression\FtJSD\src\LossFunctions.jl:19 [inlined]
#4 at C:\Users\xnerhu\.julia\environments\pyjuliapkg\pyjuliapkg\install\share\julia\stdlib\v1.10\Statistics\src\Statistics.jl:187 [inlined]
mapreduce_impl at .\reduce.jl:262
mapreduce_impl at .\reduce.jl:262
nments\pyjuliapkg\pyjuliapkg\install\share\julia\stdljl_apply at C:/workdir/src\julia.h:1982 [inlined]
do_apply at C:/workdir/src\builtins.c:768
pycall at C:\Users\xnerhu\.julia\packages\PythonCall\S5MOg\src\Core\builtins.jl:220
custom_loss_wrapper at .\none:2
l at C:\Users\xnerhu\.julia\packages\SymbolicRegression\FtJSD\src\LossFunctions.jl:19 [inlined]
#4 at C:\Users\xnerhu\.julia\environments\pyjuliapkg\pyjuliapkg\install\share\julia\stdlib\v1.10\Statistics\src\Statistics.jl:187 [inlined]
mapreduce_impl at .\reduce.jl:262
mapreduce_impl at .\reduce.jl:277 [inlined]
_mapreduce at .\reduce.jl:447
jl_apply at C:/workdir/src\julia.h:1982 [inlined]
do_apply at C:/workdir/src\builtins.c:768
pycall at C:\Users\xnerhu\.julia\packages\PythonCall\S5MOg\src\Core\builtins.jl:220
pycall at C:\Users\xnerhu\.julia\packages\Pytho#mapreduce#821 at .\reducedim.jl:357 [inlined]
mapreduce at .\reducedim.jl:357 [inlined]
#_sum#831 at .\reducedim.jl:1015 [inlined]
_sum at .\reducedim.jl:1015 [inlined]
#sum#829 at .\reducedim.jl:1011 [inlined]
sum at .\reducedim.jl:1011 [inlined]
_mean at C:\Users\xnerhu\.julia\environments\pyjuliapkg\pyjuliapkg\install\share\julia\stdlib\v1.10\Statistics\src\Statistics.jl:187
_mean at C:\Users\xnerhu\.julia\_mean at C:\Users\xnerhu\.julia\environments_mapreduce at .\reduce.jl:447
ia\environments\pyjuliapkg\pyjuliapkg\install\share\julia\stdlib\v1.10\Statistics\src\Statistics.jl:104 [inlined]
mean at C:\Users\xnerhu\.julia\environments\pyjuliapkg\pyjuliapkg\install\share\julia\stdlib\v1.10\Statistics\src\Statistics.jl:104 [inlined]
_loss at C:\Users\xnerhu\.julia\packages\SymbolicRegression\FtJSD\src\LossFunctions.jl:20
unknown function (ip: 00000138b4df92cf)
unknown function (ip: 00000138_mapreduce_dim at .\reducedim.jl:365 [inlined]
#mapreduce#821 at .\reducedim.jl:357 [inlined]
mapreduce at .\reducedim.jl:357 [inlined]
#_sum#831 at .\reducedim.jl:1015 [inlined]
_sum at .\reducedim.jl:1015 [inlined]
#sum#829 at .\reducedim.jl:1011 [inlined]
sum at .\reducedim.jl:1011 [inlined]
_mean at C:\Users\xnerhu\.julia\environments\pyjuliapkg\pyjuliapkg\install\share\julia\stdlib\v1.10\Statistics\src\Statistics.jl:187
_mean at C:\Users\xnerhu\.julia\environments\pyjuliapkg\pyjuliapkg\install\share\julia\stdlib\v1_mean at C:\Users\xnerhu\.julia\environments\pyjuliapkg\pyjuliapkg\install\share\julia\stdlib\v1.10\Statistics\src\Statistics.jl:186
_eval_loss at C:\Users\xnerhu\.julia\packages\SymbolicRegression\FtJSD\src\LossFunctions.jl:67
#mean#1 at C:\Users\xnerhu\.julia\environments\pyjuliapkg\pyjuliapkg\install\share\julia\stdlib\v1.10\Statistics\src\Statistics.jl:104 [inlined]
mean at C:\Users\xnerhu\.julia\environments\pyjuliapkg\pyjuliapkg\install\share\julia\stdlib\v1.10\Statistics\src\Statistics.jl:104 [inlined]
_loss at C:\Users\xnerhu\.julia\packages\SymbolicRegression\FtJSD\src\LossFunctions.jl:20
jl:105
_loss at C:\Users\xnerhu\.julia\packages\SymbolicRegression\FtJSD\src\LossFunctions.jl:20
unknown function (ip: 00000138b4df92cf)
unknown function (ip: 00000138b4df92cf)
SymbolicRegression\FtJSD\src\LossFunctions.jl:20
jl:105
Statistics\src\Statistics.jl:104 [inlined]
]
mean at C:\Users\xnerhu\.julia\environments\pyjuliapkg\pyjuliapkg\install\share\julia\stdlib\v1.10\Statistics\src\Statistics.jl:104 [inlined]
_loss at C:\Users\xnerhu\.julia\packages\SymbolicRegression\FtJSD\src\LossFunctions.jl:20
unknown function (ip: 00000138b4df92cf)
SymbolicRegression\FtJSD\src\LossFunctions.jl:20
b\v1.10\Stati#score_func#5 at C:\Users\xnerhu\.julia\packages\SymbolicRegression\FtJSD\src\LossFunctions.jl:164 [inlined]
score_func at C:\Users\xnerhu\.julia\packages\SymbolicRegression\FtJSD\src\LossFunctions.jl:161 [inlined]
#PopMember#2 at C:\Users\xnerhu\.julia\packages\SymbolicRegression\FtJSD\src\PopMember.jl:99
#PopMember#2 at C:\Users\xnerhu\.julia\p#PopMember#2 at C:\Users\xnerhu\.julia\packages\SymbolicRegression\FtJSD\src\PopMember.jl:99
1#eval_loss#3 at C:\Users\xnerhu\.julia\packages\SymbolicRegression\FtJSD\src\LossFunctions.jl:105
_eval_loss at C:\Users\xnerhu\.julia\packages\SymbolicRegression\FtJSD\src\LossFunctions.jl:67
eval_loss at C:\Users\xnerhu\.julia\packages\SymbolicRegression\FtJSD\src\LossFunctions.jl:97 [inlined]
#score_func#5 at C:\Users\xnerhu\.julia\packages\SymbolicRegression\FtJSD\src\LossFunctions.jl:164 [inlined]
score_func at C:\Users\xnerhu\.julia\packages\SymbolicRegression\FtJSD\src\LossFunctions.jl:161 [inlined]
#PopMember#2 at C:\Users\xnerhu\.julia\packages\SymbolicRegression\FtJSD\src\PopMember.jl:99
#PopMember#2 at C:\Users\xnerhu\.julia\packages\SymbolicRegression\FtJSD\src\PopMember.jl:99
1 [ieval_loss at C:\Users\xnerhu\.julia\packages\SymbolicRegression\FtJSD\src\LossFunctions.jl:97 [inlinPopMember at C:\Users\xnerhu\.julia\packages\SymbolicRegression\FtJSD\src\PopMember.jl:88 [inlined]
#2 at .\none:0 [inlined]
iterate at .\generator.jl:47 [inlined]
ages\SymbolicRegression\FtJSD\src\PopMember.jl:88 [inlined]
PopMember at C:\Users\xnerhu\.julia\paccollect at .\array.jl:834
collect at .\array.jl:834
u\.julia\packages\SymbolicRegression\FtJSD\src\PopMember.jl:88 [inlined]
#2 at .\none:0 [inlined]
iterate at .\generator.jl:47 [inlined]
collect at .\array.jl:834
#Population#1 at C:\Users\xnerhu\.julia\packages\SymbolicRegression\FtJSD\src\Population.jl:49 [inlined]#score_func#5 at C:\Users\xnerhu\.julia\packages\SymbolicRegression\FtJSD\src\LossFunctions.jl:164 [inlined]
score_func at C:\Users\xnerhu\.julia\packages\SymbolicRegression\FtJSD\src\LossFunctions.jl:161 [inlined]
#PopMember#2 at C:\Users\xnerhu\.julia\packages\SymbolicRegression\FtJSD\src\PopMember.jl:99
#PopMember#2 at C:\Users\xnerhu\.julia\packages\SymbolicRegression\FtJSD\src\PopMember.jl:99
1 [inlined]Population at C:\Users\xnerhu\.julia\packages\SymbolicRegression\FtJSD\src\Population.jl:35 [inlined]
macro expansion at C:\Users\xnerhu\.julia\packages\SymbolicRegression\FtJSD\src\SymbolicRegression.jl:765 [inlined]
#54 at C:\Users\xnerhu\.julia\packages\SymbolicRegression\FtJSD\src\SearchUtils.jl:116
#54 at C:\Users\xnerhu\.julia\packages\SymbolicRegression\FtJSD\src\SearchUtils.jl:116
8 [inlined]
PopMember at C:\Users\xnerhu\.julia\packages\SymbolicRegression\FtJSD\src\PopMember.jl:88 [inlined]
#2 at .\none:0 [inlined]
iterate at .\generator.jl:47 [inlined]
collect at .\array.jl:834
collect at .\array.jl:834
7 [inlined]
ages\SymbolicRegression\FtJSD\src\PopMember.jl:88 [inlined]
:765Population at C:\Users\xnerhu\.julia\packages\SymbolicRegression\FtJSD\src\Population.jl:35 [inlined]
macro expansion at C:\Users\xnerhu\.julia\packages\SymbolicRegression\FtJSD\src\SymbolicRegression.jl:765 [inlined]
#54 at C:\Users\xnerhu\.julia\packages\SymbolicRegression\FtJSD\src\SearchUtils.jl:116
jfptr_YY.54_21945 at C:\Users\xnerhu\.julia\compiled\v1.10\SymbolicRegression\X2eIS_JE3Mg.dll (unknown line)
#Population#1 at C:\Users\xnerhu\.julia\packages\SymbolicRegression\FtJSD\src\Population.jl:49 [inlined]
Population at C:\Users\xnerhu\.julia\packages\SymbolicRegression\FtJSD\src\Population.jl:35 [inlined]
macro expansion at C:\Users\xnerhu\.julia\packages\SymbolicRegression\FtJSD\src\SymbolicRegression.jl:765 [inlined]
#54 at C:\Users\xnerhu\.julia\packages\SymbolicRegression\FtJSD\src\SearchUtils.jl:116
#54 at C:\Users\xnerhu\.julia\packages\SymbolicRegression\FtJSD\src\SearchUtils.jl:116
Regression.jl:765 [injl_apply at C:/workdir/src\julia.h:1982 [inlined]
start_task at C:/workdir/src\task.c:1238
Allocations: 17550813 (Pool: 17526745; Big: 24068); GC: 21
jfptr_YY.54_21945 at C:\Users\xnerhu\.julia\compiled\v1.10\SymbolicRegression\X2eIS_JE3Mg.dll (unknown line)
jl_apply at C:/workdir/src\julia.h:1982 [inlined]
start_task at C:/workdir/src\task.c:1238
Allocations: 17550813 (Pool: 17526745; Big: 24068); GC: 21

also, how can I do loss_function intead of elemenetwise_loss? I think seval needs to have explicit types in args

juliacall.JuliaError: MethodError: no method matching custom_loss_wrapper(::Node{Float32}, ::Dataset{Float32, Float32, Matrix{Float32}, Vector{Float32}, Nothing, @NamedTuple{}, Nothing, Nothing, Nothing, Nothing}, ::Options{Int64, DynamicExpressions.OperatorEnumModule.OperatorEnum, Node, false, false, nothing, StatsBase.Weights{Float64, Float64, Vector{Float64}}})

[xnerhu]

btw is it possible to do multivaritive prediction (multiple outpouts)?