Access to all dimensional output

[zyc123456zzz]

Hi, Miles,

Sorry to bother you. I struggled with the customized loss_func, but I am not sure why I can not access the full prediction results in the loss function.

Basically, for the training dataset: input data shape is (7000, 14), output data shape is (7000, 4). Then I want to do a classification task. The output data is like: [0,0,1,0]. I want to customize my own loss function, it is basically calculating the L2 distance.

loss function

loss_func = """
  dict = Dict(
      1 => [19.0, 2.0],
      2 => [20.0, 2.0],
      3 => [21.0, 2.0],
      4 => [32.0, 19.0],
  )
  function eval_loss(tree, dataset::Dataset{T,L}, options)::L where {T,L}
      prediction, flag = eval_tree_array(tree, dataset.X, options)
      if !flag
          return L(Inf)
      end
      
      index_max = Tuple.(argmax(prediction, dims=2))
      last_index = last.(index_max)
  
      target_max = Tuple.(argmax(dataset.y, dims=2))
      target_index = last.(target_max)
  
      # Convert prediction argmax to dictionary values
      y_pred = [dict[i] for i in last_index]
  
      # Convert ground-truth one-hot to class indices
      y_true = [dict[i] for i in target_index]
      mse = mean([sum((yp .- yt).^2) for (yp, yt) in zip(y_pred, y_true)])
  
      return mse
  end
"""

However, I find that the prediction data and dataset.y here are all shape (7000). Here I want to fully access all the input data whose shape should be (7000, 4).
I am not sure how to access the whole training data instead of one dimensional data. Hope to get your response!

[MilesCranmer]

Quick comment: I would put dict's definition inside the function; otherwise it will be very slow b/c Julia can't optimise it into the function.

For multi-dimensional data, the right way is to use a TemplateExpressionSpec, and put each of your four features into X, rather than in y. For example, like this:

spec = TemplateExpressionSpec(
    expressions=["f1", "f2", "f3", "shared"],
    variable_names=["x1", "x2", "x3", "y1", "y2", "y3"],
    combine="""
        v = shared(x1, x2, x3)
        y1_predicted = v + f1(x1, x2, x3)
        y2_predicted = v + f2(x1, x2, x3)
        y3_predicted = v + f3(x1, x2, x3)
        
        residuals = (
            abs2(y1 - y1_predicted) +
            abs2(y2 - y2_predicted) +
            abs2(y3 - y3_predicted)
        )
        
        residuals
"""
)

And you would just pass y as a vector of zeros

[MilesCranmer]

@zyc123456zzz I have now added an example to the docs! https://ai.damtp.cam.ac.uk/pysr/examples/#13-vector-valued-expressions - check it out

[zyc123456zzz]

Thank you very much!!! I give it a try now.