Skip to content

yax #157

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Draft
wants to merge 2 commits into
base: main
Choose a base branch
from
Draft

yax #157

wants to merge 2 commits into from

Conversation

lazarusA
Copy link
Member

No description provided.

@lazarusA
Copy link
Member Author

Currently is failing with

ERROR: MethodError: no method matching !(::Float64)
The function `!` exists, but no method is defined for this combination of argument types.

Closest candidates are:
  !(::Static.True)
   @ Static ~/.julia/packages/Static/SeEGr/src/Static.jl:517
  !(::Bool)
   @ Base bool.jl:35
  !(::Missing)
   @ Base missing.jl:101
  ...

Stacktrace:
  [1] (::XFunction{typeof(!), XOutput{Tuple{}, Int64}, Tuple{}})(x::Float64)
    @ YAXArrays.Xmap ~/.julia/packages/YAXArrays/cl9qV/src/DAT/xmap.jl:318
  [2] apply_function(f::DiskArrayEngine.ElementFunction{…}, xout::Tuple{…}, xin::Tuple{…})
    @ DiskArrayEngine ~/.julia/packages/DiskArrayEngine/ZCmJW/src/userfuncs.jl:85
  [3] innercode(cI::CartesianIndex{…}, f::DiskArrayEngine.ElementFunction{…}, xin::Tuple{…}, xout::Tuple{…})
    @ DiskArrayEngine ~/.julia/packages/DiskArrayEngine/ZCmJW/src/runner.jl:35
  [4] run_block_single(loopRanges::Tuple{…}, f::DiskArrayEngine.ElementFunction{…}, inbuffers::Tuple{…}, outbuffers::Tuple{…})
    @ DiskArrayEngine ~/.julia/packages/DiskArrayEngine/ZCmJW/src/runner.jl:50
  [5] run_block
    @ ~/.julia/packages/DiskArrayEngine/ZCmJW/src/runner.jl:41 [inlined]
  [6] run_block
    @ ~/.julia/packages/DiskArrayEngine/ZCmJW/src/runner.jl:80 [inlined]
  [7] default_loopbody(inow::Tuple{…}, op::DiskArrayEngine.GMDWop{…}, inbuffers_pure::Tuple{…}, outbuffers::Tuple{…}, threaded::Bool, outars::Tuple{…}, cb::Tuple{…}, runfilter::Tuple{}, piddir::Nothing)
    @ DiskArrayEngine ~/.julia/packages/DiskArrayEngine/ZCmJW/src/runner.jl:176
  [8] run_loop(::DiskArrayEngine.LocalRunner, op::DiskArrayEngine.GMDWop{…}, inbuffers_pure::Tuple{…}, outbuffers::Tuple{…}, threaded::Bool, outars::Tuple{…}, loopranges::DiskArrayEngine.ProductArray{…}, cb::Tuple{…}, runfilter::Tuple{}; groupspecs::Nothing)
    @ DiskArrayEngine ~/.julia/packages/DiskArrayEngine/ZCmJW/src/runner.jl:186
  [9] run_loop(runner::DiskArrayEngine.LocalRunner, loopranges::DiskArrayEngine.ProductArray{…}; groupspecs::Nothing)
    @ DiskArrayEngine ~/.julia/packages/DiskArrayEngine/ZCmJW/src/runner.jl:166
 [10] run_loop(runner::DiskArrayEngine.LocalRunner, loopranges::DiskArrayEngine.ProductArray{Tuple{…}, 2, Tuple{…}})
    @ DiskArrayEngine ~/.julia/packages/DiskArrayEngine/ZCmJW/src/runner.jl:165
 [11] readblock!(::DiskArrayEngine.GMWOPResult{…}, ::Matrix{…}, ::UnitRange{…}, ::Vararg{…})
    @ DiskArrayEngine ~/.julia/packages/DiskArrayEngine/ZCmJW/src/diskarrayresults.jl:52
 [12] readblock_checked!
    @ ~/.julia/packages/DiskArrays/WgDSb/src/indexing.jl:294 [inlined]
 [13] getindex_disk_nobatch!(out::Vector{…}, a::DiskArrayEngine.GMWOPResult{…}, i::Tuple{…})
    @ DiskArrays ~/.julia/packages/DiskArrays/WgDSb/src/indexing.jl:110
 [14] getindex_disk!
    @ ~/.julia/packages/DiskArrays/WgDSb/src/indexing.jl:69 [inlined]
 [15] readblock!
    @ ~/.julia/packages/DiskArrays/WgDSb/src/subarray.jl:34 [inlined]
 [16] readblock_checked!
    @ ~/.julia/packages/DiskArrays/WgDSb/src/indexing.jl:294 [inlined]
 [17] getindex_disk_nobatch!(out::Nothing, a::DiskArrays.SubDiskArray{…}, i::Tuple{…})
    @ DiskArrays ~/.julia/packages/DiskArrays/WgDSb/src/indexing.jl:106
 [18] getindex_disk!
    @ ~/.julia/packages/DiskArrays/WgDSb/src/indexing.jl:69 [inlined]
 [19] getindex_disk
    @ ~/.julia/packages/DiskArrays/WgDSb/src/indexing.jl:58 [inlined]
 [20] getindex
    @ ~/.julia/packages/DiskArrays/WgDSb/src/indexing.jl:312 [inlined]
 [21] _iterate_disk(a::DiskArrays.SubDiskArray{Float64, 1, DiskArrayEngine.GMWOPResult{…}, Tuple{…}, false})
    @ DiskArrays ~/.julia/packages/DiskArrays/WgDSb/src/iterator.jl:104
 [22] iterate
    @ ~/.julia/packages/DiskArrays/WgDSb/src/iterator.jl:112 [inlined]
 [23] iterate
    @ ~/.julia/packages/DimensionalData/hv9KC/src/array/array.jl:107 [inlined]
 [24] checkindex
    @ ./abstractarray.jl:766 [inlined]
 [25] checkbounds
    @ ./abstractarray.jl:689 [inlined]
 [26] checkbounds
    @ ./abstractarray.jl:699 [inlined]
 [27] _getindex
    @ ./multidimensional.jl:914 [inlined]
 [28] getindex
    @ ./abstractarray.jl:1312 [inlined]

BernhardAhrens
BernhardAhrens reviewed Sep 27, 2025
View reviewed changes
projects/RbQ10/Q10_dd.jl

ds_p_f = ds_yax[col=At(forcing_names ∪ predictor_names)]
ds_t = ds_yax[col=At(target_names)]
ds_t_nan = .!isnan.(ds_t) # produces 1×35064 YAXArray{Float32, 2}, not a Bool
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This has to be done with map otherwise it's not a Bool

BernhardAhrens
BernhardAhrens reviewed Sep 27, 2025
View reviewed changes
src/utils/loss_fn.jl
#return mean(abs2, yh .- yt)

# Option 2: Use YAXArray directly but map has to be used
return mean(x -> x, map((a,b)->(a-b)^2, ŷ[y_nan], y[y_nan]))
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also here broadcast does not work - map works or we it an array. Not quite sure how we proceed from here. Should our model give a DimArray back? Not at all a YaxArrays / DimArrays expert ;-)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

in principle all types should work, so YAXArrays should be fine.

BernhardAhrens
BernhardAhrens reviewed Sep 27, 2025
View reviewed changes
projects/RbQ10/Q10_dd.jl
ds_t_nan = .!isnan.(ds_t) # produces 1×35064 YAXArray{Float32, 2}, not a Bool
ds_t_nan = map(x -> !isnan(x), ds_t) # 1×35064 YAXArray{Bool, 2}

ls = EasyHybrid.lossfn(RbQ10, ds_p_f, (ds_t, ds_t_nan), ps, st, LoggingLoss())
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This runs now

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice, yes indeed it seems to be the nan mask. Let's the most generic version so that all input cases work.

BernhardAhrens
BernhardAhrens reviewed Sep 28, 2025
View reviewed changes
src/utils/loss_fn.jl
end
function loss_fn(ŷ, y, y_nan, ::Val{:mse})
return mean(abs2, (ŷ[y_nan] .- y[y_nan]))
# Option 1: Convert to Array and compute MSE
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Option 1 would be converting to an array but what I am not sure is where it would use a view, when it copies and when comes into memory and so on

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@BernhardAhrens BernhardAhrens BernhardAhrens left review comments

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@lazarusA @BernhardAhrens