TTreeIndex fails under Valgrind due to long double emulation

[amete]

Check duplicate issues.

• Checked for duplicates

Description

In ATLAS, we make use of custom TTree indices via TTree::BuildIndex(). Recently, we encountered an issue when running jobs under Valgrind: entries that are accessible under normal execution become unreadable when running the same code under Valgrind.

Upon investigation, it appears that TTreeIndex uses long double for internal calculations. However, Valgrind does not fully emulate 80-bit long double precision on x86_64 systems. It simulates long double as a 64-bit double, losing precision beyond a certain limit.

As a result, when index values fall in the range where double loses precision but long double remains exact, the index lookup fails under Valgrind. This affects our ability to read ROOT files in Valgrind-based memory checks when using such index values.

We understand that this is fundamentally a limitation of Valgrind’s long double emulation, not ROOT itself, but we’re wondering if anything can be done on the ROOT side to work around the issue. Many thanks.

Reproducer

#include <TFile.h>
#include <TTree.h>
#include <TTreeIndex.h>

#include <format>
#include <iostream>

int main() {

  // A large enough index that can be represented by long double but not double
  const Long64_t value = 9007199254743041LL;

  // Write
  {
    TFile file("file.root", "RECREATE");
    TTree tree("tree", "tree");
    Long64_t index{value};
    tree.Branch("index", &index);
    tree.Fill();
    tree.Write();
    file.Close();
  }

  //Read
  {
    TFile file("file.root", "READ");
    TTree* tree = file.Get<TTree>("tree");
    tree->BuildIndex("index");
    auto entry = tree->GetEntryNumberWithIndex(value); 
    std::cout << std::format("Entry: {} for Value: {}", entry, value) << std::endl;
    file.Close();
  }

  return 0;
}

Running this against the head of main (e.g., dev3LCG nightly), one gets:

Entry: 0 for Value: 9007199254743041

under normal execution while it fails under Valgrind with:

Warning in <TTreeIndex::TTreeIndex>: In tree entry 0, major value index=9007199254743040.000000 possibly out of range for internal `long double`
Entry: -1 for Value: 9007199254743041

ROOT version

ROOT Version: 6.37.01
Built for linuxx8664gcc on Aug 05 2025, 22:50:58
From heads/master@v6-37-01-7673-g25c19af92e

Installation method

LCG

Operating system

EL9

Additional context

No response

[ferdymercury]

Is there a way for you to split the index in "major" and "minor"? Like upper and lower bytes Then each of them will be individually smaller and fit in 32 bits, avoiding potential overflows. You might need this recent patch: #14967 (comment)

Something like:

BuildIndex("whatever >> 32", "whatever & 0xffff...")

[ferdymercury]

tree->BuildIndex("index >> 32", "index & 0xFFFFFFFF");
// ...
auto entry = tree->GetEntryNumberWithIndex(value >> 32, value & 0xFFFFFFFF);

[ferdymercury]

it appears that TTreeIndex uses long double for internal calculations

Yes, TTreeIndex is forced to do that because it uses any custom TTreeFormula that the user might pass, which means that the result could be anything.

You could try this patch to circumvent this limitation: #19561

[amete]

Thanks for the prompt response @ferdymercury. Replacing:

tree->BuildIndex("index");
auto entry = tree->GetEntryNumberWithIndex(value);

with

tree->BuildIndex("index >> 32", "index & 0xFFFFFFFF");
auto entry = tree->GetEntryNumberWithIndex(value >> 32, value & 0xFFFFFFFF);

seems to work under normal execution, but it also fails under Valgrind, silently returning -1 without the warning message that the former emits (I tried it in the latest dev3LCG nightly built).

Ideally, we'd like to find a solution that works with existing files, including those read with older ROOT versions still used in production (e.g., 6.32). While it's possible to work around the issue by manually building the index → entry mapping and bypassing TTree::BuildIndex entirely, that approach is obviously less than ideal.

[ferdymercury]

You are welcome.

Could you try with #19561

and then calling tree->BuildIndex("index", "", true); ?

Or that wouldn't work in Valgrind either?

[amete]

I can certainly give that a try when it gets picked up by the dev3LCG nightly.

[amete]

Hi @ferdymercury,

I just tested this locally w/ #19561. The code doesn't seem to accept an empty string for the minor index, i.e.:

tree->BuildIndex("index", "", true);

yields:

Error in <TTreeFormula::Compile>:  Empty String
Error in <TTreeIndex::TreeIndex>: Cannot build the index with major=index, minor=
Entry: -1 for Value: 9007199254743041

but this seems to work:

tree->BuildIndex("index >> 32", "index & 0xFFFFFFFF",true, true);
auto entry = tree->GetEntryNumberWithIndex(value >> 32, value & 0xFFFFFFFF);

both under normal execution, as well as under Valgrind.

PS : Just for fun, this also works:

tree->BuildIndex("index","0",true);

[ferdymercury]

Thanks for the feedback! We have amended the PR so that you can call BuildIndex("index", true); directly.