Wrong internal version of `TAxis` when parsing TH2F histogram

[MarcoRiggirello]

I can't parse correctly some TH2F histograms like the one in this file:

th2f.root.gz

julia> using UnROOT

julia> f = ROOTFile("th2f.root")
ROOTFile with 1 entry and 15 streamers.
th2f.root
└─ myHisto (TH2F)


julia> f["myHisto"]
ERROR: UndefVarError: `TAxis_0` not defined in `UnROOT`
Suggestion: check for spelling errors or missing imports.
Stacktrace:
 [1] stream!(io::IOBuffer, fields::Dict{Symbol, Any}, ::Type{UnROOT.TAxis}; check::Bool)
   @ UnROOT ~/.julia/packages/UnROOT/4c5To/src/streamers.jl:590
 [2] stream!
   @ ~/.julia/packages/UnROOT/4c5To/src/streamers.jl:586 [inlined]
 [3] TH(io::UnROOT.MmapStream, tkey::UnROOT.TKey32, refs::Dict{Int32, Any})
   @ UnROOT ~/.julia/packages/UnROOT/4c5To/src/bootstrap.jl:987
 [4] TH2F(io::UnROOT.MmapStream, tkey::UnROOT.TKey32, refs::Dict{Int32, Any})
   @ UnROOT ~/.julia/packages/UnROOT/4c5To/src/bootstrap.jl:949
 [5] _getindex(f::ROOTFile, s::String)
   @ UnROOT ~/.julia/packages/UnROOT/4c5To/src/root.jl:178
 [6] #150
   @ ~/.julia/packages/UnROOT/4c5To/src/root.jl:163 [inlined]
 [7] get!(default::UnROOT.var"#150#151"{ROOTFile, String}, h::Dict{Any, Any}, key::String)
   @ Base ./dict.jl:458
 [8] getindex(f::ROOTFile, s::String)
   @ UnROOT ~/.julia/packages/UnROOT/4c5To/src/root.jl:162
 [9] top-level scope
   @ REPL[3]:1

However, if I check the streamer TAxis version via UnROOT it says that it is v4. . .

julia> UnROOT.streamerfor(f, "TAxis")
UnROOT.StreamerInfo(UnROOT.TStreamerInfo{UnROOT.TObjArray}("TAxis", "", 0x5a496e70, 10, UnROOT.TObjArray("", 0, Any[UnROOT.TStreamerBase
  version: UInt16 0x0004
  fOffset: Int64 0
  fName: String "TNamed"
  fTitle: String "The basis for a named object (name, title)"
  fType: Int32 67
  fSize: Int32 0
  fArrayLength: Int32 0
  fArrayDim: Int32 0
  fMaxIndex: Array{Int32}((5,)) Int32[0, -541636036, 0, 0, 0]
  fTypeName: String "BASE"
  fXmin: Float64 0.0
  fXmax: Float64 0.0
  fFactor: Float64 0.0
  fBaseVersion: Int32 1
, UnROOT.TStreamerBase
  version: UInt16 0x0004
  fOffset: Int64 0
  fName: String "TAttAxis"
  fTitle: String "Axis attributes"
  fType: Int32 0
  fSize: Int32 0
  fArrayLength: Int32 0
  fArrayDim: Int32 0
  fMaxIndex: Array{Int32}((5,)) Int32[0, 1550843710, 0, 0, 0]
  fTypeName: String "BASE"
  fXmin: Float64 0.0
  fXmax: Float64 0.0
  fFactor: Float64 0.0
  fBaseVersion: Int32 4
, UnROOT.TStreamerBasicType
  version: UInt16 0x0004
  fOffset: Int64 0
  fName: String "fNbins"
  fTitle: String "Number of bins"
  fType: Int32 3
  fSize: Int64 4
  fArrayLength: Int32 0
  fArrayDim: Int32 0
  fMaxIndex: Array{Int32}((5,)) Int32[0, 0, 0, 0, 0]
  fTypeName: String "int"
  fXmin: Float64 0.0
  fXmax: Float64 0.0
  fFactor: Float64 0.0
, UnROOT.TStreamerBasicType
  version: UInt16 0x0004
  fOffset: Int64 0
  fName: String "fXmin"
  fTitle: String "Low edge of first bin"
  fType: Int32 8
  fSize: Int64 8
  fArrayLength: Int32 0
  fArrayDim: Int32 0
  fMaxIndex: Array{Int32}((5,)) Int32[0, 0, 0, 0, 0]
  fTypeName: String "double"
  fXmin: Float64 0.0
  fXmax: Float64 0.0
  fFactor: Float64 0.0
, UnROOT.TStreamerBasicType
  version: UInt16 0x0004
  fOffset: Int64 0
  fName: String "fXmax"
  fTitle: String "Upper edge of last bin"
  fType: Int32 8
  fSize: Int64 8
  fArrayLength: Int32 0
  fArrayDim: Int32 0
  fMaxIndex: Array{Int32}((5,)) Int32[0, 0, 0, 0, 0]
  fTypeName: String "double"
  fXmin: Float64 0.0
  fXmax: Float64 0.0
  fFactor: Float64 0.0
, UnROOT.TStreamerObjectAny
  version: UInt16 0x0004
  fOffset: Int64 0
  fName: String "fXbins"
  fTitle: String "Bin edges array in X"
  fType: Int32 62
  fSize: Int32 24
  fArrayLength: Int32 0
  fArrayDim: Int32 0
  fMaxIndex: Array{Int32}((5,)) Int32[0, 0, 0, 0, 0]
  fTypeName: String "TArrayD"
  fXmin: Float64 0.0
  fXmax: Float64 0.0
  fFactor: Float64 0.0
, UnROOT.TStreamerBasicType
  version: UInt16 0x0004
  fOffset: Int64 0
  fName: String "fFirst"
  fTitle: String "First bin to display"
  fType: Int32 3
  fSize: Int64 4
  fArrayLength: Int32 0
  fArrayDim: Int32 0
  fMaxIndex: Array{Int32}((5,)) Int32[0, 0, 0, 0, 0]
  fTypeName: String "int"
  fXmin: Float64 0.0
  fXmax: Float64 0.0
  fFactor: Float64 0.0
, UnROOT.TStreamerBasicType
  version: UInt16 0x0004
  fOffset: Int64 0
  fName: String "fLast"
  fTitle: String "Last bin to display"
  fType: Int32 3
  fSize: Int64 4
  fArrayLength: Int32 0
  fArrayDim: Int32 0
  fMaxIndex: Array{Int32}((5,)) Int32[0, 0, 0, 0, 0]
  fTypeName: String "int"
  fXmin: Float64 0.0
  fXmax: Float64 0.0
  fFactor: Float64 0.0
, UnROOT.TStreamerBasicType
  version: UInt16 0x0004
  fOffset: Int64 0
  fName: String "fBits2"
  fTitle: String "Second bit status word"
  fType: Int32 12
  fSize: Int64 2
  fArrayLength: Int32 0
  fArrayDim: Int32 0
  fMaxIndex: Array{Int32}((5,)) Int32[0, 0, 0, 0, 0]
  fTypeName: String "unsigned short"
  fXmin: Float64 0.0
  fXmax: Float64 0.0
  fFactor: Float64 0.0
, UnROOT.TStreamerBasicType
  version: UInt16 0x0004
  fOffset: Int64 0
  fName: String "fTimeDisplay"
  fTitle: String "On/off displaying time values instead of numerics"
  fType: Int32 18
  fSize: Int64 1
  fArrayLength: Int32 0
  fArrayDim: Int32 0
  fMaxIndex: Array{Int32}((5,)) Int32[0, 0, 0, 0, 0]
  fTypeName: String "bool"
  fXmin: Float64 0.0
  fXmax: Float64 0.0
  fFactor: Float64 0.0
, UnROOT.TStreamerString
  version: UInt16 0x0004
  fOffset: Int64 0
  fName: String "fTimeFormat"
  fTitle: String "Date&time format, ex: 09/12/99 12:34:00"
  fType: Int32 65
  fSize: Int32 24
  fArrayLength: Int32 0
  fArrayDim: Int32 0
  fMaxIndex: Array{Int32}((5,)) Int32[0, 0, 0, 0, 0]
  fTypeName: String "TString"
  fXmin: Float64 0.0
  fXmax: Float64 0.0
  fFactor: Float64 0.0
, UnROOT.TStreamerObjectPointer
  version: UInt16 0x0004
  fOffset: Int64 0
  fName: String "fLabels"
  fTitle: String "List of labels"
  fType: Int32 64
  fSize: Int32 8
  fArrayLength: Int32 0
  fArrayDim: Int32 0
  fMaxIndex: Array{Int32}((5,)) Int32[0, 0, 0, 0, 0]
  fTypeName: String "THashList*"
  fXmin: Float64 0.0
  fXmax: Float64 0.0
  fFactor: Float64 0.0
, UnROOT.TStreamerObjectPointer
  version: UInt16 0x0004
  fOffset: Int64 0
  fName: String "fModLabs"
  fTitle: String "List of modified labels"
  fType: Int32 64
  fSize: Int32 8
  fArrayLength: Int32 0
  fArrayDim: Int32 0
  fMaxIndex: Array{Int32}((5,)) Int32[0, 0, 0, 0, 0]
  fTypeName: String "TList*"
  fXmin: Float64 0.0
  fXmax: Float64 0.0
  fFactor: Float64 0.0
])), Set(Any["TNamed", "TAttAxis"]))

. . . while ROOT says it's v10

root [1] _file0->GetStreamerInfoList()->ls()
[...]
StreamerInfo for class: TAxis, version=10, checksum=0x5a496e70
  TNamed         BASE            offset=  0 type=67 The basis for a named object (name, title)
  TAttAxis       BASE            offset=  0 type= 0 Axis attributes     
  int            fNbins          offset=  0 type= 3 Number of bins      
  double         fXmin           offset=  0 type= 8 Low edge of first bin
  double         fXmax           offset=  0 type= 8 Upper edge of last bin
  TArrayD        fXbins          offset=  0 type=62 Bin edges array in X
  int            fFirst          offset=  0 type= 3 First bin to display
  int            fLast           offset=  0 type= 3 Last bin to display 
  unsigned short fBits2          offset=  0 type=12 Second bit status word
  bool           fTimeDisplay    offset=  0 type=18 On/off displaying time values instead of numerics
  TString        fTimeFormat     offset=  0 type=65 Date&time format, ex: 09/12/99 12:34:00
  THashList*     fLabels         offset=  0 type=64 List of labels      
  TList*         fModLabs        offset=  0 type=64 List of modified labels

[tamasgal]

Thanks Marco, I will have a look. Apparently there is some wrong parsing logic which results in shifted offsets...

[Moelf]

https://mattermost.web.cern.ch/files/9u9ahbe68jfpmkyqz4em1kiour/public?h=Evjp1D7OB2_bd32ZpHH7wIvDbq55SwYmWGgjJMG_-_s

one more problematic file

[tamasgal]

Another problematic one, which btw. was produced by using hadd. Also cannot be read by uproot. Maybe @jpivarski has some idea. Unfortunately I still did not have time to look at it but will try now.

test.root.zip

[tamasgal]

Apparently, hadding two files works but adding three or more produces a file which cannot be read. Here is one where 3 files were added. The tarball below contains the three files which can be read individually and the hadded file which cannot be read:

root_hadd_TH2D.tar.gz

Example with uproot

>>> uproot.open("detector_06.root")['detector'].axis(0).labels()
['6']
>>> uproot.open("detector_01.root")['detector'].axis(0).labels()
['1']
>>> uproot.open("detector_03.root")['detector'].axis(0).labels()
['3']
>>> uproot.open("detector_merged.root")['detector'].axis(0).labels()
>>>