Skip to content

NetCDF compound types #104

@arjanfetahu

Description

@arjanfetahu

Describe the bug

Hello!
I'm having problems in opening a Sentinel-5P NetCDF4 file downloaded from the https://s5phub.copernicus.eu/dhus/#/home. It is a ~3GB sized file with nested groups i.e:

  • BAND6_RADIANCE
    • STANDARD_MODE
      • GEODATA
      • INSTRUMENT
      • OBSERVATIONS
  • METADATA
  • OBSERVATIONS

To Reproduce

file = "home/user/DATA/S5P_OFFL_L1B_RA_BD6_20190813T101520_20190813T115649_09492_01_010000_20190813T134640.nc"
ds = NCDataset(file,"r")

Expected behavior

I tried it with Python and Panoply and it works as it should, hence is not a data formatting issue.

Environment

  • operating system: Ubuntu 20.04
  • Julia version: [julia 1.5.2, official binaries from https://julialang.org/downloads/
  • NCDatasets version: master

Full output

NCDataset: /home/user/DATA/S5P_OFFL_L1B_RA_BD6_20190813T101520_20190813T115649_09492_01_010000_20190813T134640.nc
Group: /

Global attributes
  Conventions          = CF-1.6
  institution          = KNMI
  library_information  =
/s5p_gpfs02/sw/IPF_S5P_L01b/S5P_IPF_L1B_V01.00.00/lib/libCOREFoundation.so (TROPOMI L01b CORE library) v1.0.0.19183 Build 201803121815 by tropdev@knmi-tropd-l01o.knmi.ssc-campus.nl
/s5p_gpfs02/sw/IPF_S5P_L01b/S5P_IPF_L1B_V01.00.00/lib/libGeo.so (TROPOMI L01b Geolocation library) v1.0.0.19183 Build 201803121817 by tropdev@knmi-tropd-l01o.knmi.ssc-campus.nl
/s5p_gpfs02/sw/IPF_S5P_L01b/S5P_IPF_L1B_V01.00.00/lib/libMath.so (TROPOMI L01b MATH library) v1.0.0.19183 Build 201803121817 by tropdev@knmi-tropd-l01o.knmi.ssc-campus.nl
/mnt/sw/IPF_S5P_L01b/current/lib/TROP_L0InputSubsystem.so (TROPOMI L0 Input Subsystem) v1.0.0.19194 Build 201803131217 by tropdev@knmi-tropd-l01o.knmi.ssc-campus.nl
/mnt/sw/IPF_S5P_L01b/current/lib/TROP_KeyDataSubSystem.so (TROPOMI Key Data Input Subsystem) v1.0.0.19194 Build 201803131218 by tropdev@knmi-tropd-l01o.knmi.ssc-campus.nl
/mnt/sw/IPF_S5P_L01b/current/lib/TROP_L1BOutputSubsystem.so (TROPOMI L1b Output Subsystem) v1.0.0.19194 Build 201803131217 by tropdev@knmi-tropd-l01o.knmi.ssc-campus.nl
/mnt/sw/IPF_S5P_L01b/current/lib/TROP_Algorithms.so (TROPOMI L0-1b Algorithm Library) v1.0.0.19194 Build 201803131218 by tropdev@knmi-tropd-l01o.knmi.ssc-campus.nl
/mnt/sw/IPF_S5P_L01b/current/lib/TROP_L0InputAlgo.so (TROP L0 Input Algorithm Library) v1.0.0.19194 Build 201803131217 by tropdev@knmi-tropd-l01o.knmi.ssc-campus.nl
/mnt/sw/IPF_S5P_L01b/current/lib/TROP_L1BOutputAlgo.so (TROP L1b Output Algorithm Library) v1.0.0.19194 Build 201803131217 by tropdev@knmi-tropd-l01o.knmi.ssc-campus.nl
  orbit                = 9492
  orbit_begin_icid     = 1901
  orbit_end_icid       = 1951
  orbit_type           = O01
  orbit_type_id        = 1
  processor_version    = 1.0.0.19194
  summary              = Radiance product band 6 (NIR detector)
  time_coverage_end    = 2019-08-13T11:35:18Z
  time_coverage_start  = 2019-08-13T10:36:54Z
  time_reference       = 2019-08-13T00:00:00Z
  title                = Sentinel-5p TROPOMI Level 1b Radiance product band 6 (NIR detector)
Groups
  NCDataset: /home/user/DATA/S5P_OFFL_L1B_RA_BD6_20190813T101520_20190813T115649_09492_01_010000_20190813T134640.nc
  Group: BAND6_RADIANCE

  Groups
  NCDataset: /home/user/DATA/S5P_OFFL_L1B_RA_BD6_20190813T101520_20190813T115649_09492_01_010000_20190813T134640.nc
  Group: STANDARD_MODE

  Dimensions
     time = 1
     scanline = 4173
     spectral_channel = 497
     ground_pixel = 448
     ncorner = 4
     nsettings = 5
     nbinningregions = 18

  Groups
  NCDataset: /home/uesr/DATA/S5P_OFFL_L1B_RA_BD6_20190813T101520_20190813T115649_09492_01_010000_20190813T134640.nc
  Group: OBSERVATIONS

  Variables
    scanline     (4173)
      Datatype:    Int32
      Dimensions:  scanline
      Attributes:
       comment              = This dimension variable defines the indices along track; index starts at 0
       long_name            = along track dimension index
       units                = 1

    delta_time     (4173 × 1)
      Datatype:    Int32
      Dimensions:  scanline × time
      Attributes:
       comment              = Time difference with time for each measurement
       _FillValue           = -2147483647
       long_name            = offset from the reference start time of measurement
       units                = milliseconds since 2019-08-13 00:00:00

    viewing_zenith_angle     (448 × 4173 × 1)
      Datatype:    Float32
      Dimensions:  ground_pixel × scanline × time
      Attributes:
       comment              = Zenith angle of the satellite at the ground pixel location on the reference ellipsoid. Angle is measured away from the vertical.
       coordinates          = longitude latitude
       _FillValue           = 9.96921e36
       long_name            = viewing zenith angle
       max_val              = 180.0
       min_val              = 0.0
       units                = degree
       standard_name        = platform_zenith_angle
...
...
...
  NCDataset: /home/user/DATA/S5P_OFFL_L1B_RA_BD6_20190813T101520_20190813T115649_09492_01_010000_20190813T134640.nc
  Group: INSTRUMENT

  Variables
Error showing value of type NCDataset{Nothing}:
ERROR: AssertionError: class == NC_VLEN
Stacktrace:
 [1] nc_inq_var(::Int32, ::Int32) at /home/user/.julia/packages/NCDatasets/t0uQw/src/netcdf_c.jl:1345
 [2] variable(::NCDataset{NCDataset{NCDataset{NCDataset{Nothing}}}}, ::String) at /home/user/.julia/packages/NCDatasets/t0uQw/src/variable.jl:80
 [3] show(::IOContext{REPL.Terminals.TTYTerminal}, ::NCDataset{NCDataset{NCDataset{NCDataset{Nothing}}}}; indent::String) at /home/user/.julia/packages/NCDatasets/t0uQw/src/dataset.jl:477
 [4] show(::IOContext{REPL.Terminals.TTYTerminal}, ::NCDataset{NCDataset{NCDataset{Nothing}}}; indent::String) at /home/user/.julia/packages/NCDatasets/t0uQw/src/dataset.jl:494 (repeats 3 times)
 [5] show at /home/user/.julia/packages/NCDatasets/t0uQw/src/dataset.jl:443 [inlined]
 [6] show(::IOContext{REPL.Terminals.TTYTerminal}, ::MIME{Symbol("text/plain")}, ::NCDataset{Nothing}) at ./multimedia.jl:47
 [7] display(::REPL.REPLDisplay, ::MIME{Symbol("text/plain")}, ::Any) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.5/REPL/src/REPL.jl:214
 [8] display(::REPL.REPLDisplay, ::Any) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.5/REPL/src/REPL.jl:218
 [9] display(::Any) at ./multimedia.jl:328
 [10] #invokelatest#1 at ./essentials.jl:710 [inlined]
 [11] invokelatest at ./essentials.jl:709 [inlined]
 [12] print_response(::IO, ::Any, ::Bool, ::Bool, ::Any) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.5/REPL/src/REPL.jl:238
 [13] print_response(::REPL.AbstractREPL, ::Any, ::Bool, ::Bool) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.5/REPL/src/REPL.jl:223
 [14] (::REPL.var"#do_respond#54"{Bool,Bool,REPL.var"#64#73"{REPL.LineEditREPL,REPL.REPLHistoryProvider},REPL.LineEditREPL,REPL.LineEdit.Prompt})(::Any, ::Any, ::Any) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.5/REPL/src/REPL.jl:822
 [15] #invokelatest#1 at ./essentials.jl:710 [inlined]
 [16] invokelatest at ./essentials.jl:709 [inlined]
 [17] run_interface(::REPL.Terminals.TextTerminal, ::REPL.LineEdit.ModalInterface, ::REPL.LineEdit.MIState) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.5/REPL/src/LineEdit.jl:2355
 [18] run_frontend(::REPL.LineEditREPL, ::REPL.REPLBackendRef) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.5/REPL/src/REPL.jl:1144
 [19] (::REPL.var"#38#42"{REPL.LineEditREPL,REPL.REPLBackendRef})() at ./task.jl:356

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions