Skip to content

Task 1 – Core Architecture Redesign (NLnet)

Jump to bottom
Pierre Raybaut edited this page Jul 8, 2025 · 5 revisions

Task 1 – Core Architecture Redesign (NLnet)

This page documents the results of Task 1 of the NLnet-funded DataLab refactoring project.

Summary

This task focused on decoupling the computation engine, object model, and I/O layer from the existing DataLab application (cdl module), in order to prepare a standalone processing library.

Report #1 - Date: 2025-06-26

ℹ️ Since the date of this report, the sigima_ module has been renamed to sigima, and has been externalized into a standalone package named Sigima. Other changes have also been made to the DataLab codebase, but this report will not cover those changes.

Milestone Status

The following paragraphs summarize the status of each milestone at the time of this report.

✅ 1a. Decouple core data model and computational functions

All core signal/image processing features have been migrated from the cdl module to a new internal module named sigima_, which is fully independent from GUI-related logic.

Achieving this milestone involved a specification and design phase that started in early May 2025, followed by a development phase that began in late May 2025. The development phase was completed on June 26, 2025, with the merge of the feature/sigima branch into the develop branch (branch feature/sigima has been deleted since then).

The work behind this task involved the following commits:

  • Commit e383b71: First commit, initial creation of sigima_ module (which was initially named sigima but later renamed to sigima_ to avoid conflicts with the future standalone package).
  • Commit d59f7ea: Last commit, marking the completion of the core migration.

🟡 1b. Decouple I/O features (excluding HDF5)

Basic I/O functions have been successfully extracted to sigima_, but the I/O plugin system from cdl.plugins is still pending migration.

➡️ Remaining action:

  • Implement a flexible I/O plugin architecture within sigima_, similar to the one in cdl.

🚧 1c. Redesign the API for the new core library

The API redesign is underway. A new object model for signals and images is in place, but still relies on some shared infrastructure from the cdl package.

➡️ Remaining actions (as listed in sigima_/__init__.py):

  • Remove all dependencies on cdl
  • Remove all references to Conf (if no longer needed)
  • Evaluate setting a minimum NumPy version to enable advanced type hints
  • Clarify the role of the PREFIX attribute in BaseObj

✅ 1d. Refactor the application layer to integrate the new core

The cdl package has been updated to use sigima_ as its processing backend.
All previously embedded logic in cdl now delegates to sigima_.

The work behind this task has been done together with the work on milestone 1a, and is reflected in the same commits.

💡 Even though this milestone was initially planned to be completed after the core migration, it was more efficient to implement it in parallel.

Code References

Next steps

  • Extract sigima_ into a standalone sigima package
  • Rename cdl to datalab, and cdl package to datalab-platform
  • Move tests and documentation
  • Implement plugin systems in sigima
Clone this wiki locally