Dev plans for transforms

[whitews]

This is a post to discuss the plans for moving the Transform classes from FlowKit to FlowUtils along with the FlowJo biex implementation to support the Python cytometry community. See scverse/pytometry#47

[whitews]

@burtonrj

A couple items to get the discussion started...

1. What should the API should look like for the transform classes? I'd like to keep the functions public for those that are already using them or prefer them. So something like flowutils.transforms.LogicleTransform with the functions in flowutils.transforms.funcs.logicle. That's my first thought.
2. Should there be a mechanism to identify GatingML 2.0 compliant transform classes?

[burtonrj]

Hi Scott, completely agree RE the API - keeping the functions public is also great for testing and transparency. Are we happy with the C implementation of the Logicle transform? Speaking for ignorance here but I've never seen a comparison between FlowUtils implementation and the FastLogicle cpp implementation commonly used in the R packages.

RE the GatingML 2.0 compliance. Could we achieve this with a robust abstract class or even a Protocol for GatingML 2.0 compliant transforms?

[whitews]

Hey Ross!

I've been satisfied with the C implementation so far. Its origins come from the C++ implementation in the old fcm library. But that used SWIG as the wrapper and was a maintenance headache. The code wasn't that long or complicated, so I ported it to straight C and used the Python C API to avoid any external wrapper dependencies. I do use the NumPy C API so that is a build-time dependency, but the recent change to pyproject.toml solves this quite nicely.

As I understand it, the FastLogicle is essentially a binned lookup table wrapped around the C++ Logicle implementation. Is that correct? I'm sure that approach would be faster for sufficiently large arrays, but wouldn't give the exact values due to the binning. This binning issue pops up in a few places, including in FlowJo gating operations, and in some cases can cause significant differences in results.

I guess the TLDR is I moved over time to an implementation that I could understand & debug, which is also faster than a Python implementation. That said, some benchmarks on all these variations would be interesting to see (C vs C++ vs Python, along with variations using Cython, etc.). Definitely welcome a discussion on any of this.

For the GML compliance, I like using a base class or protocol. We can play with that after the initial transfer of the Transform classes. One thing is that I am going to keep wrapper classes in FlowKit to retain the id argument but will leave that out of the FlowUtils implementation. In the context of a gating strategy, the individual transforms and comp matrices need identifiers, but in FlowUtils having to specify IDs/names for each instance would probably be annoying to users.