NumPy to NDArrayView zero-copy conversion

[Olobir]

Hello NuMojo maintainers,

First of all, thank you for all your work on NuMojo!!!

I usually use Polars, NumPy, and Numba to write my "fast" Python code. My goal is to integrate Mojo into my workflows.
I want to zero-copy convert existing data structures (for example, a NumPy ndarray or a Polars DataFrame) into a NuMojo NDArrayView, performing in-place computations, and then reflecting those changes back in Python.
I've seen that a NDArrayView type is mentioned as "TODO". I assume it's on NuMojos roadmap.
I haven't found a NuMojo utility function to wrap a NumPy array without copying, yet.

Is there an existing helper in NuMojo for creating a zero-copy NDArrayView from a NumPy ndarray?

I noticed that Modular has introduced a high-performance “LayoutTensor” type in Mojo. Will NuMojo eventually be build on top of the native Mojo tensor APIs?

I've tried to implement a "numpy_to_ndarray_view()" function which seems to work as it should. I would be very happy to get some feedback if it's OK like this:

from python import Python, PythonObject
from memory.unsafe_pointer import UnsafePointer
from numojo.core.ndstrides import NDArrayStrides
from numojo.prelude import *
from numojo.routines.io.formatting import GLOBAL_PRINT_OPTIONS

def main():
    var np = Python.import_module("numpy")
    var numpy_arr = np.array(Python.list(1, 2, 3), dtype=np.float64)
    var numojo_arr = numpy_to_ndarray_view[DType.float64](numpy_arr)

    print("NumPy:", numpy_arr)
    print("  own data:", numpy_arr.flags["OWNDATA"])
    print("NuMojo:", numojo_arr._array_to_string(0, 0, GLOBAL_PRINT_OPTIONS))
    print("  own data:", numojo_arr.flags["OWNDATA"])

    # NumPy: [1. 2. 3.]
    #   own data: True
    # NuMojo: [1.0000 2.0000 3.0000]
    #   own data: False

    numojo_arr.item(0) = 99

    print("After modification of NuMojo view, original data has changed, too.")
    print("NumPy:", numpy_arr)
    print("NuMojo:", numojo_arr._array_to_string(0, 0, GLOBAL_PRINT_OPTIONS))

    # After modification of NuMojo view, original data has changed, too.
    # NumPy: [99.  2.  3.]
    # NuMojo: [99.0000  2.0000  3.0000]

fn numpy_to_ndarray_view[dtype: DType](
    array: PythonObject
) raises -> NDArray[dtype]:
    """
    Zero-copy convert a NumPy ndarray to a NuMojo NDArray.
    Supports both C- and F-contiguous layouts.

    Parameters
        dtype: The element type for the resulting NDArray (default f64).
        array: A NumPy ndarray of matching dtype, C- or F-contiguous.
    Returns
        A zero-copy NDArray[dtype] view over the same buffer.
    """
    # Check memory layout
    var is_c = Bool(array.flags["C_CONTIGUOUS"])
    var is_f = Bool(array.flags["F_CONTIGUOUS"])
    if not (is_c or is_f):
        raise Error("numpy_to_ndarray_view: array must be C- or F-contiguous")

    # set arguments
    # List comprehension support starts in v25.4 (2025-06-18)
    # var shape_list: List[Int] = [Int(dim) for dim in array.shape]
    var shape_list = List[Int]()
    for dim in array.shape:
        shape_list.append(Int(dim))
    var shape = NDArrayShape(shape_list)

    var ptr: UnsafePointer[Scalar[dtype]] = array.ctypes.data.unsafe_get_as_pointer[dtype]()

    var offset: Int = Int(0)  # we start at the first element

    var order = String("C" if is_c else "F")
    var strides = NDArrayStrides(shape, order)
    
    # build array view
    var view = NDArray[dtype](shape, ptr, offset, strides)
    return view

Kind regards
Olobir

[forfudan]

Hi @Olobir! It is a very good question! From months ago, we were trying to prepare for this array view to avoid excessive copying during array manipulation and conversion. For example, this discussion and the PR 175 are some early changes in design to allow shared memory. The branch "arrayview" contains a pilot ArrayView on the matrix type.

However, the only thing that blocked us from proceeding further is the lack of support of parameterized trait in Mojo. Without this functionality, we cannot build a generic ArrayView type. So we have to wait for some months (or years? Hope not).

Another feature that we are waiting for is the reference system. Note that the ref keyword is only available in Mojo v25.4. We need to wait for it being more mature.

I noticed that Modular has introduced a high-performance “LayoutTensor” type in Mojo. Will NuMojo eventually be build on top of the native Mojo tensor APIs?

Probably not, I think. We built the NDArray from very low level. The performance difference between LayoutTensor and NDArray is not necessary significant.

[Olobir]

Thank you very much for the explanation, @forfudan .
I'm keeping my fingers crossed that we don't have to wait for too long for the missing Mojo functionalities.
Keep up the good work!