Better memory management #50

alphaville · 2024-11-07T01:08:24Z

This is an alternative to PR #49; this is still WIP.

Main Changes

Introduce T** m_d_ptrMatrices, which is a raw pointer to the matrices of the tensor (to replace pointersToMatrices)

TODOs

New memory management scheme (malloc/free)
addAB: use gemm when nMats=1
Slicing: set m_d_ptrMatrix = nullptr when slicing with axis=0 or axis=1
Remove unit test tensorPointersToMatrices (method deprecation)
Get rid of pointersToMatrices() in leastSquaresBatched
CholeskyBatchFactoriser: get rid of pointersToMatrices
CholeskyBatchFactoriser: throw error if nMats=1
Reshaping: new test needed (e.g., reshape + multiply with addAB)
Unit test with reshaping and slicing

Note

It probably makes sense to always have m_d_ptrMatrix = nullptr whenever nMats=1 and throw an error when the user attempts to use any batched method with nMats=1.

Associated Issues

Closes Do not allocate new memory from within a method #48

Allocating and destroying memory No memory leaks evident atm

Introduce initialisePointersToMatricesData to initialise m_d_ptrMatrices; appropriate checks for safety.

Also in the slice constructor, keep m_d_ptrMatrices=nullptr when we dont slice along axis=2

use cublasDGemm if nMats=1

When necessary, memory is reallocated for m_d_ptrMatrices within reshape

No need to free/reallocate when we can reuse the already allocated memory space (when the number of matrices decreases); no new memory allocation when numMats=1

and format code

include/tensor.cuh

test/testTensor.cu

ruairimoran

check comments. Looks great

Unit test: Slice along axis=2 and reshape Write documentation for DTensor<T>::reshape Fix formatting issues in testTensor

ruairimoran

nice

Free all allocated memory if allocation fails Fix code formatting in testTensor allocateOnDevice made void

alphaville added 8 commits November 7, 2024 01:06

Create pointer-to-matrices memory (m_d_ptrMatrices)

aa1317b

Allocating and destroying memory No memory leaks evident atm

Initialise m_d_ptrMatrices

2ac61cf

Introduce initialisePointersToMatricesData to initialise m_d_ptrMatrices; appropriate checks for safety.

Use initialisePointersToMatricesData to initialise m_d_ptrMatrices

3338852

Also in the slice constructor, keep m_d_ptrMatrices=nullptr when we dont slice along axis=2

update addAB to use preallocated memory

36bf6ed

use cublasDGemm if nMats=1

mirror implementation for DTensor<float>::addAB

4e5da4d

Get rid of pointersToMatrices() in LS

296efd4

New method DTensor<T>::ptrMatrices() exposes m_d_ptrMatrices

02564d5

Completely got rid of pointersToMatrices()

f2e86c7

alphaville mentioned this pull request Nov 7, 2024

Memory management improvements #49

Closed

5 tasks

alphaville added 6 commits November 7, 2024 02:36

No memory allocation for m_d_ptrMatrices unless nMats > 1

424a251

ptrMatrices now declared as a const method

b2857cd

Update DTensor<T>::reshape

7c32dae

When necessary, memory is reallocated for m_d_ptrMatrices within reshape

Better memory management when reshaping

225ba0d

No need to free/reallocate when we can reuse the already allocated memory space (when the number of matrices decreases); no new memory allocation when numMats=1

Safe memory allocation in reshape and unit test

d4b5472

Test reshape more thoroughly

5b3a677

and format code