einsum support

[ubIQio]

Hi Zoltán,

I was wondering if you had any intention of supporting einsum in the foreseeable future?

Asking for a friend :)

[v923z]

This goes a bit in the direction of symbolic computations, but the real question is, whether einsum offers anything beyond defining aliases. If so, wouldn't it make more sense to implement this in python?

A somewhat similar question was raised in #278, and I came to the conclusion that we should, perhaps, add snippets, and collect such things there. In fact, numpy does exactly this: /site-packages/numpy/core/einsumfunc.py.

[ubIQio]

That sounds like the way to go; it seems to me that this feature would greatly facilitate the migration of deep learning projects to edge devices.

[v923z]

@ubIQio Do you want to take a stab at it?

[ubIQio]

OK, I'll have a look!

[v923z]

@ubIQio The difficulty is that you want to have the same namespace (numpy).

[adamnpeace]

Hi there @v923z,

I'm just taking a look at this and also interested in getting support for einsum. I'm not sure I follow when you say that it's similar to np.r_[], since in vanilla NumPy, einsum actually calls the numpy/core/src/multiarray/einsum C implementation which includes the parser and the loops that the einsum string specify.

Would it be possible for us to integrate this into ulab or am I missing something?

Thanks!

[v923z]

@adamnpeace

einsum actually calls the numpy/core/src/multiarray/einsum C implementation which includes the parser and the loops that the einsum string specify.

Yeah, you are right. Sorry, I totally tanked it. So, as far as I see, in numpy, this is a bit of a mix of python, and C. Do you think we have to do that, or you want to implement everything in C? Also, I would like to know, what the optimize keyword does. The documentation is a bit sketchy on this subject.

[adamnpeace]

@v923z As far as I am aware, optimize is a newer concept in NumPy 1.12 that allows the caller to specify a way of preprocessing more complex operation strings at the Python layer (by executing inner parts of the einsum first to reduce the total number of flops), but by default NumPy will just pass the args straight to c_einsum and execute the string left-to-right at the C layer.

I think it's definitely possible to preserve most of the einsum functionality and speed by simply porting the C implementation, without any need for the python implementation for the time being. I'll take a stab at it and see what I can do!

[v923z]

@adamnpeace

I think it's definitely possible to preserve most of the einsum functionality and speed by simply porting the C implementation, without any need for the python implementation for the time being. I'll take a stab at it and see what I can do!

The numpy implementation is here
https://github.com/numpy/numpy/blob/5cae51e794d69dd553104099305e9f92db237c53/numpy/core/src/multiarray/einsum.c.src#L769, but it makes heavy use of a number of numpy facilities, so porting is not gonna be straightforward.

Of course, the first question is, whether you want to have complete numpy compatibility. You might have noticed that most of the functions in ulab are incomplete implementations in the sense not all possible keywords are supported etc. I am definitely OK with partial implementations, and I am really glad, if someone lends me a hand with the code. I am working on a new feature, so my resources are a bit tied down at the moment, and that is why I haven't really picked up einsum.