Support SQL Table Valued Functions

[hatyo]

This provides support to table-valued-functions, it introduces the necessary framework support across different components: parsing, semantic analysis, plan generation, planning, and execution, to pre-defined table-valued functions.

Two pre-defined table-functions are introduced in this PR:

1. values function:

Most major SQL vendors support this function, the relational engine now supports this function as well:

select * from values (1, 'a', 3.0), (4, 'b', 4.5);
-- returns two rows:
-- 1, 'a', 3.0
-- 4, 'b', 4.5

It also enables naming the resulting table along with its columns:

select * from values (1, 2.0, (3, 4, 'foo')), (10, 90.2, (5, 6.0, 'bar')) as A(B, C, W(X, Y, Z))
-- returns two rows with the following metadata:
-- B  | C    | W(X, Y, Z)
-- 1  | 2.0  | (3, 4.0, 'foo')
-- 10 | 90.2 | (5, 6.0, 'bar')

Notice how the value 4 in the first row's nested structure W.Y was promoted to 4.0, this is because this function analyzes all the rows, and for each leaf in every (nested) tuple, it looks for a type that accommodates for all the individual values of the leaf across all rows (max type) and injects type promotions if necessary.
Also note that the type aliasing works with nested array and structs, to some degrees addresses some of the issues reported in #3231, and in the future, the new logic can be leveraged to resolve that ticket as well.

2. range function:

This is also a very common function, it returns a range between:

• given start (inclusive), defaulting to 0.
• given end (exclusive).
• optional step size.

select ID from range (1, 5)
-- returns 4 rows
-- 1
-- 2
-- 3
-- 4

The implementation of this TVF has streaming semantics. In other words, it does not pre-materialize all the values in the range, but rather return them on-demand.

With this PR, we can hopefully start adding more advanced TVFs easily.

This resolves #3282.

[normen662]

General comment:
I am not sure if this changes the path of INSERT INTO T VALUES (...). What is that QGM if it is changed. What is the QGM for SELECT * FROM VALUES(...)? is it supposed to use an ExplodeValue?

[hatyo]

General comment: I am not sure if this changes the path of INSERT INTO T VALUES (...). What is that QGM if it is changed. What is the QGM for SELECT * FROM VALUES(...)? is it supposed to use an ExplodeValue?

Yes, the ExplodeValue remains to be used in:

• INSERT from VALUES codepath (it is almost exactly as it was before this PR).
• SELECT from VALUES codepath.

I decided to keep it that way anticipating that a switch to using a more general TableFunctionValue et al will cause hundreds of plans to be changed, dramatically increasing the scope of this PR. I want to split this legwork across (multiple) PRs if possible.

The QGM of SELECT FROM VALUES(...) is the following:

Ending up with a single physical VALUES box.

[normen662]

Since these values are fake anyway, when we spoke I thought we had agreed on using one of these for both arrays and the new stream value.

[hatyo]

Yes, but this can only happen once StreamingValue is able to handle both arrays (explode) and streamed values (e.g. range). Then we can use FirstOrDefaultStreamingValue that expects an underlying StreamingValue. please let me know if this is aligned with your thinking.

By the way, I think there is a different in the empty semantics between FirstOrDefaultStreamingValue and FirstOrDefaultValue. We can talk about it offline if you want.

[normen662]

Three options:

• FirstOrDefaultStreamingValue goes away with the suggested changes elsewhere,
• FirstOrDefaultStreamingValue properly implements plan serialization together with putting PFirstOrDefaultStreamingValue into PValue
• FirstOrDefaultStreamingValue does not implement plan serialization at all (but I think the constraint on the types will then fail to serialize, so properly two options only).

[hatyo]

PFirstOrDefaultStreamingValue already implements plan serialization, I forgot adding it as a possibility in PValue. The code works because it is only used currently in DerivationProperty.
I just added it to PValue.

[normen662]

Right, but the derivation is then put into a constraint that is then put into the continuation. I am not sure why this worked.

[normen662]

I am not really sure how this can pass with the plan serialization problem I think exists due to the FirstOrDefaultStreamingValue problem.

[hatyo]

what's the plan serialization problem?
the FirstOrDefaultStreamingValue is not even serialized, it is only used in the DerivationProperty to represent the semantics of the derivation.

[normen662]

This is guaranteed by the class definition to be a StreamingValue. In the very remote case that we change that without afterthought I think a lot of things would break anyway so why not just cast it?

[hatyo]

It is extra defensive, considering the proto message is something sent from an external client.

[normen662]

Forgot one more thing: can you please update ReleaseNotes.md? Thanks!

[hatyo]

Forgot one more thing: can you please update ReleaseNotes.md? Thanks!

I don't think we should do that manually anymore. There should be a CI job that maintains this file.