RestGetIndicesAction matches REST queries that start with _ even when no indicies can ever match that

[thecoop]

This came out as an issue due to #106820

All REST API calls that start with _ match RestGetIndicesAction due to the generic catch-all route /{index}, even though no index can ever start with _. This messes up capability checks on invalid APIs, returning true (as it matched a handler) even though that can't ever work. This also causes some odd behaviour with other APIs, with POST /_typo returning 405 rather than 404, as it matches /{index} even though it could never work.

We should limit wildcards to not match _. There are various levels we could do this at.

[elasticsearchmachine]

Pinging @elastic/es-core-infra (Team:Core/Infra)

[thecoop]

Initial thoughts on possible solutions:

1. Make wildcards of the exact format {index} to not match _ (hacky but quick)
2. Make wildcards in general not match urls starting with _ (need to audit all wildcards to see if there are any that do need to match _)
3. Make route definitions into regexes rather than string matches, and specify index wildcards as starting with [A-Za-z] (could be slow, and possibly confusing with existing wildcard matches)

[thecoop]

Option 2 is not valid, as there are some wildcards that do require _ to match (eg RestNodesInfoAction, expecting _all as a special-case).

This means we would need to either do the specific hacky fix just for {index}, or introduce regexes to wildcard matches. This will affect performance of rest handler matches.

[thecoop]

Option 3 is difficult. When a route wildcard is declared, it has to be the same as all other wildcards in the same path location declared by other handlers. This means that if we were to use a regex declared in the route path, the regex would have to be copied by all other routes as well - there are 192 routes that start with /{index}.

So then we would have to declare a regex for any specific wildcard either at the top-level, outside of any specific rest handlers, or in the single rest handler 'responsible' for that wildcard, if that could indeed be determined. This gives the relationships between rest handlers and the rest controller a very sticky web-like quality that is quite undesired.

But the other option is to completely rewrite how rest handlers are declared to a single controlling class that determines all paths for all handlers, or at least has significant oversight.

[thecoop]

We're going to rewrite PathTrie to support this, once we have a proposal for how to handle _ and non-_ wildcards.

[thecoop]

Also related - #25563