Recursive plan output for queries involving selectable stored procedures

[dyemanov]

Related to issue #5322.

Prior to v3.0, if the query includes selectable procedures, their plans were embedded into the main query plan. This was changed in v3 due to refactoring and because such "mixed" plans were often hardly readable. But users kept complaining the feature was useful, at least to analyze plans for possible NATURAL scans inside. I'm still pessimistic about that, but the explained plan format allows to show embedded plans nicely indented, so why not. This PR re-allows embedding nested plans into the main one. It's controlled by the configuration parameter and disabled by default (thus compatible with v3 and v4).

Example:

set term ^;

create or alter procedure p1 returns (a int)
as
begin
  select null from rdb$database into :a;
  suspend;

  for select rdb$relation_id from rdb$relations into :a
  do suspend;
end^

create or alter procedure p2 returns (a int)
as
begin
  select null from rdb$database into :a;
  suspend;

  for select rdb$relation_id from rdb$relations into :a
  do suspend;

  for select a from p1 into :a
  do suspend;
end^

create or alter procedure p3 returns (a int)
as
begin
  select null from rdb$database into :a;
  suspend;

  for select rdb$relation_id from rdb$relations into :a
  do suspend;

  for select a from p3 into :a
  do suspend;
end^

set term ;^

SQL> select * from p1;

Select Expression
    -> Procedure "P1" Scan -- explain the P1 body below (for PlanRecursionLimit > 0)
        -> Select Expression (line 4, column 3)
            -> Singularity Check
                -> Table "RDB$DATABASE" Full Scan
        -> Select Expression (line 7, column 3)
            -> Table "RDB$RELATIONS" Full Scan

SQL> select * from p2;

Select Expression
    -> Procedure "P2" Scan -- explain the P2 body below (for PlanRecursionLimit > 0)
        -> Select Expression (line 4, column 3)
            -> Singularity Check
                -> Table "RDB$DATABASE" Full Scan
        -> Select Expression (line 7, column 3)
            -> Table "RDB$RELATIONS" Full Scan
        -> Select Expression (line 10, column 3)
            -> Procedure "P1" Scan -- explain the P1 body below (for PlanRecursionLimit > 1)
                -> Select Expression (line 4, column 3)
                    -> Singularity Check
                        -> Table "RDB$DATABASE" Full Scan
                -> Select Expression (line 7, column 3)
                    -> Table "RDB$RELATIONS" Full Scan

SQL> select * from p3;

Select Expression
    -> Procedure "P3" Scan -- explain the P3 body below (for PlanRecursionLimit > 0)
        -> Select Expression (line 4, column 3)
            -> Singularity Check
                -> Table "RDB$DATABASE" Full Scan
        -> Select Expression (line 7, column 3)
            -> Table "RDB$RELATIONS" Full Scan
        -> Select Expression (line 10, column 3)
            -> Procedure "P3" Scan -- recursion for P3 is detected, not going deeper

Questions:

• Is the PlanRecursionLimit parameter name OK or should it be better named MaxPlanRecursion?
• Should we also support per-connection parameter override (via DPB and/or session management statements)? IMO, this could be useful when analyzing things in special tools or dedicated ISQL sessions without a need to change the server configuration.

[sim1984]

1. MaxRecursionPlanDepth
2. Yes, sure. For analysis tools, this is very convenient. You can change the depth of detail at any time.

[sim1984]

I would prefer the session environment statement. So that you do not have to do reconnections.

[hvlad]

Does you considered to show plan of separate PSQL objects as separate entities ?
For example

SQL> select * from p2;

-- query itself
Select Expression
    -> Procedure "P2" Scan

-- Procedure P2 
    -> Select Expression (line 4, column 3)
        -> Singularity Check
            -> Table "RDB$DATABASE" Full Scan
    -> Select Expression (line 7, column 3)
        -> Table "RDB$RELATIONS" Full Scan
    -> Select Expression (line 10, column 3)
        -> Procedure "P1" Scan 

-- Procedure P1
    -> Select Expression (line 4, column 3)
        -> Singularity Check
            -> Table "RDB$DATABASE" Full Scan
    -> Select Expression (line 7, column 3)
        -> Table "RDB$RELATIONS" Full Scan

[aafemt]

Taking into account that stored routines cannot have variable plans and show plans for all routes, there is no point to output plan for any function more than once.

[hvlad]

Does you considered to show plan of separate PSQL objects as separate entities ?

Forgot to add - it will have big effect (readability and text size) when SP2 calls SP1 many times

[dyemanov]

I would prefer the session environment statement. So that you do not have to do reconnections.

I'm afraid they will not affect plan depth inside MON$ tables, because plans are dumped by other attachments that may see different value of the max plan depth. Would it be OK if the session override will not affect MON$ tables (they will use config setting)?

[dyemanov]

Does you considered to show plan of separate PSQL objects as separate entities ? For example

SQL> select * from p2;

-- query itself
Select Expression
    -> Procedure "P2" Scan

-- Procedure P2 
    -> Select Expression (line 4, column 3)
        -> Singularity Check
            -> Table "RDB$DATABASE" Full Scan
    -> Select Expression (line 7, column 3)
        -> Table "RDB$RELATIONS" Full Scan
    -> Select Expression (line 10, column 3)
        -> Procedure "P1" Scan 

-- Procedure P1
    -> Select Expression (line 4, column 3)
        -> Singularity Check
            -> Table "RDB$DATABASE" Full Scan
    -> Select Expression (line 7, column 3)
        -> Table "RDB$RELATIONS" Full Scan

From one side, I like this - because the truncation at the 64KB limit is likely to happen after then main plan, while in my implementation it may happen in the middle of the main plan. Also I agree with @aafemt that it does not make sense to print the same routine multiple times. From another side, it makes impossible to track who calls who, which is quite straightforward in my output.

[sim1984]

Maybe this setting shouldn't affect the plans in mon$? We have mon$compiled_statement, which allows you to detail procedure plans.

[hvlad]

Does you considered to show plan of separate PSQL objects as separate entities ? For example

SQL> select * from p2;

-- query itself
Select Expression
    -> Procedure "P2" Scan

-- Procedure P2 
    -> Select Expression (line 4, column 3)
        -> Singularity Check
            -> Table "RDB$DATABASE" Full Scan
    -> Select Expression (line 7, column 3)
        -> Table "RDB$RELATIONS" Full Scan
    -> Select Expression (line 10, column 3)
        -> Procedure "P1" Scan 

-- Procedure P1
    -> Select Expression (line 4, column 3)
        -> Singularity Check
            -> Table "RDB$DATABASE" Full Scan
    -> Select Expression (line 7, column 3)
        -> Table "RDB$RELATIONS" Full Scan

From one side, I like this - because the truncation at the 64KB limit is likely to happen after then main plan, while in my implementation it may happen in the middle of the main plan. Also I agree with @aafemt that it does not make sense to print the same routine multiple times. From another side, it makes impossible to track who calls who, which is quite straightforward in my output.

May be I didn't get your point, but at the sample above it is clear that:

• query calls P2
• P2 calls P1

[dyemanov]

Yeah, you're right. I need to think more about this.

[sim1984]

In such a conclusion of the plan, there is a question in what order the procedure plans should be printed. Depth or width traversal?

[hvlad]

In such a conclusion of the plan, there is a question in what order the procedure plans should be printed. Depth or width traversal?

Width, i.e. first should be printed the objects that referenced directly by the query (i.e. at 1st level), than objects that referenced by the just printed objects, and so on.

Honestly, I see no real need to use more than 1 level of references. In the sample above, it will not print plan of P1.
If one need to see plan of P1, it could just prepare query with it. This is how it works before fb3.

[romansimakov]

In such a conclusion of the plan, there is a question in what order the procedure plans should be printed. Depth or width traversal?

Width, i.e. first should be printed the objects that referenced directly by the query (i.e. at 1st level), than objects that referenced by the just printed objects, and so on.

Honestly, I see no real need to use more than 1 level of references. In the sample above, it will not print plan of P1. If one need to see plan of P1, it could just prepare query with it. This is how it works before fb3.

This forces a user to perform such recursion or implements a tool. An example could be comparing or monitoring plans during time or after upgrade using trace for example.

[asfernandes]

This case seems broken:

create or alter procedure p1 returns (o1 integer)
as
begin
    for select rdb$relation_id from rdb$database into o1 do suspend;
end!

create or alter procedure p2 returns (o1 integer)
as
begin
    for select o1 from p1 into o1 do suspend;
end!

set plan on!
set explain off!
select * from p2!

PLAN (
    -> PLAN (
        -> PLAN RDB$DATABASE NATURAL))

[livius2]

Taking into account that stored routines cannot have variable plans and show plans for all routes, there is no point to output plan for any function more than once.

What about EXECUTE STATEMENT inside? But maybe this is for trace only, not the prepare stage.