PostgreSQL在何处处理 sql查询之四十七

本文涉及的产品
云原生数据库 PolarDB MySQL 版,Serverless 5000PCU 100GB
简介:

再次上溯:

复制代码
/*--------------------
 * subquery_planner
 *      Invokes the planner on a subquery.  We recurse to here for each
 *      sub-SELECT found in the query tree.
 *
 * glob is the global state for the current planner run.
 * parse is the querytree produced by the parser & rewriter.
 * parent_root is the immediate parent Query's info (NULL at the top level).
 * hasRecursion is true if this is a recursive WITH query.
 * tuple_fraction is the fraction of tuples we expect will be retrieved.
 * tuple_fraction is interpreted as explained for grouping_planner, below.
 *
 * If subroot isn't NULL, we pass back the query's final PlannerInfo struct;
 * among other things this tells the output sort ordering of the plan.
 *
 * Basically, this routine does the stuff that should only be done once
 * per Query object.  It then calls grouping_planner.  At one time,
 * grouping_planner could be invoked recursively on the same Query object;
 * that's not currently true, but we keep the separation between the two
 * routines anyway, in case we need it again someday.
 *
 * subquery_planner will be called recursively to handle sub-Query nodes
 * found within the query's expressions and rangetable.
 *
 * Returns a query plan.
 *--------------------
 */
Plan *
subquery_planner(PlannerGlobal *glob, Query *parse,
                 PlannerInfo *parent_root,
                 bool hasRecursion, double tuple_fraction,
                 PlannerInfo **subroot)
{
    int            num_old_subplans = list_length(glob->subplans);
    PlannerInfo *root;
    Plan       *plan;
    List       *newHaving;
    bool        hasOuterJoins;
    ListCell   *l;

    /* Create a PlannerInfo data structure for this subquery */
    root = makeNode(PlannerInfo);
    root->parse = parse;
    root->glob = glob;
    root->query_level = parent_root ? parent_root->query_level + 1 : 1;
    root->parent_root = parent_root;
    root->plan_params = NIL;
    root->planner_cxt = CurrentMemoryContext;
    root->init_plans = NIL;
    root->cte_plan_ids = NIL;
    root->eq_classes = NIL;
    root->append_rel_list = NIL;
    root->rowMarks = NIL;
    root->hasInheritedTarget = false;

    root->hasRecursion = hasRecursion;
    if (hasRecursion)
        root->wt_param_id = SS_assign_special_param(root);
    else
        root->wt_param_id = -1;
    root->non_recursive_plan = NULL;

    /*
     * If there is a WITH list, process each WITH query and build an initplan
     * SubPlan structure for it.
     */
    if (parse->cteList)
        SS_process_ctes(root);

    /*
     * Look for ANY and EXISTS SubLinks in WHERE and JOIN/ON clauses, and try
     * to transform them into joins.  Note that this step does not descend
     * into subqueries; if we pull up any subqueries below, their SubLinks are
     * processed just before pulling them up.
     */
    if (parse->hasSubLinks)
        pull_up_sublinks(root);

    /*
     * Scan the rangetable for set-returning functions, and inline them if
     * possible (producing subqueries that might get pulled up next).
     * Recursion issues here are handled in the same way as for SubLinks.
     */
    inline_set_returning_functions(root);

    /*
     * Check to see if any subqueries in the jointree can be merged into this
     * query.
     */
    parse->jointree = (FromExpr *)
        pull_up_subqueries(root, (Node *) parse->jointree, NULL, NULL);

    /*
     * If this is a simple UNION ALL query, flatten it into an appendrel. We
     * do this now because it requires applying pull_up_subqueries to the leaf
     * queries of the UNION ALL, which weren't touched above because they
     * weren't referenced by the jointree (they will be after we do this).
     */
    if (parse->setOperations)
        flatten_simple_union_all(root);

    /*
     * Detect whether any rangetable entries are RTE_JOIN kind; if not, we can
     * avoid the expense of doing flatten_join_alias_vars().  Also check for
     * outer joins --- if none, we can skip reduce_outer_joins(). This must be
     * done after we have done pull_up_subqueries, of course.
     */
    root->hasJoinRTEs = false;
    hasOuterJoins = false;
    foreach(l, parse->rtable)
    {
        RangeTblEntry *rte = (RangeTblEntry *) lfirst(l);

        if (rte->rtekind == RTE_JOIN)
        {
            root->hasJoinRTEs = true;
            if (IS_OUTER_JOIN(rte->jointype))
            {
                hasOuterJoins = true;
                /* Can quit scanning once we find an outer join */
                break;
            }
        }
    }

    /*
     * Preprocess RowMark information.    We need to do this after subquery
     * pullup (so that all non-inherited RTEs are present) and before
     * inheritance expansion (so that the info is available for
     * expand_inherited_tables to examine and modify).
     */
    preprocess_rowmarks(root);

    /*
     * Expand any rangetable entries that are inheritance sets into "append
     * relations".  This can add entries to the rangetable, but they must be
     * plain base relations not joins, so it's OK (and marginally more
     * efficient) to do it after checking for join RTEs.  We must do it after
     * pulling up subqueries, else we'd fail to handle inherited tables in
     * subqueries.
     */
    expand_inherited_tables(root);

    /*
     * Set hasHavingQual to remember if HAVING clause is present.  Needed
     * because preprocess_expression will reduce a constant-true condition to
     * an empty qual list ... but "HAVING TRUE" is not a semantic no-op.
     */
    root->hasHavingQual = (parse->havingQual != NULL);

    /* Clear this flag; might get set in distribute_qual_to_rels */
    root->hasPseudoConstantQuals = false;

    /*
     * Do expression preprocessing on targetlist and quals, as well as other
     * random expressions in the querytree.  Note that we do not need to
     * handle sort/group expressions explicitly, because they are actually
     * part of the targetlist.
     */
    parse->targetList = (List *)
        preprocess_expression(root, (Node *) parse->targetList,
                              EXPRKIND_TARGET);

    parse->returningList = (List *)
        preprocess_expression(root, (Node *) parse->returningList,
                              EXPRKIND_TARGET);

    preprocess_qual_conditions(root, (Node *) parse->jointree);

    parse->havingQual = preprocess_expression(root, parse->havingQual,
                                              EXPRKIND_QUAL);

    foreach(l, parse->windowClause)
    {
        WindowClause *wc = (WindowClause *) lfirst(l);

        /* partitionClause/orderClause are sort/group expressions */
        wc->startOffset = preprocess_expression(root, wc->startOffset,
                                                EXPRKIND_LIMIT);
        wc->endOffset = preprocess_expression(root, wc->endOffset,
                                              EXPRKIND_LIMIT);
    }

    parse->limitOffset = preprocess_expression(root, parse->limitOffset,
                                               EXPRKIND_LIMIT);
    parse->limitCount = preprocess_expression(root, parse->limitCount,
                                              EXPRKIND_LIMIT);

    root->append_rel_list = (List *)
        preprocess_expression(root, (Node *) root->append_rel_list,
                              EXPRKIND_APPINFO);

    /* Also need to preprocess expressions for function and values RTEs */
    foreach(l, parse->rtable)
    {
        RangeTblEntry *rte = (RangeTblEntry *) lfirst(l);

        if (rte->rtekind == RTE_FUNCTION)
            rte->funcexpr = preprocess_expression(root, rte->funcexpr,
                                                  EXPRKIND_RTFUNC);
        else if (rte->rtekind == RTE_VALUES)
            rte->values_lists = (List *)
                preprocess_expression(root, (Node *) rte->values_lists,
                                      EXPRKIND_VALUES);
    }

    /*
     * In some cases we may want to transfer a HAVING clause into WHERE. We
     * cannot do so if the HAVING clause contains aggregates (obviously) or
     * volatile functions (since a HAVING clause is supposed to be executed
     * only once per group).  Also, it may be that the clause is so expensive
     * to execute that we're better off doing it only once per group, despite
     * the loss of selectivity.  This is hard to estimate short of doing the
     * entire planning process twice, so we use a heuristic: clauses
     * containing subplans are left in HAVING.    Otherwise, we move or copy the
     * HAVING clause into WHERE, in hopes of eliminating tuples before
     * aggregation instead of after.
     *
     * If the query has explicit grouping then we can simply move such a
     * clause into WHERE; any group that fails the clause will not be in the
     * output because none of its tuples will reach the grouping or
     * aggregation stage.  Otherwise we must have a degenerate (variable-free)
     * HAVING clause, which we put in WHERE so that query_planner() can use it
     * in a gating Result node, but also keep in HAVING to ensure that we
     * don't emit a bogus aggregated row. (This could be done better, but it
     * seems not worth optimizing.)
     *
     * Note that both havingQual and parse->jointree->quals are in
     * implicitly-ANDed-list form at this point, even though they are declared
     * as Node *.
     */
    newHaving = NIL;
    foreach(l, (List *) parse->havingQual)
    {
        Node       *havingclause = (Node *) lfirst(l);

        if (contain_agg_clause(havingclause) ||
            contain_volatile_functions(havingclause) ||
            contain_subplans(havingclause))
        {
            /* keep it in HAVING */
            newHaving = lappend(newHaving, havingclause);
        }
        else if (parse->groupClause)
        {
            /* move it to WHERE */
            parse->jointree->quals = (Node *)
                lappend((List *) parse->jointree->quals, havingclause);
        }
        else
        {
            /* put a copy in WHERE, keep it in HAVING */
            parse->jointree->quals = (Node *)
                lappend((List *) parse->jointree->quals,
                        copyObject(havingclause));
            newHaving = lappend(newHaving, havingclause);
        }
    }
    parse->havingQual = (Node *) newHaving;

    /*
     * If we have any outer joins, try to reduce them to plain inner joins.
     * This step is most easily done after we've done expression
     * preprocessing.
     */
    if (hasOuterJoins)
        reduce_outer_joins(root);

    /*
     * Do the main planning.  If we have an inherited target relation, that
     * needs special processing, else go straight to grouping_planner.
     */
    if (parse->resultRelation &&
        rt_fetch(parse->resultRelation, parse->rtable)->inh)
        plan = inheritance_planner(root);
    else
    {
        plan = grouping_planner(root, tuple_fraction);
        /* If it's not SELECT, we need a ModifyTable node */
        if (parse->commandType != CMD_SELECT)
        {
            List       *returningLists;
            List       *rowMarks;

            /*
             * Set up the RETURNING list-of-lists, if needed.
             */
            if (parse->returningList)
                returningLists = list_make1(parse->returningList);
            else
                returningLists = NIL;

            /*
             * If there was a FOR UPDATE/SHARE clause, the LockRows node will
             * have dealt with fetching non-locked marked rows, else we need
             * to have ModifyTable do that.
             */
            if (parse->rowMarks)
                rowMarks = NIL;
            else
                rowMarks = root->rowMarks;

            plan = (Plan *) make_modifytable(parse->commandType,
                                             parse->canSetTag,
                                       list_make1_int(parse->resultRelation),
                                             list_make1(plan),
                                             returningLists,
                                             rowMarks,
                                             SS_assign_special_param(root));
        }
    }

    /*
     * If any subplans were generated, or if there are any parameters to worry
     * about, build initPlan list and extParam/allParam sets for plan nodes,
     * and attach the initPlans to the top plan node.
     */
    if (list_length(glob->subplans) != num_old_subplans ||
        root->glob->nParamExec > 0)
        SS_finalize_plan(root, plan, true);

    /* Return internal info if caller wants it */
    if (subroot)
        *subroot = root;

    return plan;
}
复制代码

上溯

复制代码
PlannedStmt *
standard_planner(Query *parse, int cursorOptions, ParamListInfo boundParams)
{
    PlannedStmt *result;
    PlannerGlobal *glob;
    double        tuple_fraction;
    PlannerInfo *root;
    Plan       *top_plan;
    ListCell   *lp,
               *lr;

    /* Cursor options may come from caller or from DECLARE CURSOR stmt */
    if (parse->utilityStmt &&
        IsA(parse->utilityStmt, DeclareCursorStmt))
        cursorOptions |= ((DeclareCursorStmt *) parse->utilityStmt)->options;

    /*
     * Set up global state for this planner invocation.  This data is needed
     * across all levels of sub-Query that might exist in the given command,
     * so we keep it in a separate struct that's linked to by each per-Query
     * PlannerInfo.
     */
    glob = makeNode(PlannerGlobal);

    glob->boundParams = boundParams;
    glob->subplans = NIL;
    glob->subroots = NIL;
    glob->rewindPlanIDs = NULL;
    glob->finalrtable = NIL;
    glob->finalrowmarks = NIL;
    glob->resultRelations = NIL;
    glob->relationOids = NIL;
    glob->invalItems = NIL;
    glob->nParamExec = 0;
    glob->lastPHId = 0;
    glob->lastRowMarkId = 0;
    glob->transientPlan = false;

    /* Determine what fraction of the plan is likely to be scanned */
    if (cursorOptions & CURSOR_OPT_FAST_PLAN)
    {
        /*
         * We have no real idea how many tuples the user will ultimately FETCH
         * from a cursor, but it is often the case that he doesn't want 'em
         * all, or would prefer a fast-start plan anyway so that he can
         * process some of the tuples sooner.  Use a GUC parameter to decide
         * what fraction to optimize for.
         */
        tuple_fraction = cursor_tuple_fraction;

        /*
         * We document cursor_tuple_fraction as simply being a fraction, which
         * means the edge cases 0 and 1 have to be treated specially here.    We
         * convert 1 to 0 ("all the tuples") and 0 to a very small fraction.
         */
        if (tuple_fraction >= 1.0)
            tuple_fraction = 0.0;
        else if (tuple_fraction <= 0.0)
            tuple_fraction = 1e-10;
    }
    else
    {
        /* Default assumption is we need all the tuples */
        tuple_fraction = 0.0;
    }

    /* primary planning entry point (may recurse for subqueries) */
    top_plan = subquery_planner(glob, parse, NULL,
                                false, tuple_fraction, &root);

    /*
     * If creating a plan for a scrollable cursor, make sure it can run
     * backwards on demand.  Add a Material node at the top at need.
     */
    if (cursorOptions & CURSOR_OPT_SCROLL)
    {
        if (!ExecSupportsBackwardScan(top_plan))
            top_plan = materialize_finished_plan(top_plan);
    }

    /* final cleanup of the plan */
    Assert(glob->finalrtable == NIL);
    Assert(glob->finalrowmarks == NIL);
    Assert(glob->resultRelations == NIL);
    top_plan = set_plan_references(root, top_plan);
    /* ... and the subplans (both regular subplans and initplans) */
    Assert(list_length(glob->subplans) == list_length(glob->subroots));
    forboth(lp, glob->subplans, lr, glob->subroots)
    {
        Plan       *subplan = (Plan *) lfirst(lp);
        PlannerInfo *subroot = (PlannerInfo *) lfirst(lr);

        lfirst(lp) = set_plan_references(subroot, subplan);
    }

    /* build the PlannedStmt result */
    result = makeNode(PlannedStmt);

    result->commandType = parse->commandType;
    result->queryId = parse->queryId;
    result->hasReturning = (parse->returningList != NIL);
    result->hasModifyingCTE = parse->hasModifyingCTE;
    result->canSetTag = parse->canSetTag;
    result->transientPlan = glob->transientPlan;
    result->planTree = top_plan;
    result->rtable = glob->finalrtable;
    result->resultRelations = glob->resultRelations;
    result->utilityStmt = parse->utilityStmt;
    result->subplans = glob->subplans;
    result->rewindPlanIDs = glob->rewindPlanIDs;
    result->rowMarks = glob->finalrowmarks;
    result->relationOids = glob->relationOids;
    result->invalItems = glob->invalItems;
    result->nParamExec = glob->nParamExec;

    return result;
}
复制代码

 再上溯:

复制代码
/*****************************************************************************
 *
 *       Query optimizer entry point
 *
 * To support loadable plugins that monitor or modify planner behavior,
 * we provide a hook variable that lets a plugin get control before and
 * after the standard planning process.  The plugin would normally call
 * standard_planner().
 *
 * Note to plugin authors: standard_planner() scribbles on its Query input,
 * so you'd better copy that data structure if you want to plan more than once.
 *
 *****************************************************************************/
PlannedStmt *
planner(Query *parse, int cursorOptions, ParamListInfo boundParams)
{
    PlannedStmt *result;

    if (planner_hook)
        result = (*planner_hook) (parse, cursorOptions, boundParams);
    else
        result = standard_planner(parse, cursorOptions, boundParams);
    return result;
}
复制代码

对 standard_planner 分析:

复制代码
/*
 * Query -
 *      Parse analysis turns all statements into a Query tree
 *      for further processing by the rewriter and planner.
 *
 *      Utility statements (i.e. non-optimizable statements) have the
 *      utilityStmt field set, and the Query itself is mostly dummy.
 *      DECLARE CURSOR is a special case: it is represented like a SELECT,
 *      but the original DeclareCursorStmt is stored in utilityStmt.
 *
 *      Planning converts a Query tree into a Plan tree headed by a PlannedStmt
 *      node --- the Query structure is not used by the executor.
 */
typedef struct Query
{
    NodeTag        type;

    CmdType        commandType;    /* select|insert|update|delete|utility */

    QuerySource querySource;    /* where did I come from? */

    uint32        queryId;        /* query identifier (can be set by plugins) */

    bool        canSetTag;        /* do I set the command result tag? */

    Node       *utilityStmt;    /* non-null if this is DECLARE CURSOR or a
                                 * non-optimizable statement */

    int            resultRelation; /* rtable index of target relation for
                                 * INSERT/UPDATE/DELETE; 0 for SELECT */

    bool        hasAggs;        /* has aggregates in tlist or havingQual */
    bool        hasWindowFuncs; /* has window functions in tlist */
    bool        hasSubLinks;    /* has subquery SubLink */
    bool        hasDistinctOn;    /* distinctClause is from DISTINCT ON */
    bool        hasRecursive;    /* WITH RECURSIVE was specified */
    bool        hasModifyingCTE;    /* has INSERT/UPDATE/DELETE in WITH */
    bool        hasForUpdate;    /* FOR UPDATE or FOR SHARE was specified */

    List       *cteList;        /* WITH list (of CommonTableExpr's) */

    List       *rtable;            /* list of range table entries */
    FromExpr   *jointree;        /* table join tree (FROM and WHERE clauses) */

    List       *targetList;        /* target list (of TargetEntry) */

    List       *returningList;    /* return-values list (of TargetEntry) */

    List       *groupClause;    /* a list of SortGroupClause's */

    Node       *havingQual;        /* qualifications applied to groups */

    List       *windowClause;    /* a list of WindowClause's */

    List       *distinctClause; /* a list of SortGroupClause's */

    List       *sortClause;        /* a list of SortGroupClause's */

    Node       *limitOffset;    /* # of result tuples to skip (int8 expr) */
    Node       *limitCount;        /* # of result tuples to return (int8 expr) */

    List       *rowMarks;        /* a list of RowMarkClause's */

    Node       *setOperations;    /* set-operation tree if this is top level of
                                 * a UNION/INTERSECT/EXCEPT query */

    List       *constraintDeps; /* a list of pg_constraint OIDs that the query
                                 * depends on to be semantically valid */
} Query;
复制代码

一点一点地分析吧:

    if (parse->utilityStmt &&
        IsA(parse->utilityStmt, DeclareCursorStmt))
        cursorOptions |= ((DeclareCursorStmt *) parse->utilityStmt)->options;

parse->utilityStmt 是false,所以不成立。

再接着:

复制代码
/*
     * Set up global state for this planner invocation.  This data is needed
     * across all levels of sub-Query that might exist in the given command,
     * so we keep it in a separate struct that's linked to by each per-Query
     * PlannerInfo.
     */
    glob = makeNode(PlannerGlobal);

    glob->boundParams = boundParams;
    glob->subplans = NIL;
    glob->subroots = NIL;
    glob->rewindPlanIDs = NULL;
    glob->finalrtable = NIL;
    glob->finalrowmarks = NIL;
    glob->resultRelations = NIL;
    glob->relationOids = NIL;
    glob->invalItems = NIL;
    glob->nParamExec = 0;
    glob->lastPHId = 0;
    glob->lastRowMarkId = 0;
    glob->transientPlan = false;
复制代码

这一段只是设置了一个初始化好的 PlannerGlobal 指针。

复制代码
#define newNode(size, tag) \
( \
    AssertMacro((size) >= sizeof(Node)),        /* need the tag, at least */ \
    newNodeMacroHolder = (Node *) palloc0fast(size), \
    newNodeMacroHolder->type = (tag), \
    newNodeMacroHolder \
)
#endif   /* __GNUC__ */


#define makeNode(_type_)        ((_type_ *) newNode(sizeof(_type_),T_##_type_))
复制代码

接下来,对于我的SQL : select id, val from tst04 where id>1 ,

cursorOPtion 是false,所以不成立,变成:

复制代码
if (cursorOptions & CURSOR_OPT_FAST_PLAN)
    {
       ...
    }
    else
    {
        /* Default assumption is we need all the tuples */
        tuple_fraction = 0.0;
    }
复制代码

接下来:

    /* primary planning entry point (may recurse for subqueries) */
    top_plan = subquery_planner(glob, parse, NULL,
                                false, tuple_fraction, &root);








 
相关实践学习
使用PolarDB和ECS搭建门户网站
本场景主要介绍基于PolarDB和ECS实现搭建门户网站。
阿里云数据库产品家族及特性
阿里云智能数据库产品团队一直致力于不断健全产品体系,提升产品性能,打磨产品功能,从而帮助客户实现更加极致的弹性能力、具备更强的扩展能力、并利用云设施进一步降低企业成本。以云原生+分布式为核心技术抓手,打造以自研的在线事务型(OLTP)数据库Polar DB和在线分析型(OLAP)数据库Analytic DB为代表的新一代企业级云原生数据库产品体系, 结合NoSQL数据库、数据库生态工具、云原生智能化数据库管控平台,为阿里巴巴经济体以及各个行业的企业客户和开发者提供从公共云到混合云再到私有云的完整解决方案,提供基于云基础设施进行数据从处理、到存储、再到计算与分析的一体化解决方案。本节课带你了解阿里云数据库产品家族及特性。
目录
相关文章
|
14天前
|
SQL
sql语句加正则 简化查询
sql语句加正则 简化查询
14 0
sql语句加正则 简化查询
|
1月前
|
SQL
sql server链接查询
sql server链接查询
17 1
|
1月前
|
SQL
sql server简单查询
sql server简单查询
14 1
|
1月前
|
关系型数据库 分布式数据库 数据库
PolarDB常见问题之加了索引但是查询没有使用如何解决
PolarDB是阿里云推出的下一代关系型数据库,具有高性能、高可用性和弹性伸缩能力,适用于大规模数据处理场景。本汇总囊括了PolarDB使用中用户可能遭遇的一系列常见问题及解答,旨在为数据库管理员和开发者提供全面的问题指导,确保数据库平稳运行和优化使用体验。
|
22天前
|
SQL 关系型数据库 MySQL
mysql一条sql查询出多个统计结果
mysql一条sql查询出多个统计结果
14 0
|
1月前
|
SQL
sql高级查询
sql高级查询
12 0
|
1月前
|
SQL 存储 数据可视化
10个高级的 SQL 查询技巧
10个高级的 SQL 查询技巧
|
6天前
|
SQL 存储 Oracle
关系型数据库查询数据的语句
本文介绍了关系型数据库中的基本SQL查询语句,包括选择所有或特定列、带条件查询、排序、分组、过滤分组、表连接、限制记录数及子查询。SQL还支持窗口函数、存储过程等高级功能,是高效管理数据库的关键。建议深入学习SQL及相应数据库系统文档。
8 2
|
1月前
|
SQL 数据库
sql server高级查询,看这篇文章就够了
sql server高级查询,看这篇文章就够了
22 0
|
1月前
|
SQL
T-SQL 语句查询
T-SQL 语句查询
55 0

热门文章

最新文章