浅谈mysql的子查询

mysql的子查询的优化一直不是很友好，一直有受业界批评比较多,也是我在sql优化中遇到过最多的问题之一，你可以点击这里 ，这里来获得一些信息，mysql在处理子查询的时候，会将子查询改写,通常情况下，我们希望由内到外，也就是先完成子查询的结果，然后在用子查询来驱动外查询的表，完成查询，但是恰恰相反，子查询不会先被执行；今天希望通过介绍一些实际的案例来加深对mysql子查询的理解：

案例：用户反馈数据库响应较慢，许多业务动更新被卡住；登录到数据库中观察，发现长时间执行的sql；

| 10437 | usr0321t9m9 | 10.242.232.50:51201 | oms | Execute | 1179 | Sending

Sql为：
select tradedto0_.* from a1 tradedto0_ where tradedto0_.tradestatus='1'
and (tradedto0_.tradeoid in (select orderdto1_.tradeoid from a2 orderdto1_ where
orderdto1_.proname like '%??%' or orderdto1_.procode like '%??%')) and tradedto0_.undefine4='1'
and tradedto0_.invoicetype='1' and tradedto0_.tradestep='0' and (tradedto0_.orderCompany like '0002%') order by tradedto0_.tradesign ASC, tradedto0_.makertime desc limit 15;

2.其他表的更新被阻塞：
update a1 set tradesign='DAB67634-795C-4EAC-B4A0-78F0D531D62F',
markColor=' #CD5555', memotime='2012-09- 22', markPerson='??' where tradeoid in ('gy2012092204495100032') ；
为了尽快恢复应用，将其长时间执行的sql kill掉后，应用恢复正常;
3.分析执行计划:
db@3306 ：explain select tradedto0_.* from a1 tradedto0_ where tradedto0_.tradestatus='1' and (tradedto0_.tradeoid in (select orderdto1_.tradeoid
from a2 orderdto1_ where orderdto1_.proname like '%??%' or orderdto1_.procode like '%??%')) and tradedto0_.undefine4='1' and tradedto0_.invoicetype='1' and tradedto0_.tradestep='0' and (tradedto0_.orderCompany like '0002%') order by tradedto0_.tradesign ASC, tradedto0_.makertime desc limit 15;
+----+--------------------+------------+------+---------------+------+---------+------+-------+-----
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+--------------------+------------+------+---------------+------+---------+------+-------+-----
| 1 | PRIMARY | tradedto0_ | ALL | NULL | NULL | NULL | NULL | 27454 | Using where; Using filesort |
| 2 | DEPENDENT SUBQUERY | orderdto1_ | ALL | NULL | NULL | NULL | NULL | 40998 | Using where |
+----+--------------------+------------+------+---------------+------+---------+------+-------+-----
从执行计划上，我们开始一步一步地进行优化：
首先，我们看看执行计划的第二行，也就是子查询的那部分，orderdto1_进行了全表的扫描，我们看看能不能添加适当的索引：
A.使用覆盖索引:
db@3306：alter table a2 add index ind_a2(proname,procode,tradeoid);
ERROR 1071 (42000): Specified key was too long; max key length is 1000 bytes
添加组合索引超过了最大key length限制：
B．查看该表的字段定义：

 db@3306 ：DESC  a2 ;
+---------------------+---------------+------+-----+---------+-------+
| FIELD               | TYPE          | NULL | KEY | DEFAULT | Extra |
+---------------------+---------------+------+-----+---------+-------+
| OID                 | VARCHAR(50)   | NO   | PRI | NULL    |       |
| TRADEOID            | VARCHAR(50)   | YES  |     | NULL    |       |
| PROCODE             | VARCHAR(50)   | YES  |     | NULL    |       |
| PRONAME             | VARCHAR(1000) | YES  |     | NULL    |       |
| SPCTNCODE           | VARCHAR(200)  | YES  |     | NULL    |       |

C．查看表字段的平均长度：

db@3306 ：SELECT MAX(LENGTH(PRONAME)),avg(LENGTH(PRONAME)) FROM a2;
+----------------------+----------------------+
| MAX(LENGTH(PRONAME)) | avg(LENGTH(PRONAME)) |
+----------------------+----------------------+
|    95              |       24.5588 |

D．缩小字段长度

ALTER TABLE MODIFY COLUMN PRONAME VARCHAR(156);

再进行执行计划分析：
db@3306 ：explain select tradedto0_.* from a1 tradedto0_ where tradedto0_.tradestatus='1' and (tradedto0_.tradeoid in (select orderdto1_.tradeoid from a2 orderdto1_ where orderdto1_.proname like '%??%' or orderdto1_.procode like '%??%')) and tradedto0_.undefine4='1' and tradedto0_.invoicetype='1' and tradedto0_.tradestep='0' and (tradedto0_.orderCompany like '0002%') order by tradedto0_.tradesign ASC, tradedto0_.makertime desc limit 15;
+----+--------------------+------------+-------+-----------------+----------------------+---------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+--------------------+------------+-------+-----------------+----------------------+---------+
| 1 | PRIMARY | tradedto0_ | ref | ind_tradestatus | ind_tradestatus | 345 | const,const,const,const | 8962 | Using where; Using filesort |
| 2 | DEPENDENT SUBQUERY | orderdto1_ | index | NULL | ind_a2 | 777 | NULL | 41005 | Using where; Using index |
+----+--------------------+------------+-------+-----------------+----------------------+---------+

发现性能还是上不去，关键在两个表扫描的行数并没有减小（8962*41005），上面添加的索引没有太大的效果，现在查看t表的执行结果：
db@3306 ：select orderdto1_.tradeoid from t orderdto1_ where orderdto1_.proname like '%??%' or orderdto1_.procode like '%??%';
Empty set (0.05 sec)
结果集为空，所以需要将t表的结果集做作为驱动表；
4．通过上面测试验证，普通的mysql子查询写法性能上是很差的，为mysql的子查询天然的弱点，需要将sql进行改写为关联的写法：
select tradedto0_.* from a1 tradedto0_ ,(select orderdto1_.tradeoid from a2 orderdto1_ where orderdto1_.proname like '%??%' or orderdto1_.procode like '%??%')t2 where tradedto0_.tradestatus='1' and (tradedto0_.tradeoid=t2.tradeoid ) and tradedto0_.undefine4='1' and tradedto0_.invoicetype='1' and tradedto0_.tradestep='0' and (tradedto0_.orderCompany like '0002%') order by tradedto0_.tradesign ASC, tradedto0_.makertime desc limit 15;
5.查看执行计划：
db@3306 ：explain select tradedto0_.* from a1 tradedto0_ ,(select orderdto1_.tradeoid from a2 orderdto1_ where orderdto1_.proname like '%??%' or orderdto1_.procode like '%??%')t2 where tradedto0_.tradestatus='1' and (tradedto0_.tradeoid=t2.tradeoid ) and tradedto0_.undefine4='1' and tradedto0_.invoicetype='1' and tradedto0_.tradestep='0' and (tradedto0_.orderCompany like '0002%') order by tradedto0_.tradesign ASC, tradedto0_.makertime desc limit 15;
+----+-------------+------------+-------+---------------+----------------------+---------+------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+------------+-------+---------------+----------------------+---------+------+
| 1 | PRIMARY | NULL | NULL | NULL | NULL | NULL | NULL | NULL | Impossible WHERE noticed after reading const tables |
| 2 | DERIVED | orderdto1_ | index | NULL | ind_a2 | 777 | NULL | 41005 | Using where; Using index |
+----+-------------+------------+-------+---------------+----------------------+---------+------+

6.执行时间：

db@3306 ：select tradedto0_.* from a1 tradedto0_ ,(select orderdto1_.tradeoid from a2 orderdto1_ where orderdto1_.proname like '%??%' or orderdto1_.procode like '%??%')t2 where tradedto0_.tradestatus='1' and (tradedto0_.tradeoid=t2.tradeoid ) and tradedto0_.undefine4='1' and tradedto0_.invoicetype='1' and tradedto0_.tradestep='0' and (tradedto0_.orderCompany like '0002%') order by tradedto0_.tradesign ASC, tradedto0_.makertime desc limit 15;
Empty set (0.03 sec)

缩短到了毫秒；

总结：
1. mysql子查询在执行计划上有着明显的弱点，需要将子查询进行改写
可以参考：
a. 生产库中遇到mysql的子查询：http://hidba.org/?p=412
b. 内建的builtin InnoDB,子查询阻塞更新：http://hidba.org/?p=456
2. 在表结构设计上，不要随便使用varchar(N)的大字段，导致无法使用索引
可以参考：
a. JDBC内存管理—varchar2(4000)的影响：http://hidba.org/?p=31
b. innodb中大字段的限制：http://hidba.org/?p=144
c. innodb使用大字段text，blob的一些优化建议： http://hidba.org/?p=551