Comments on: Error: 1104 – SQLSTATE: 42000 ER_TOO_BIG_SELECT http://iderror.com/errors/mysql/mysql-server/error-1104-sqlstate-42000-er_too_big_select/ You have errors, we have solutions! Fri, 10 Jun 2011 15:06:54 +0000 hourly 1 http://wordpress.org/?v=3.2.1 By: srinivas pangulurihttp://iderror.com/errors/mysql/mysql-server/error-1104-sqlstate-42000-er_too_big_select/comment-page-1/#comment-7744 srinivas panguluri Wed, 18 Mar 2009 01:49:13 +0000 http://localhost/idError/?p=7551#comment-7744 The actual problem: "Maximum Join Size" is not the same as "Number of Records Returned." If you have to join two 1000-row tables, you'll have a maximum join size of around 1000x1000 = 1000000, since 1000000 different row combinations have to be compared, even if only one row matches! MySQL will optimize away many of these rows if you've got a WHERE clause, but still, the join size in queries often ends up being enormous.The reason MySQLCC sets the max join size is to avoid user error -- the logic is, if you've got a join size of 1000000 you're probably doing something wrong. Sometimes optimization solves this, sometimes the query is completely the opposite of what you intended, but often, you actually *do* want a join size of a million. The selection in server settings only goes up to 2^24 = 16 million (as far as I can tell, this number was selected arbitrarily), and even that is not always enough (the MySQL variable MAX_JOIN_SIZE is optional and has a limit of 2^32 = 4 billion, so any value from 1 to 4 billion should be available).When using the command-line client, MAX_JOIN_SIZE is ignored by default, since SQL_BIG_SELECTS=1. But MySQLCC enforces a MAX_JOIN_SIZE, even when you don't want one. The solution: Apply the patch I posted earlier. Then set the "Maximum join size" to 0; it will set SQL_BIG_SELECTS=1 and therefore not return an error on big queries.That said, in most cases you shouldn't go above a join size of a million if only 1 record is returned (except for GROUP BY queries, I suppose). If you're getting this error, chances are your query could be optimized considerably. The actual problem: “Maximum Join Size” is not the same as “Number of Records Returned.” If you have to join two 1000-row tables, you’ll have a maximum join size of around 1000×1000 = 1000000, since 1000000 different row combinations have to be compared, even if only one row matches! MySQL will optimize away many of these rows if you’ve got a WHERE clause, but still, the join size in queries often ends up being enormous.

The reason MySQLCC sets the max join size is to avoid user error — the logic is, if you’ve got a join size of 1000000 you’re probably doing something wrong. Sometimes optimization solves this, sometimes the query is completely the opposite of what you intended, but often, you actually *do* want a join size of a million. The selection in server settings only goes up to 2^24 = 16 million (as far as I can tell, this number was selected arbitrarily), and even that is not always enough (the MySQL variable MAX_JOIN_SIZE is optional and has a limit of 2^32 = 4 billion, so any value from 1 to 4 billion should be available).

When using the command-line client, MAX_JOIN_SIZE is ignored by default, since SQL_BIG_SELECTS=1. But MySQLCC enforces a MAX_JOIN_SIZE, even when you don’t want one. The solution: Apply the patch I posted earlier. Then set the “Maximum join size” to 0; it will set SQL_BIG_SELECTS=1 and therefore not return an error on big queries.

That said, in most cases you shouldn’t go above a join size of a million if only 1 record is returned (except for GROUP BY queries, I suppose). If you’re getting this error, chances are your query could be optimized considerably.

]]>