If you have a query that:
- have a lot of subqueries
- join many different tables or subqueries
- going to return a huge amount of data
- take forever to run (more than 1 minutes)
- you have a busy database server
You can run your query in a hundred times faster speed by storing some groups of subqueries results into a temp table first before running the final joins using these temp tables.
This way, you will have:
- a smaller sets of queries to run
- which will reduce the memory usage
- and reduce the chance of blocking other running queries
- and it runs a hell lot faster.
Let’s make a super simple example.
--Old and Slow --let's assume that huge_table_a and huge_table_b both contain 1 billion records each SELECT tbl_a.item, tbl_a.profit_share, tbl_b.profit_share, FROM (SELECT item, Sum(profit) / Max(profit) AS profit_share FROM (SELECT item, profit, UNNEST(regions) AS region FROM huge_table_a WHERE region = "south") GROUP BY item) tbl_a JOIN (SELECT item, Sum(profit) / Max(profit) AS profit_share FROM (SELECT item, profit, UNNEST(regions) AS region FROM huge_table_a WHERE region = "north") GROUP BY item) tbl_b ON tbl_a.item = tbl_b.item ORDER BY tbl_a.profit_share DESC;
Using Temp Tables
-- Will run a lot faster especially in a busy DB CREATE TEMP TABLE tmp_table1 AS ( SELECT item, Sum(profit) / Max(profit) AS profit_share FROM (SELECT item, profit, UNNEST(regions) AS region FROM huge_table_a WHERE region = "south") GROUP BY item ); CREATE TEMP TABLE tmp_table2 AS ( SELECT item, Sum(profit) / Max(profit) AS profit_share FROM (SELECT item, profit, UNNEST(regions) AS region FROM huge_table_a WHERE region = "south") GROUP BY item ); SELECT tbl_a.item, tbl_a.profit_share, tbl_b.profit_share, FROM tmp_table1 tbl_a JOIN tmp_table2 tbl_b ON tbl_a.item = tbl_b.item ORDER BY tbl_a.profit_share DESC;