Also, this is significantly decreases Redshift cluster performance in a matter of reading and writing data, because everything (read and write queries) goes through the leader node, and as it says above, when the leader node is overloaded, there will be a performance issue.This action causes you data to be written much slower. When you decide to turn this flag to off, all data is gathered from all of the data nodes into a single node, the leader node, because it needs to reorganize the sorting of the rows to output and also compress it if needed as a single stream. It means, that the data is written directly from the data nodes themselves, which is much faster because it's doing it in parallel and skips the leader node. When you unload data from Redshift while the flag PARALLEL is TRUE, it will create at least X files, when X is the number of nodes you choose to construct the Redshift cluster of, in the first place. The purpose of leader node, is to control the data nodes, it hold the necessary information to work with all data in Redshift, either read or write. Each cluster includes at least 2 servers, when one of them is a leader node and the rest are data nodes.

The most important reason is because of the way a Redshift cluster designed.

If you ask why the use of PARALLEL FALSE is not recommended, I'll try to explain it in several points: Therefore, it will alway add at least the prefix 000, because Redshift doesn't know what size of the file he is going to output in the first place, so he's adding this suffix in case the output will reach the size of 6.2 GB. So, for example, if you unload 13.4 GB of data, UNLOAD creates theįollowing three files. The maximum size for a data file is 6.2 GB. More data files serially, sorted absolutely according to the ORDER BYĬlause, if one is used. If PARALLEL is OFF or FALSE, UNLOAD writes to one or suffix (when the exists only when the compression is enabled), because there is a limit to a file size that Redshift can output, as says in the documentation:īy default, UNLOAD writes data in parallel to multiple files,Īccording to the number of slices in the cluster.

Even then, the file will always include the 000. As it says in Amazon Redshift UNLOAD document, if you do not want it to be split into several parts, you can use PARALLEL FALSE, but it is strongly recommended to leave it enabled.

Redshift unload