Allow mixing for pretokenized data.

[GeorgiosSmyrnis]

This enables mixing of pretokenized data with the tokenize_shuffle.py script. This is allowed by the --pretok_tars flag, which assumes that the tarfiles that the script contain already tokenized data.

[GeorgiosSmyrnis]

This now also fixes a rare issue where the dataset produced by tokenize shuffle becomes broken due to duplicate file names within the tarfiles. While this could only happen if tokenizing the same sequence of tokens, this now converts the naming scheme within a tarfile to a simple name of the format {shard_index}_{iterator}.