I have a directory that contains 61 small files that all have the same schema. I know how to use the union operation to manually combine these files, but it's going to take a long time. Is there a way to have Trifacta automatically apply a recipe to all 61 files?
When you select data you can define a folder as a Trifacta dataset, and then in this case all files stored in the folder will be seen as only one large file. No more union need to work on all files at the same time.
Wow, that's such a time saver! So I assume if more files end up in that folder, Trifacta will just automatically add those new files into the single large file?
Yes, all new small files will be part of the large logic file seen via Trifacta. And any job running on the large file will include data from new small files.
