Restrict number of columns in output step

[Saurabh309]

My Dev dataset (Bigquery) has four columns: f_name, raw_f_name, l_name, raw_l_name, I don't have raw columns in Production.

I am creating recipe in Dev and apply transformation on non raw columns and storing output in a big query table without raw columns.

If I delete for raw columns in my recipe and migrate it to prod, prod flow complains about missing raw columns.

Any way I can control the output of the flow?

[APrasad_Tri]

Hi @Saurabh Mittal? , thank you for reaching out.

It seems your final table structure is expecting the 4 columns because the first time when you created the table it set up column structure. In order to remove the above error, you have to drop the table. This can be achieved in the publishing action of Manual settings page.

Click on the edit in Publishing action:

Set the Output dataset to the Drop table every run

This will create a new table with the latest table structure. You can revert the settings once the new table is created with the same name.

Hope this helps.

Best,

Apeksha Prasad