Community Spring Cleaning week is here! Join your fellow Maveryx in digging through your old posts and marking comments on them as solved. Learn more here!

Alteryx Designer Cloud Discussions

Find answers, ask questions, and share expertise about Alteryx Designer Cloud.
SOLVED

Automate files with similar names

43e0a22239487a85233b
8 - Asteroid

Hello folks,

?

I have a situation where some files will be uploaded into Cloud Storage using similar names, like 20220101 AAA BBB XYZ for example. and I would like to invoke a dataprep recipe for every file with AAA BBB in the name, does not matter the rest of the name. Is it possible?

?

Another example bellow that I want to trigger a recipe for any file uploaded with " Surcharges" in the file name as below:

 

How to parameterize it? Any clue?

 

Thanks!

5 REPLIES 5
APrasad_Tri
Alteryx Alumni (Retired)

Hi @Gustavo Castanheira? , thank you for reaching out.

Yes, it can be done in dataprep using parameterization. There are 2 ways to achieve this:

  1. Via Library: This approach is used when we are creating the flow first time. We can select the data and import and add it to a flow.

From Libray, select Import data and go to the data path. Click on parameterize when we hover over one of the datasets in the path

Here we can use a wildcard or Trifacta pattern to pick up all the data following the pattern.

 

  1. Via Flow: This approach is used when we add and group similar files to an existing flow.

 

Here, you have to right-click on the existing dataset and select Replace with a dataset with parameters.

This will redirect to the path where the datasets lie. Highlight the part which needs to be parameterized.

We can use wildcard or Trifacta patterns to define the highlighted part.

Please note the UI will display the first 10MB of data from 1st dataset. To get the data from all the datasets we have to run a random sample.

Hope this helps.

Best,

Apeksha

43e0a22239487a85233b
8 - Asteroid

No words to thank you! One more question, how to reference it in cloud functions in order to trigger the dataprep recipe to run over these files once in the example there is not a FileName ?

APrasad_Tri
Alteryx Alumni (Retired)

Hi @Gustavo Castanheira?, Do you mean to schedule the flow or create a pipeline?

If yes, then follow the below steps. If not then please provide a detailed description of the query.

Via Flow: On the flow page, click on the schedule

set up the scheduling period

The flow will trigger and run the outputs. For more information please refer here.

 

Via Plans: On the navigation bar, there is Plans icon. Use this icon to data pipeline and orchestration.

Create the plan and add the flow tasks i.e. the output objects in the plan to trigger the recipe

Search the flow and select the output

 

The plans will help to run all the outputs. We. can schedule the plans as well. For more information on plans, refer here.

 

Hope this helps.

Best,

Apeksha Prasad

 

43e0a22239487a85233b
8 - Asteroid

Hello Apeksha thanks for support but the scenario is different, for example, every month will drop a file in cloud storage with a standard name like that 20221001 XPTO XPTO.xls, of course the date part will change every month. Currently we have a python functions that automate every time a file drops at storage, invoke the respective dataprep recipe using a FileName var that is the same from parameters from source. In that case where we will have date patterns and more information to check, how to ensure the recipe could be run and refresh the datasets?

APrasad_Tri
Alteryx Alumni (Retired)

Hi @Gustavo Castanheira? ,

Based on my understanding of the question, jobs contain all the information. Please check the job history section to access all the jobs (plans, flow and sample) and logs. If not, then can you please describe the issue in detail? Thanks.

Best,

Apeksha