Hi I have created a flow with a new recipe which cleans and prepares the data and creates a table in GBQ. What I need is after the cleaning of the data in the flow, then run a GBQ View query to copy the data to another dataset in GBQ.

[4ff3a001681946b94681]

Basically i have created a flow as below

what I want to do now is to take the result from the create_applicants _in_clean and run a GBQ View Query on it which will copy the d=table into a new dataset called Prod

Any help is apprecaite it as I am new to Dataprep

[Trifacta_Alumni]

Hi @Vahid Razaghzadeh?,

So, your Trifacta job publishes to a table in BigQuery. Are you trying to copy this table, or to read it back into Trifacta? (Both?) Are you trying to use non-Trifacta tools to take a slice of the published table as a new dataset? Apologies, I'm a little confused!

Note that Dataprep never actually modifies a source: every time you [re-]import a datasource into Trifacta, or indeed add a dataset to a flow, you are essentially creating a copy to work with.

I hope this helps...? Could you perhaps clarify what you are trying to do?

Cheers,

Nathanael

[4ff3a001681946b94681]

Dear Nathanael

Thank you for the reply. Just to clarify below is what I need to do.

I have created a recepie in the flow which cleans the data as I need. What I want to do next is to get the output of the recepie and apply a script which I have created in GBQ. What this script does selects the columns I need and replaces a table in my GBQ. basically what Iwant is two stages in a flow, if to create an output and then run a sql view on the output. am not sure if this is possible or even a right way to do this . I looked at option of maybe referencing the output in a new flow but not sure how that works. I hope this makes sense

[Trifacta_Alumni · Accepted solution]

Hi @Vahid Razaghzadeh?,

Thanks for clarifying. Perhaps you will find the following doc useful: https://cloud.google.com/dataprep/docs/html/Build-Sequence-of-Datasets_57344607

Let me know what you think.

Best regards,

Nathanael

[4ff3a001681946b94681]

This is really helpful. thank you so much for this