Basically i have created a flow as below
what I want to do now is to take the result from the create_applicants _in_clean and run a GBQ View Query on it which will copy the d=table into a new dataset called Prod
Any help is apprecaite it as I am new to Dataprep
Solved! Go to Solution.
Hi @Vahid Razaghzadeh?,
So, your Trifacta job publishes to a table in BigQuery. Are you trying to copy this table, or to read it back into Trifacta? (Both?) Are you trying to use non-Trifacta tools to take a slice of the published table as a new dataset? Apologies, I'm a little confused!
Note that Dataprep never actually modifies a source: every time you [re-]import a datasource into Trifacta, or indeed add a dataset to a flow, you are essentially creating a copy to work with.
I hope this helps...? Could you perhaps clarify what you are trying to do?
Cheers,
Nathanael
Dear Nathanael
Thank you for the reply. Just to clarify below is what I need to do.
I have created a recepie in the flow which cleans the data as I need. What I want to do next is to get the output of the recepie and apply a script which I have created in GBQ. What this script does selects the columns I need and replaces a table in my GBQ. basically what Iwant is two stages in a flow, if to create an output and then run a sql view on the output. am not sure if this is possible or even a right way to do this . I looked at option of maybe referencing the output in a new flow but not sure how that works. I hope this makes sense
Hi @Vahid Razaghzadeh?,
Thanks for clarifying. Perhaps you will find the following doc useful: https://cloud.google.com/dataprep/docs/html/Build-Sequence-of-Datasets_57344607
Let me know what you think.
Best regards,
Nathanael
This is really helpful. thank you so much for this