Filtering a SQL input dynamically

Question

Hi community,

I am pulling data from two Snowflake tables that are massive. I get data from Table 1, and filter it down to only a few hundred IDs.

Is there a way of using these IDs in the SQL query of the Input Data tool that gets Table 2? Right now I'm pulling in a large chunk of Table 2 in, and then joining to Table 1. This works, but it's pretty slow given the size of Table 2. Is there a way of constraining Table 2's input to just be the output from the workflow built off Table 1? (Note that this is dynamic, so it's also not ideal to just copy and paste"

Thanks!

air0storm · Answer

Hi Thableaus,

Posting here for future people finding the thread. I got the Dynamic Input Working, it works fine with a OBDC. However, when you use a Dynamic Input it queries the database separately for each row of your input value. So instead of 1 big pull, the tool was now doing 200 small pulls, which ended up being much slower.

Therefore, I'll take a look at In-Db, but otherwise will just live with the slowness.

Thableaus · Answer

@air0storm I think it should work fine with the Snowflake database. I can give it a try on my end and let you know.

And yes, In-DB tools don't change any underlying data unless you use the Write In-DB tool.

Cheers,

air0storm · Answer

Thanks for the reply. 2 quick questions:

1) I watched this video: https://www.youtube.com/watch?v=ELfKYatOJZM. I am limited to using an ODBC type database. The video says that you can't use an "update where clause" function with those databases. Is that true?

2) I haven't ever used the indb tools (I'm new to connecting to datasources - I've used extracts til now). There's no risk that I change the underlying table, right? That's only if I am outputting data back there?