Smart Visual Query Builder for in-db (less tests, less time)

[saubert]

I noticed through the ODBC driver log that Alteryx doesn't care about the kind of base I precise. It tests every single kind of base to find the good one and THEN applies the queries to get the metadata info.

Here an example. I have chosen an Hive in db connection. If  I read the simba logs, i can find those lines :

Mar 01 11:37:21.318 INFO  5264 HardyDataEngine::Prepare: Incoming SQL: select USER(), APPLICATION_ID() from system.iota

Mar 01 11:37:22.863 INFO  5264 HardyDataEngine::Prepare: Incoming SQL: select USER as USER_NAME from SYSIBM.SYSDUMMY1

Mar 01 11:37:23.454 INFO  5264 HardyDataEngine::Prepare: Incoming SQL: select * from rdb$relations

Mar 01 11:37:23.546 INFO  5264 HardyDataEngine::Prepare: Incoming SQL: select first 1 dbinfo('version', 'full') from systables

Mar 01 11:37:23.707 INFO  5264 HardyDataEngine::Prepare: Incoming SQL: select #01/01/01# as AccessDate

Mar 01 11:37:23.868 INFO  5264 HardyDataEngine::Prepare: Incoming SQL: exec sp_server_info 1

Mar 01 11:37:24.093 INFO  5264 HardyDataEngine::Prepare: Incoming SQL: select top (0) * from INFORMATION_SCHEMA.INDEXES

Mar 01 11:37:24.219 INFO  5264 HardyDataEngine::Prepare: Incoming SQL: SELECT  SERVERPROPERTY('edition')

Mar 01 11:37:24.423 INFO  5264 HardyDataEngine::Prepare: Incoming SQL: select DATABASE() as `database`, VERSION() as `version`

Mar 01 11:37:24.635 INFO  5264 HardyDataEngine::Prepare: Incoming SQL: select * from sys.V_$VERSION at where RowNum<2

Mar 01 11:37:25.230 INFO  5264 HardyDataEngine::Prepare: Incoming SQL: select cast(version() as char(10)), (select 1 from pg_catalog.pg_class) as t

Mar 01 11:37:25.415 INFO  5264 HardyDataEngine::Prepare: Incoming SQL: select NAME from sqlite_master

Mar 01 11:37:25.756 INFO  5264 HardyDataEngine::Prepare: Incoming SQL: select xp_msver('CompanyName')

Mar 01 11:37:26.156 INFO  5264 HardyDataEngine::Prepare: Incoming SQL: select @@version

Mar 01 11:37:26.376 INFO  5264 HardyDataEngine::Prepare: Incoming SQL: select * from dbc.dbcinfo

Mar 01 11:37:26.522 INFO  5264 HardyDataEngine::Prepare: Incoming SQL: SELECT @@VERSION;

I can understand that when Alteryx doesn't know the kind of base he tries everything.. (eg : in memory visual query builder) but here, I have selected the Hive database and I have to loose more than 5 seconds for nothing.

[benstroud]

Beyond the efficiency issue, I'd like to point out another negative aspect of this feature. Our security/ops team had an adrenaline filled morning tracking down what appeared to be malicious probing of our database systems when these queries were noticed in our Athena logs. Were we compromised? No, it was just Alteryx duck-typing a configured database connection for some reason.

This seems like an antipattern to avoid and would be better to ask the user for the database type or programmatically get that information from the configured data source/driver. I'd like to see this pattern redesigned to avoid these probe queries. If that can't be done, please at least add a SQL comment to each probe query so it is clear in logs that Alteryx is doing the probes and not some malicious actor. Thanks

[simonaubert_bd]

One good idea would be to change the order :

1/according to the db type choosen

2/generic

3/others

[CristonS]

At this time we are unable to include this idea in our near future roadmap due to competing priorities; as such we're updating it to Not Planned. However, should this change or we become able to return to this idea in the future we'll be sure to update this status.

[simonaubert_bd]

Hello,

This idea gets some traction lately... Could you please have a loog again ?

Best regards,

Simon