Establish lineage/relationship between SAP HANA and Tableau (reports)
Issue
By default in Connect, we are unable to see data lineage between SAP HANA and Tableau. The standard Alteryx loaders (i.e. Tableau & SAP HANA) in version 2019.2 are unable to detect the HANA object link due to a misaligned naming convention (“::” vs “/”). This specifically occurs when Tableau dashboards are pointed to HANA synonyms (“::”) as opposed to the views (“/”).
Background
In SAP Hana, there are two types of objects – views (which are present in relevant schemas, e.g. “_SYS_BIC”) and synonyms. The synonyms are created for each view by default and always appear to be in the schema “PUBLIC”, however, this is not actually an explicit schema. According to SAP, public synonyms are the official access interface for the system views.
When loading the objects from SAP HANA, the object names contain the package name as well as the object name (view etc.). These two are differentiated with a “/” when loaded from the “_SYS_BIC” schema (e.g. “GSTP_VALIDATION/CA_01_C_PDA_GRA_001”), however synonyms from the “PUBLIC” schema are loaded with a differentiator of “::” (e.g. “GSTP_VALIDATION::CA_01_C_PDA_GRA_001”). An example below, where first row is the view and the second row provides the related synonym:
Thus, the lineage is not established when the SAP HANA objects from schemas are loaded (containing the “/”, and at the same time, the Tableau dashboards linked to the synonyms (“::”). This is because the object names are quite clearly different, and the loader doesn’t currently resolve the synonym to view translation (as per standard SAP HANA SQL parsing behaviour).
Loading problem analysis (using standard Alteryx loaders)
On SAP HANA side, when running the loader, user can (optionally) define schema name(s) they wish to load, however:
- When schema is set as “_SYS_BIC”, the loader only loads the views from this particular schema, with no synonyms.
- When schema is left as blank (i.e. no value entered), the loader loads everything, however in the “_SYS_BIC” schema, only synonyms are loaded and not the views.
- When schema is set as “PUBLIC”, the loader does not return anything - fails as there is no corresponding schema in SAP HANA (it is only virtual space, not an explicit schema)
The SAP HANA instance in question is very, very large and due to volume limitations, as well as the potential risk of performance degradation, it is not feasible to load everything.
When we’ve examined the H2 staging tables, we find that Connect has loaded (therefore is technically aware of) the synonyms from the “PUBLIC” schema but they are not ever loaded beyond the staging tables. This is obviously required if we wish to represent these relationships within Connect.
Idea/Fix:
- SAP HANA loader – it seems there is a problem of loading synonyms from the “PUBLIC” schema and the loader should be adjusted in order to allow this.
- Tableau loader – needs to be able to refer to synonyms defined in a SAP HANA data source, meaning whenever there is a “::” in object name and the schema name is blank (i.e. empty string representing the “PUBLIC” schema), then it should be defaulted to “PUBLIC”. This logic would ensure that a schema name is always present, even in cases where the schema name is retrieved as blank (which then would become “PUBLIC”).
- In case the schema name is left blank for any reason, it should be populated as “PUBLIC”.
- @DanH suggested to customize the Tableau loader as a temporary fix to replace the “::” with “/”, which is certainly possible but comes with a risk of incompatibility with future releases. Also the customers strong preference is to deliver a solution which only utilises standard loaders.