This site uses different types of cookies, including analytics and functional cookies (its own and from other sites). To change your cookie settings or find out more, click here. If you continue browsing our website, you accept these cookies.
Quick Guide: API Catalog in Connect
Do you use APIs for your work? Based on the information published online, in 2019 the number of APIs exceeded 22,000 entries. APIs and their endpoints are one of the very popular data sources and are used in Alteryx workflows as well. Information about used APIs is now available in Connect since version 19.4. The information is extracted from workflows’ log and cataloged.
To use the API catalog, you must install Connect version 19.4 as well as Server version 19.4. The reason behind that is the way the APIs are harvested. They are not harvested from the workflows directly but from the log in the job log. It means they are generated from the data logged on the Server. Lineage is automatically created between a workflow and an API endpoint. Thus, it's necessary to run the workflow at least once on the server to get the APIs from it.
API sites are available in Data Sources
Essential Info about API Catalog
To access the list of API endpoints, go to main menu > Data Sources > API sites. API is an asset type under Data Sources.
Note: API sites folder is by default visible only for Administrators group. The permissions can be easily modified by an Administrator. Navigate to see the permissions of the API sites folder by clicking the Action drop-down arrow and select Permissions. A window with a permission table pops up. You can change the permission under the Inherited permission table where you can see the list of all actions. Click in any field to add a user group or individual users to a permission level. For more information see Permissions .
The following information about an API endpoint can be viewed:
You can add a description for an asset, such as asset name, link to the asset page, or link to the API documentation. Note: API documentation is generated automatically.
Action: Message to a resource to perform a certain operation, such as GET, POST, PUT, DELETE.
URIs: Uniform Resource Identifier. Address which identifies a resource.
Parameter/Attribute Values: Options you can pass with the endpoint to influence the response.
Attachment: You can upload more information for the specified API.
All relationships, including data lineage, can be viewed using Nexus. Click to Open nexus, a navigable map that shows a web of connections between assets.
Details of an API endpoint
When to Use API catalog?
As a developer, you can use the API catalog to find the API you need in your workflow. The catalog provides detailed insight into how API endpoints are used in a workflow and give visibility into a workflow’s lineage.
As a user, you can check where - in which workflows - the API endpoint is used. Check Relationships to see the lineage.
Example of the Relationships section of an API endpoint
API catalog is also mentioned in the following articles: New in Connect 19.4 and Release Notes 19.4.
We're constantly working to improve your Connect experience. Find out what new features, improvements, and bug fixes we released with this version of Connect! Here is a summary of what has changed in 2019.4.
Notify on Mention / Dislike / Decertify
A notification will go to the user that was mentioned on an asset. Thanks to this feature the users can see where they were mentioned and respond easily. Also, the actions Dislike and Decertify will now trigger a notification.
Connect has been upgraded to OpenJDK11 for security and performance improvements.
Tableau Loader with New GraphQL API
We’ve added support for new Tableau GraphQL API, so you can load column-level lineage into Connect (from Tableau worksheets columns into data source columns).
Using the Tableau GraphQL API in Connect allows us to display the data source fields of a worksheet and their details that built every single field on the list (such as servers, databases, schemas, tables, and columns). All the information can be tracked by clicking a link associated with each of the data source fields.
To view this information in Connect, go to your Tableau reports, click the desired report, and the Datasource fields section.
Uploaded File Types Restricted by Default
Uploaded file types are restricted by default. These restrictions can be changed anytime by an administrator. To view and change the file types list, go to Administration > Connect Configuration > Instance Settings.
QlikView Metadata Loader Faster Parsing
We’ve improved the speed of parsing of QlikView files by smarter way of detection of metadata headers in QlikView files, so just metadata are read over the network resulting in faster harvesting especially for larger datasets.
Connect Performance Improvements
We’ve made many performance improvements regarding the speed of metadata loading, system startup, asset detail rendering, and removing inactive versions in bulk.
Asset Identity Switch
In addition to other improvements, we have added Asset Identity switch for columns. Administrators can now decide if columns will be separate assets or parts of the table asset (default). This feature improves not only performance but will bring significant load and rendering improvements as well.
To access the Asset Identity switch, go to Administration > Asset Configuration > Asset Identity.
We have added brand new API Catalog which allows you to automatically catalog APIs and their endpoints from your workflows. This feature is by default available for administrators only but it's up to them to share the API Sites folder with other users or groups.
Go to main menu > Data Sources > API Sites.
API Catalog has 2 layers. It’s listing all API sites (e.g. Tableau, Salesforce) and their endpoints (e.g. getAllDatasources).
Support for Multiple Tableau and Alteryx Servers
We’ve been following up on our customers’ feedback and added support to load multiple different servers for Tableau and Alteryx into Connect.
We've made a number of enhancements to our Loaders with 2019.4:
You can now upload metadata to Connect from Alteryx Promote.
DB2 Loader was redesigned to support different DB2 platforms at once.
S3 Loader was redesigned to speed up the loading and handle large volumes of files.
For all database loaders we have added Column references which display all column level mappings.
Here is a list of some issues fixed in the latest release. To check a complete list of fixed and known issues please see the Connect 2019.4 Release Notes. Resolved: Custom Fields Lost after Reloading Asset
Custom fields of an asset were lost once that asset was reloaded. Now after re-running the metadata loader, the custom fields stay as they are without being removed. Resolved: Admin API Not Enabled for the Fast Deployment in Gallery
Admin API was not enabled for the fast deployment in Gallery. There was no error message when the Test connection or Harvest button was pressed. Now the error message appears on all appropriate screens when Admin API is not enabled. Resolved: Only Superadmin Is Able to See the "Download Service Provider Metadata" Button
Only the superadmin was able to see the "Download Service Provider Metadata" button on the SSO configuration page. This issue was resolved by allowing this function for the members of admin role. Resolved: Only Superadmin Is Able to Perform the H2 Backup
Only superadmin was able to see and do the H2 backup in Administration Console. This issue was resolved by allowing H2 backup for the members of admin role.
Metadata Loader Categories
In this article you'll find detailed information about the loader categories and which exact information is extracted by each of the loaders we support. Each type differs in the object hierarchy, structure of the objects, and detail of the objects.
There are five types of the metadata loaders:
Visualytics (Report) Loaders
Database loaders are used to harvest metadata about database objects such as database server (RDMS), catalog, schema, table/view/procedure, column. An example can be an Oracle database with all available objects, descriptions, and relationships within the objects in the database. Connect supports database metadata loader for these technologies: Teradata, Snowflake, Apache Hive, IBM DB2, Exasol, Oracle, Microsoft SQL Server, MySQL, SAP HANA, Amazon Redshift, HP Vertica, PostgreSQL, Apache Impala.
File loaders are used to harvest metadata about file systems such as file bucket or folder with subfolder structure, individual files, sheets (when available e.g. on xlsx file), and columns on files you can parse for the structural information (xls(x), csv, yxdb, avro …). For files with structure you can store also profiling information such as number of rows, not nulls, blanks, uniqueness, and average length. Connect supports file metadata loader for the following technologies: Amazon S3, Apache Hadoop, Microsoft Azure Data Lake, Files loader (windows shared drives).
Visualytics (Report) Loaders
Visualytics are used to harvest metadata from reporting platforms such as report workbook, worksheet, datasource, and detailed columns. Connect supports visualytics metadata loader for these technologies: Tableau, Microsoft PowerBI, QlikView, Qlik Sense.
Workflow loader is used to harvest metadata from the Alteryx Gallery. Connect supports only AYX Gallery Loader.
This category describes loaders that don't belong to any of the above-mentioned categories, such as:
Alteryx Promote loader is used to harvest metadata about Promote models published on a Promote server.
Microsoft Azure Data Catalog loader is used to exchange metadata from Azure Data Catalog about databases, files, and business terms.
Salesforce loader (without data profiling) is used to harvest metadata from the Salesforce application.
Overview of Available Objects for Each Loader Type
db server, catalog, schema, table / view / procedure / column
file bucket (AmazonS3) / folder, file, sheet, column Possible data profiling (additional information about features and content of the files).
server, site, project, workbook, worksheet, datasource, column Tableau: site, project, workbook, worksheet Microsoft PowerBI: workspace, workbook Insight QlikView: folder, Qlik data files (data profiling - .qvd, .qvx); report, worksheet; Qlik Sense: stream, workbook, worksheet, report objects (names and types of charts, dimensions, measures)
workflow, input, output
Microsoft Azure Data Catalog: glossary, databases, files (without data data profiling) Alteryx Promote loader: predictive model
Salesforce loader: Salesforce tables
How to Use and Configure Loaders
More information on how to use and configure the metadata loaders you can find on the following pages:
The Alteryx Online Help pages Schedule Metadata Loaders and Load Metadata into Connect specify how to use the metadata loaders from Designer.
Now you can run metadata loaders directly in Connect by selecting Connections in the Admin Menu. You have the possibility to start the harvest immediately or to set up a regular harvest. Go to Admin Menu > Connections to configure and run the metadata loaders. For more info see: How To: Fast Deployment of Metadata Loaders in Connect.
Overview of available loaders while using the fast deployment.
Have you ever wondered what are the limits of Connect? How many entries can you store in Connect? Great questions! In fact, no hard limit on the number of entries is set. The number around 2M entries is the number on which the performance tuning in the 2019.1 release was done.
Issue - Unable to add user(s) to a group in Alteryx Connect
When trying to add a user to a group in Connect, the user is not found and therefore cannot be added to the group.
The user does not have an email address set.
Navigate to the user in Connect under People
Click the Edit button in the top right
Enter the user's email in the Email field
You should now be able to add the member to a Group.
Managing Users and Permissions in Connect
Authentication Methods for Connect
Alteryx Connect is great at finding data assets that exists within enterprises which helps data workers to shorten their journey to data-driven insight. One of the steps to refresh the metadata is to schedule the Metadata loaders. In this process, the data assets can be created, changed or removed. To take a look at this change log, we can take a look the backend of Connect - the H2 database.