This site uses different types of cookies, including analytics and functional cookies (its own and from other sites). To change your cookie settings or find out more, click here. If you continue browsing our website, you accept these cookies.
Quick Guide: API Catalog in Connect
Do you use APIs for your work? Based on the information published online, in 2019 the number of APIs exceeded 22,000 entries. APIs and their endpoints are one of the very popular data sources and are used in Alteryx workflows as well. Information about used APIs is now available in Connect since version 19.4. The information is extracted from workflows’ log and cataloged.
To use the API catalog, you must install Connect version 19.4 as well as Server version 19.4. The reason behind that is the way the APIs are harvested. They are not harvested from the workflows directly but from the log in the job log. It means they are generated from the data logged on the Server. Lineage is automatically created between a workflow and an API endpoint. Thus, it's necessary to run the workflow at least once on the server to get the APIs from it.
API sites are available in Data Sources
Essential Info about API Catalog
To access the list of API endpoints, go to main menu > Data Sources > API sites. API is an asset type under Data Sources.
Note: API sites folder is by default visible only for Administrators group. The permissions can be easily modified by an Administrator. Navigate to see the permissions of the API sites folder by clicking the Action drop-down arrow and select Permissions. A window with a permission table pops up. You can change the permission under the Inherited permission table where you can see the list of all actions. Click in any field to add a user group or individual users to a permission level. For more information see Permissions .
The following information about an API endpoint can be viewed:
You can add a description for an asset, such as asset name, link to the asset page, or link to the API documentation. Note: API documentation is generated automatically.
Action: Message to a resource to perform a certain operation, such as GET, POST, PUT, DELETE.
URIs: Uniform Resource Identifier. Address which identifies a resource.
Parameter/Attribute Values: Options you can pass with the endpoint to influence the response.
Attachment: You can upload more information for the specified API.
All relationships, including data lineage, can be viewed using Nexus. Click to Open nexus, a navigable map that shows a web of connections between assets.
Details of an API endpoint
When to Use API catalog?
As a developer, you can use the API catalog to find the API you need in your workflow. The catalog provides detailed insight into how API endpoints are used in a workflow and give visibility into a workflow’s lineage.
As a user, you can check where - in which workflows - the API endpoint is used. Check Relationships to see the lineage.
Example of the Relationships section of an API endpoint
API catalog is also mentioned in the following articles: New in Connect 19.4 and Release Notes 19.4.
Metadata Loader Categories
In this article you'll find detailed information about the loader categories and which exact information is extracted by each of the loaders we support. Each type differs in the object hierarchy, structure of the objects, and detail of the objects.
There are five types of the metadata loaders:
Visualytics (Report) Loaders
Database loaders are used to harvest metadata about database objects such as database server (RDMS), catalog, schema, table/view/procedure, column. An example can be an Oracle database with all available objects, descriptions, and relationships within the objects in the database. Connect supports database metadata loader for these technologies: Teradata, Snowflake, Apache Hive, IBM DB2, Exasol, Oracle, Microsoft SQL Server, MySQL, SAP HANA, Amazon Redshift, HP Vertica, PostgreSQL, Apache Impala.
File loaders are used to harvest metadata about file systems such as file bucket or folder with subfolder structure, individual files, sheets (when available e.g. on xlsx file), and columns on files you can parse for the structural information (xls(x), csv, yxdb, avro …). For files with structure you can store also profiling information such as number of rows, not nulls, blanks, uniqueness, and average length. Connect supports file metadata loader for the following technologies: Amazon S3, Apache Hadoop, Microsoft Azure Data Lake, Files loader (windows shared drives).
Visualytics (Report) Loaders
Visualytics are used to harvest metadata from reporting platforms such as report workbook, worksheet, datasource, and detailed columns. Connect supports visualytics metadata loader for these technologies: Tableau, Microsoft PowerBI, QlikView, Qlik Sense.
Workflow loader is used to harvest metadata from the Alteryx Gallery. Connect supports only AYX Gallery Loader.
This category describes loaders that don't belong to any of the above-mentioned categories, such as:
Alteryx Promote loader is used to harvest metadata about Promote models published on a Promote server.
Microsoft Azure Data Catalog loader is used to exchange metadata from Azure Data Catalog about databases, files, and business terms.
Salesforce loader (without data profiling) is used to harvest metadata from the Salesforce application.
Overview of Available Objects for Each Loader Type
db server, catalog, schema, table / view / procedure / column
file bucket (AmazonS3) / folder, file, sheet, column Possible data profiling (additional information about features and content of the files).
server, site, project, workbook, worksheet, datasource, column Tableau: site, project, workbook, worksheet Microsoft PowerBI: workspace, workbook Insight QlikView: folder, Qlik data files (data profiling - .qvd, .qvx); report, worksheet; Qlik Sense: stream, workbook, worksheet, report objects (names and types of charts, dimensions, measures)
workflow, input, output
Microsoft Azure Data Catalog: glossary, databases, files (without data data profiling) Alteryx Promote loader: predictive model
Salesforce loader: Salesforce tables
How to Use and Configure Loaders
More information on how to use and configure the metadata loaders you can find on the following pages:
The Alteryx Online Help pages Schedule Metadata Loaders and Load Metadata into Connect specify how to use the metadata loaders from Designer.
Now you can run metadata loaders directly in Connect by selecting Connections in the Admin Menu. You have the possibility to start the harvest immediately or to set up a regular harvest. Go to Admin Menu > Connections to configure and run the metadata loaders. For more info see: How To: Fast Deployment of Metadata Loaders in Connect.
Overview of available loaders while using the fast deployment.
Have you ever wondered what are the limits of Connect? How many entries can you store in Connect? Great questions! In fact, no hard limit on the number of entries is set. The number around 2M entries is the number on which the performance tuning in the 2019.1 release was done.