Search in the Data Hub Catalog

Last update: Edit

1 Introduction

Finding the right data to use in your app development is made easier using the search functionality in the Data Hub Catalog. The details of registered data assets can be viewed in the Asset details screen.

The Copy Data Source URI or Download contract buttons enable you to access the data source endpoints which you can use to integrate registered data sources into your enterprise applications.

You can start searching from the Data Hub Home page or click the Catalog tab to go to the Search pane and Asset Details screen.

Registered assets can be curated to add and edit further information (Catalog metadata) such as Tags, owners, and Descriptions and also set properties to the asset such as Discoverable and Validated to ensure that they are found for the appropriate uses.

This document describes the functionality of the Data Hub Catalog.

2 Details of Registered Assets

The Catalog displays the details of data sources, datasets, and attributes as provided in the published OData service contract that is used to register assets in the Data Hub Catalog. This section describes important properties of registered assets.

2.1 Versions

Every published OData service or data source (as they are known in the Catalog) has a version number, and apps that consume a datasource will consume from a specific version. Updates and changes to a service will be indicated by a change in the version number if good practice is followed by the data source originators. This may result in several versions of a registered data source available in the Catalog that will all be listed as separate items in the search results for the same-named data source.

The version of the selected data source is displayed in the Asset Details.

2.2 Environments

The Data Hub Catalog is a register of apps that are deployed to a particular environment and the services or data sources published from the apps that are deployed to the same environment. This means that each registered data source is a unique endpoint which is the location of the OData service contract that includes the version of the service running in a specific environment.

The environment also provides an indication of the quality of the dataset that is available. Shared datasets that are available from a production environment will have production-level data, while those in non-production environments (acceptance, development) could be populated with data that may not be reliable for building stable apps but be useful for development work.

Search results show the data source endpoints. Therefore, if a version of a service is deployed on both a test and acceptance environment, a search on the service name in the Data Hub Catalog will have two hits of the two endpoints.

2.3 Asset Descriptions

The description that is included as part of the published service metadata. This description can be further curated at the data source, dataset, and attribute level by owners and curators to provide further details of the exposed datasets and the associated data.

3 Search in the Data Hub Catalog

When searching in the Data Hub Catalog, the following fields are searched:

  • Data source or service endpoint: Name, Description, Tags
  • Application: Name
  • Dataset: Name, Description
  • Attribute: Name, Description
  • Association: Name

3.1 Searching for Assets

From the Data Hub Home page, you can search the Catalog in the following ways:

data hub home page

  • Type a search term in the search box and click Search (search strings can be a minimum of three characters and consist of alphanumeric characters)
  • Click one of the tags given in the Search suggestions
  • Click one of the services under Most Popular Services
  • Click the Catalog tab

Any of the above actions will take you to the Search screen.

3.2 Search Screen

The Search screen is divided into the search pane on the left, the asset details of the selected asset in the centre panel, and the asset metadata panel on the right.

search details

4 Search Pane

The collapsable Search pane is used to search for registered assets in the Data Hub Catalog:

search pane

Enter a search string in the Search area comprising a minimum of 3 alpha-numeric characters.

The wildcard * can also be used to imply an empty search but it is not necessary as search without specifying any search string will return all registered items.

The search is carried out asset metadata that includes the following:

  • all application names
  • data sources, datasets (or entity sets)
  • attribute
  • tags
  • descriptions

4.2 Filters

You can filter search results by environment type. By default, the Production environment filter is active and this restricts search to assets in the production environment.

To specify the environment type for the search, click Filter:

filter box

In the Filters dialog box, check the Environment Type that you want to restrict your search to and click, Apply Filters. The search results will only display hits for the specified search string in the checked environments.

The number of filters that are active for the current search is displayed adjacent to the filter:

filter active

Click Clear Filters to clear all the checked environments and click Apply Filters to see search results in all environments.

4.3 Search Results

The number of items satisfying the search criteria (search string plus filters) are shown above the search results list. Search results include assets that match the search string and satisfy the active filters. The order of the items presented in the search results will be a combination of the following:

  • Closest match to the search string
  • Popularity of the service (the number of connections)
  • Validated assets before non-validated items

When an item in the search results is selected, the Catalog tab displays the Details of the asset and the Landscape tab shows the network of connections and dependencies of the selected item in the Data Hub Landscape.

5 Selected Asset Details

When you click on an asset (data source or dataset) in the search results, the details are displayed in this panel.

5.1 Details of a Selected Data Source

The contract of the published OData service (the $metadata document) contains the definitions details of what is exposed in the service. This includes the metadata of the exposed datasets (or entity sets in Mendix Studio Pro) and their exposed attributes, associations, types, and accessibility. The contract metadata is displayed in the Data Source details along with any Catalog-curated metadata.

When a data source is selected in the search results, the following details are displayed:

service details

  • Application icon

  • Name of the data source

  • Non-discoverable icon – if the data source has been set to non-discoverable (by default, no icon indicates that it can be seen by all users)

  • Validated icon – if it has been set for the asset

  • Environment Name – where the app is deployed

  • Version number of the service

  • Connections – number of apps that consume the service

  • A description of the data source

  • All Datasets that are exposed in the data source (you can expand each one to see details of the attributes and associations)

You can perform the following actions from this screen:

  • Copy Data Source URI – click to copy the URI of the data source contract to the clipboard. This URI can be used to integrate the data source in other enterprise applications.
  • Share Data Source – click to copy the link to this asset detail page to the clipboard so that you can share it with others.
  • Download – retrieve and save the OData contract from the data source endpoint to your computer.
  • Copy Dataset URI – click to copy the URI of the dataset to the clipboard for use in other business applications.

5.2 Details for a Selected Dataset

When a Dataset is selected in the search results, the following details are displayed in the Search Details panel.

search details entity

5.2.1 General Information

The source and endpoint details of the dataset are displayed:

  • Dataset name

  • Part of – a link to the data source details page that the dataset is exposed in

  • Version number of the data source that the dataset is exposed in

  • Connections – the number of apps that consume this dataset

  • A description of the dataset

You can perform the following actions from this screen:

  • Copy Dataset URI – click to copy the URI of the dataset to the clipboard for use in other business applications
  • Share Dataset – click to copy the link to this dataset detail page to the clipboard so that it can be shared with others

5.2.2 Dataset Information

The Attributes tab lists the attributes that are exposed for the dataset in the OData service.

Under the Associations tab for each dataset, the associations are displayed:

associations info

  • Name – the name of the association that is exposed in the OData service contract.

  • Navigates to – the dataset the association is made with. Click the link to see the details of the associated dataset in the Catalog.

6 Metadata Panel

The metadata panel at the right of the asset details screen displays details from the OData service metadata contract and values that have been curated in the Data Hub Catalog:

metadata pane

6.1 Tags

The tags that have been assigned to the data source during curation.

6.2 Business Owner

A link to the business owner of the data exposed in the data source. Business owners can be added as a curation task.

6.3 Technical Owner

The technical contact of the app; by default this is the owner who registered the OData service.

For apps hosted in the Mendix Cloud, the Technical Owner is the app developer that deployed the app.

Technical owners can be changed as a curation task.

6.4 Discoverability

When a data source is registered, by default, it is Discoverable in the Data Hub Catalog. When this is set, all users can find it, view the details, and consume it. The owners of an asset and curators can set a data source as non-discoverable, which means it is not visible to users unless they are the owner or a curator.

See Curate Bar for changing Discoverability as the owner of the data source or curator.

The following discoverability values can be set:

  • Discoverable – all users of the Data Hub Catalog and Studio Pro can see and consume the asset provided they meet the requirements of the Classification
  • Non-Discoverable – the asset is not visible in the Catalog and only owners, Data Hub curators, and the Data Hub Admin can find, use, and curate the service.

6.5 Validated

Indicates if the data source has been Validated. See Curate Bar for changing Validated as an owner of the data source or curator.

6.6 Access Level

Displays the access classification of the data exposed by the service: end-users of the app will only be able to see the information must have the appropriate user role to access the data:

  • Public – classified as public and available to all users, internal and external to the organization
  • Internal – restricted to the members of the organization

6.7 Application

A link to the application from which the data source was published in the given environment.

6.8 Environment Type

The environment type indicates the quality and the status of the data that the exposed datasets connect to. The following environment types can be specified:

  • Production
  • Non-Production
  • Sandbox (the Mendix Free App environment)

7 Curate Bar

The Curate Bar is displayed in the asset detail screen if you are the owner of the selected asset or a curator:

called out curate bar

The following actions can be carried out:

  • Edit Metadata – edit information that is displayed in the Catalog for the asset:
  • Discoverable/Validated – set the discoverability of the data source, and set a data source or dataset as validated
    • Discoverable – all users of Data Hub and Studio Pro can see and consume the service in combination with the classification of the data
    • Not Discoverable – the service is not visible; only owners of the service, Data Hub curators, and the Data Hub Admin can access the service
    • Validated – indicates if the data source or dataset has been validated

For further details, see Curate Registered Assets.

8 Data Source and Dataset URIs

The data source URI is the location of the service contract of the data source – also known as the service endpoint. The endpoints of all exposed datasets (entity sets) are defined in the contract. From the details screen of the data source and dataset, you can copy the URIs to the clipboard by clicking the Copy Data Source URI and Copy Dataset URI, respectively. These URIs can be used for directly accessing the contract and resource in other BI applications.

9 Download the Metadata Contract of a Data Source

For a selected data source, you can click Download to download the OData service contract that is located at the data source endpoint. A .zip file that includes the all the files that make up the full metadata contract is generated and downloaded.

The resulting .zip file is named as follows:

DataHub_<service_name>_<service_version>_<technology>.zip

The string <technology> identifies the OData version (v3 or v4) in the file name.

For the following example:

download example

When you click Download the following file is downloaded: DataHub_SAP_Intelligence_1.0_OData4.zip

This zip file has the folder: DataHub_SAP_Intelligence_1.0_OData4 which contains the all the metadata files that define the service.

10 Viewing Search Results in the Data Hub Landscape

When an item is selected in the search results pane, you can click the Landscape tab to see the network of connections and dependencies for the selected asset. This provides a graphical representation to indicate the context and relevance of a selected item and the data for the exposed datasets.