Data Assets
Data assets are tables and views from the underlying data source (mostly a warehouse). Data assets can be related to entities and they are being used for creating entity features.
Data assets YAML file
Data assets can be modified either via code or via Lynk Studio UI. The below example shows a YAML file for the data asset db_prod.core.orders
See the following example:
# db_prod.core.orders.yml
asset: db_prod.core.orders
key: order_id
business_key: []
defaults:
time_field: order_date
measures:
- name: count_orders
description: count of orders
sql: count(1)
- name: total_order_amount
description: sum of order amount
sql: sum({total_amount})
asset
asset
Data Asset name and location as it appears in your underlying Data Warehouse (db.schema.name
)
key
key
The asset key field (primary key). It is important to state the correct data asset key in order to avoid duplications and errors. Lynk will automatically find and suggest data asset keys after the discovery process is completed.
business_key
[optional]
business_key
[optional]Sometimes the key is just "id
".
In such cases, some combination of other fields might represent the level of granularity of that data asset, and make more sense business wise. We call this combination of fields the business_key
of that asset.
defaults
defaults
Holds the defaults for the data asset. See default time field for example.
time_field
[optional]
time_field
[optional]It is recommended to choose a default time field for each data asset. Lynk will use the default time field to aggregate and filter time-related queries on features.
For example, if we have a data asset for orders db_prod.core.orders and we set the default_time_field
to be order_date, Lynk will use this field for time-based aggregations by default.
Measures
Measures
Measures are reusable components that define how the data asset fields should be aggregated. Lynk applies the measure logic once a feature of type metric feature is created and consumed.
In practice, measures are definitions of how aggregate functions should be applied to fields;
# db_prod.core.orders.yml
measures:
- name: total_order_amount
description: sum of order amount
sql: sum({total_amount})
Name
Name
Give the measure a name. This will be used when creating metric features and also will be shown on the Studio UI. It is recommended to give measures informative names that indicate their purpose.
Description
[optional]
Description
[optional]Describe the measure. It is recommended to give measures informative names that indicate their purpose - for other team members to be able to reuse the measure and for AI apps as well.
SQL
SQL
The measure definition. It should be composed of an aggregate function and a field. It is possible to chain multiple aggregate functions and / or multiple fields, just like you would do on plain SQL when needed.
Some more examples:
# db_prod.core.orders.yml
measures:
- name: count_orders
description: count of orders
sql: count(1)
- name: total_order_amount
description: sum of order amount
sql: sum({total_amount})
- name: successful_order_amount
description: sum of successful orders amount
sql: sum(IFF({order_status} = 'success', {total_amount}, 0))
Last updated