Filters
Filtering data assets is simple and can be done on a feature definition level.
See the following example:
filters
filters
Filters are defined in the filters
property. This is an array that can get as many filter inputs as needed, where each input can be defined as a SQL statement or a YAML statement (type
: sql
/ field
).
When defining multiple filters
inputs, Lynk will chain these filters with an and
statement between them and then apply them : filter_1 and filter_2 and filter_3
type
type
Use the filters type
property to determine which method to use for the filter.
The options for type
are:
sql
field
sql
(type)
sql
(type)Use sql code to define how a data asset should be filtered.
When setting type: sql
, you will need to add the sql
property with the SQL definition of the relation.
In this example, we use SQL to filter the orders
data asset to retrieve only records where the order_date
field is in 2025 for the feature orders_count_2025
.
fields
(type)
fields
(type)Use simple YAML-style to define how a data asset should be filtered.
When setting type: fields
, you will need to add the fields
property with the fields and the operator that define the relation.
Example (fields
):
fields
):In the above example we filter the data asset db_prod.core.orders
to retrieve only records where the value
of the field
order_status is 'success'.
Using the filters
type fields
, the following properties should be specified:
field
field
The field name in the data asset, which we would like to apply a filtering logic on.
Operator
Operator
The operator to be applied on the field
, that defines the logic of which values we would like to filter in the results. The options for the operator property are:
is
is
Equals to.
"A equals
B" translates to "A =
B".
Usage example (is
operator):
Retrieves only records where the value
of the field
order_status is 'success'.
is_not
is_not
Does not equal to.
"A is_not
B" translates to "A !=
B" (or "A <>
B").
Usage example (is_not
operator):
Retrieves only records where the value
of the field
order_status is not 'success'.
between
between
Between two values
"A between
B and C" translates to "A >=
B and A <=
C
Usage example (between
operator):
Retrieves only records where the values
of the field
order_date are between '2024-01-01' and 2025-01-01.
Note that the order matters - the first value is the lower bound and the second value is the upper bound of the between
operator.
gt
gt
Greater than.
"A gt
B" translates to "A >
B".
Usage example (gt
operator):
Retrieves only records where the value
of the field
order_amount is greater than 100.
gte
gte
Greater than or equals to.
"A gte
B" translates "to a >=
b".
Usage example (gte
operator):
Retrieves only records where the value
of the field
order_amount is greater than or equals 100.
lt
lt
Lower than.
"A lt
B" translates "to a <
b".
Usage example (lt
operator):
Retrieves only records where the value
of the field
order_amount is lower than 100.
lte
lte
Lower than or equals to.
"A lte
B" translates "to a <=
b".
Usage example (lte
operator):
Retrieves only records where the value
of the field
order_amount is lower than or equals 100.
is_set
is_set
The value is not NULL.
"A is_set
" translates to "A is not null
".
Usage example (is_set
operator):
Retrieves only records where the value
of the field
order_amount is set. Meaning, order_amount is not Null.
Note that in the case of the operator is_set
there is no need to add the values
property.
is_not_set
is_not_set
The value is NULL.
"A is_not_set
" translates to "A is null
".
Usage example (is_not_set
operator):
Retrieves only records where the value
of the field
order_amount is not set. Meaning, order_amount is Null.
Note that in the case of the operator is_not_set
there is no need to add the values
property.
Values
Values
The values to accept.
Lynk will apply the operator
logic to the field
and the given values
. The records that will return are the records where that filtering logic returns "true" (see this example)\
Last updated