Using AMDL
ARIC Model Definition Language (AMDL™) is used to specify rules and logic within Real-Time Decisioning. It is a declarative language for specifying state updates and executions on each event that passes through the system.
AMDL can be used to create business rules and workflows. These are two different approaches to fraud and financial crime risk analytics, but both employ broadly the same syntax and rely on the same underlying concepts such as events, entities, and state. For more information including definitions of these terms, see Terminology in Real-Time Decisioning in the Dashboard.
Real-Time Decisioning is an event-driven system, where every event contains a reference, such as an ID field, to one or more entities of different types—for example, a merchant and a consumer. A Real-Time Decisioning event typically represents some real-world event, such as a card transaction.
For detailed information on the AMDL elements—including data types, annotations, and methods—see AMDL Reference.
AMDL expressions
Copy section link
AMDL is made up of expressions, which are the basic building blocks of both business rules and analytical workflows. The general form of any AMDL expression is as follows:
An AMDL expression consists of a scope, name, definition, and one or more optional annotations.
-
Scope: The type of expression. Different scopes are available depending on whether the expression is a business rules or workflow. Some scopes allow you to define expressions, while others only allow you to reference values.
-
Name: The name by which the definition should be referenced in output and other AMDL expressions. A name must be unique within a scope, but can be shared by AMDL expressions of different scope. For instance,
state.fooandvalues.foocould both distinctly exist. -
Definition: The value to which the AMDL expression evaluates if run.
-
Annotation: Information about the AMDL expression. Annotations are optional, and an AMDL expression can have zero, one, or many annotations.
Event data
Copy section link
The Real-Time Decisioning system is event driven. Events within the system must match a defined JSON schema. If a posted event fails to validate against this schema, it is rejected by Real-Time Decisioning. The following example event illustrates the structure of the JSON data and provides some examples of data types that can occur in an event, including strings, numbers, datetimes, dates, and booleans.
Any AMDL expression can refer to data contained in the event payload using the event.fieldName syntax; for example, the value for the eventTime field in the current event could be referenced as event.eventTime.
If there are several nested levels of JSON objects within the event, they can be accessed in a similar way to JSON objects, using periods to denote level transitions.
For example, to reference the issuer field in the example transaction event above:
Alternatively, you can use the square-bracket map accessor syntax to access one or more levels—for example:
You may need to use this syntax if the property names are reserved AMDL keywords such as state or values, or if the field names begin with numbers:
Note
This cannot be used for root-level fields within the event data, as the AMDL parser expectsevent. at the beginning of any reference to event data.
Tags in output
Copy section link
The purpose of creating AMDL expressions is to trigger effects and actions when an event or a series of events matches a certain set of criteria.
You can use a tag, such as @tag(review) or @tag(suspicious), to append information to the transaction payload, which can then drive any functionality your company has implemented downstream through API integrations.
This allows you to use real-time data within transactions for purposes such as sending an SMS message to a cardholder to confirm that they meant to make a transaction, or using a tag to make any arbitrary downstream API calls.
Business rules can trigger tags using annotations. Analytical workflows can trigger the same actions through scorecards, which can be used to implement a variety of effects. For more information, see Business rule basics and Workflow basics. Business rules and analytical workflows can also produce other outputs, including scores, and can suppress tag output. For more information on other outputs from business rules, see Other business rule effects.
For reference information on tags, see @tag(<namespace>="<tagValue>") in
AMDL Reference.
Comments
Copy section link
A comment provides information about an expression to other users.
Comments can be written using //… and /, which are ignored by the parser:
-
//causes everything remaining on that line to be interpreted as a comment. -
/*causes everything to be treated as a comment until a closing*/is found.
Conditions
Copy section link
A condition is a boolean expression that consists of an operator and one or more operands and evaluates to either true or false. For a list of AMDL operators, see Operators in the AMDL Reference.
The following example shows a simple condition that is true if the value in the country field in the event is US:
Conditions can also be numerical comparisons, such as event.amount.baseValue > 100.
A boolean variable, or something that returns a boolean value, can be used in place of a condition.
For example, using the example event from the Event data section, this condition can be rewritten to check if a transaction was accepted:
Or you could simply write:
You can link conditions using the boolean operators AND, OR, and NOT.
In AMDL, these are represented by &&, ||, and !, respectively.
For example, the following condition is true if the transaction event took place in Duluth, US—that is, the country is US and the city is Duluth:
The following condition is true if the event took place in a city called Duluth or in the US, or both:
The following condition is only true if the event took place neither in the US nor in a city called Duluth:
Parentheses ensure that comparisons are executed in the correct order.
In the above example, they ensure that the combination of conditions using OR is evaluated first, then the NOT.
Parentheses are also needed when you combine different boolean operators, such as AND and OR, to ensure the logic is evaluated correctly.
For example, this condition is true if the city is Duluth and the country is US or UK:
However, this condition is true if the city is Duluth, US (city is Duluth, country is US), or the country is UK (if the country is the UK, the city has no effect):
In business rules, rules are usually written using conditions. A rule is a condition that, if it evaluates to true, can trigger actions such as those defined in tags. For an example, see Example 1: A simple rule.
In analytical workflows, a rule set can be executed or not executed based on whether an AMDL condition evaluates to true. For an example, see Example 4: Introducing conditions and durations.
Conditions can also be used in the main AMDL block of a rule set, and using scorecards these can be used to trigger effects such as those defined in tags. For more information, see Workflow basics.
State
Copy section link
A variable’s state provides information accumulated over that entity’s history The state of variables affects their persistence during expression execution: either entity or global.
Entity state
Copy section link
Every entity that is processed by Real-Time Decisioning in an event has its own AMDL state, which is that entity’s AMDL-specific storage area within the Real-Time Decisioning behavioral profile database. This state persists between events. For example, you could store a consumer’s registration time, which is then accessible when processing all future events with that consumer. Alternatively, you could build a history of all the transaction amounts of a merchant over the past month, and access that information in future events. All state variables can be updated by values in an event and in other state variables. AMDL state is automatically initialized by the system the first time it is needed per entity.
State variables are specified using an update expression—that is, an expression providing a single value that is then used to update that state variable. The state variable is updated with the value of the update expression on every event.
Every state definition requires a single update expression, but how the value of that expression is stored depends upon the type of the state variable. The default type of a state variable is a single value. The variable contains the last value it was updated with and, each time the state is updated, that value is overwritten with a new one.
Annotations can be used to modify the variable type, as described in Annotations.
Global state
Copy section link
Comparing an entity’s behavior to fixed threshold values is one way to detect anomalous events and assess fraud and financial crime risk. However, sometimes you need to compare against a background or average value that describes the behavior of the population as a whole. For example, in many cases there might be seasonal variations in average transaction values and volumes, so that comparing transaction values to a fixed threshold might generate a much higher volume of false positives during periods of increased spending such as the run-up to Christmas or Black Friday.
This is where global state becomes useful. A global state expression defines a single variable that is updated by events for any entity of a given type so that it stores information derived from the entire population of entities. One of the main uses of global state is to compare the activity of one entity with a threshold or comparison value derived from the behavior of the whole population.
Note
Global state is not updated as frequently as entity state - for this reason, when referring to a global expression, the value might be up to several seconds out of date. For this reason, globals can be used very effectively to track rolling averages and similar aggregate values, but not to track values or collections that need to be completely up to date at all times. Therefore, it is highly recommended that all global variables are declared as an aggregate type of some sort—for example, a collection or a rolling average declared as an aggregate type.
State syntax
Copy section link
Entity state uses the state scope, and global state uses the globals scope.
The exact syntax for state and global expressions depends on whether you are defining this state in the context of business rules or workflows.
State in business rules
Copy section link
For business rules, the syntax for entity and global state is the same as for any other AMDL expression. For example, to store the timestamp of the last event for each entity:
This stores the state of entities of the type defined for this expression.
To refer to the stored value of this state expression for the entity of the relevant type in the current event, you can simply use the scope state and the expression name.
For example, to refer to the time of the last event for the current entity:
Important
In business rules, when an event is being processed, state is updated after other AMDL expressions have been evaluated. This means that a rule that references state uses the value of the state as it was before the current event was processed. In fact, even when referenced within another state update, a state variable is accessed as it was before the current event. This is necessary to preserve well-defined expression evaluation so that it is not dependent upon microsecond variations in the processing speed of each rule. However, this does not diminish the computation power of AMDL because the set of prior states plus the event contents comprises the complete known information about that entity.Global state expressions are defined in a similar way—for example, to store a rolling average of transaction values across all merchants, this expression could be defined under the merchant entity type:
This expression uses the @rollingAverage annotation.
See Annotations for more information on using annotations to modify how state is stored.
This annotation, applied to entity or global state variables, keeps a track of the rolling average within a specified time period. This time period must be specified. It cannot keep track of a mean value over all time because it is implemented as an exponential moving average for performance and storage purposes. In general, with global state you want to know about overall trends across the entity population.
State in workflows
Copy section link
For workflows, which relate to one or more event types, you can define state expressions for different entity types, enabling each workflow to update the state for any of the entities present in the relevant event type or types.
This means that you need to define the entity type that a state update expression relates to in the definition of the expression itself.
The basic syntax for defining and referring to entity state in workflows is state.<entity type>.<state expression name>.
For example, to store the timestamp of each event in the state for the customer entity type:
To refer to this state expression, you can use the same syntax, including the entity type:
Note the use of .single().
This is a method that takes a collection with a single element and returns that element as a single value.
For a complete list of methods, see Methods in the AMDL Reference.
This is necessary because state.<entity type>.<state expression name> returns a collection containing the value of the state expression for each entity of the specified type in the event being evaluated.
Usually there is only one entity of each type, so you can use the .single() method to flatten that collection to a single value.
If there are multiple entities of a given type present in a particular event type, you may need to use other strategies to select the state of the appropriate entity.
See Rule references.
Global state works differently.
Because the globals scope returns a single value, there is no need to use the .single() method to select one value from the output.
However, it is still necessary to define the entity type that the expression relates to within the expression itself—global state is referenced using the syntax globals.<entity type>.<name> to store a global variable for the specified entity type.
For example, to store a rolling average of transaction values across all merchants:
This could then be referenced as globals.merchant.averageAmount.
Important
In analytical workflows, state (entity state and global state) is only updated at the very end of each rule set evaluation. That means that if you refer to a state expression that is defined in the same rule set, the value used in the evaluation of the current event will be that which has been stored from previous events. If you want the state value to be updated by the current event before you reference it in the same analytical workflow, you must define the state in one rule set and then use it in the following rule set in the sequence. This allows the state to be updated. The following rule set then receives these updates and executes accordingly.
Annotations
Copy section link
Many annotations can be used to change their value when an expression is evaluated or to modify the effects of an expression. Several can be used in both business rules and analytical workflows, but some can be used only in business rules. The examples in Business rule basics and Workflow basics introduce some of the most important annotations that can be used in this way. You can use annotations in several ways, including:
-
Determine how a state or global expression stores data. This allows you to create expressions that store data in several ways, such as:
-
Store a collection of values. See Collections.
-
Store a rolling average. This annotation changes a state or global expression from one that stores a single value into one that stores an exponentially weighted moving average: For example,
@rollingAverage(12h). See@rollingAverage(<duration>)in the Annotations section of the AMDL Reference. -
Store a value the first time the expression is updated.
-
Set a default value for an expression. See
@defaultValue(<defaultValue>)in the Annotations section of the AMDL Reference.
-
-
Change what happens when a rule is evaluated (business rules only). For business rules, you can use an annotation to trigger an effect such as a tag to output a risk score or to suppress tags to prevent them from being generated. For analytical workflows, the same actions can be triggered using scorecards and effects.
-
Change when an expression is evaluated (business rules only). For business rules, the
@eventTypeannotation can be used to limit the evaluation of an expression to a one or more event types. For analytical workflows, this action is performed when a workflow is mapped to one or more event types. -
Provide additional information to other users (business rules only). Annotations allow you to add commentary and descriptions to your business rule expressions.
For a complete list of annotations, see Annotations.
Static values
Copy section link
Use the values scope to define static values, which are constants that do not change unless a user edits the expression. These values can be referenced in other expressions. For example, to define a threshold value that can be used in other expressions:
To reference the value in another expression, use the standard reference syntax—for example, values.<nameOfExpression>.
Typically, static values are used to prevent repetition of a value across multiple AMDL definitions, and to allow it to be easily changed.
For instance, you might have a list of countries that is shared by several rules, and so store it in a values definition to allow easy modification.
Transient variables
Copy section link
Transient variables are AMDL expressions that are recalculated for each event and not stored for future events.
These variables are declared in a similar way to entity state, but with the scope var.
For instance, you could calculate the exchange rate used in each event by finding the ratio of the value in the original currency to the value in the base currency:
Other AMDL expressions, in the var scope and other scopes, can reference the results of these calculations using the standard reference syntax—that is, var.<nameOfVariable> as shown in the following example:
In business rules, the use of transient variables can help organize a set of expressions that all contain similar expressions with slight variations, or that contain references to the result of a common calculation.
For example, many expressions might refer to the exchange rate defined above as var.fxRate.
In workflows, transient variables are important because they provide the easiest way to calculate risk scores and other values for use in scorecards. For a simple example, see Example 1: A simple rule set.
To create a scorecard that makes use of the transaction amount, you would need to create a transient variable that stores that value from the event data:
The values of transient variables are available to other expressions and scorecards within the same rule set, but not to other rule sets or workflows. Only state and global variables and support variables can be updated in one rule set and referenced in another within the same workflow, and only state and global variables can be defined in one workflow and referenced in another. For more information, see Workflow basics.
Conditional assignment
Copy section link
For many use cases, you need to conditionally assign a value or update a state expression. For example, you might want to store the value of the most recent accepted transaction, the time of the last unsuccessful login attempt, or the beneficiary of the most recent payment over $1000 in value.
AMDL uses the ? conditional operator for conditional assignment.
This operator acts like an if/then statement in other languages, providing a way to evaluate an expression only if a certain condition is true.
The ? operator must be preceded by a boolean expression—that is, a condition or some other expression that evaluates to true or false.
It must be followed by the expression or value to evaluate if the condition is true.
For example, to create a state expression that stores the value of the most recent card not present (CNP) transaction:
Note that this uses business rule syntax for the state expression. For more on state expressions, see State.
This state variable is updated only for events where the transactionType field is CNP.
For other events, the condition before the ? operator evaluates to false, or evaluation stops altogether if the transactionType field is missing from the event.
When this happens, the expression stops evaluation and any value currently in the state variable is therefore not overwritten.
When using the conditional operator, you can also supply an expression to evaluate if the condition is false. This enables you to implement an if/then/else logic, storing one value if the condition is true and another if it false. If the evaluation of the initial condition stops, the expression is not updated.
For example, given two event types—a deposit and a withdrawal—in some cases, such as tracking a customer’s balance, you might want to calculate a signed version of the event value, which is negative for a withdrawal and positive for a deposit. You can do this using a transient variable:
If the event type is deposit, this variable takes its value from event.amount.baseValue; otherwise, if the initial condition is false, it takes the value of this field multiplied by -1.
For more information on the conditional operator, including other uses, see Ternary operator.
Dates, times, and durations
Copy section link
Dates, times and durations in AMDL make it simple to compare times and base your logic on the elapsed time between two events.
Datetimes must be written in ISO-8601 format, such as 2020-02-01T12:34:56Z, and must have a time zone designator.
This can be Z for UTC, or an offset in the format + (for time zones ahead of UTC) or - (for time zones behind UTC) followed by hh for an integer number of hours, or an offset in hours and minutes expressed as hhmm or hh:mm.
Durations, which represent elapsed time, are written by appending a unit to an integer.
For example, 7d represents 7 days.
The duration units that can be used in AMDL are shown below.
| Unit | Meaning | Example (all = 1 day) |
|---|---|---|
d |
Days |
1d |
h |
Hours |
24h |
m |
Minutes |
1440m |
s |
Seconds |
86400s |
In AMDL, you can subtract one datetime from another to give a duration, and you can add durations to and subtract durations from a datetime to get another datetime. For example, this expression gives the datetime three hours after the datetime of the current event:
This expression gives the datetime 30 minutes before the current event:
The expression below shows the use of datetimes and durations in a condition.
The condition is true if less than 60 days have elapsed between the current event and the time the customer registered for their account.
The first part of the expression evaluates to a duration representing the amount of time elapsed between the eventTime and the accountOpenDateTime. This is then compared to a fixed duration of 60 days (60d).
Note
The order of the two fields is important. If the account open date is known to be before the time of the current event, which is most likely, this order results in a positive duration. Inverting the order would result in a negative duration, which would always be less than 60 days, and result in incorrect logic.
Collections
Copy section link
In AMDL, collections are data structures that contain multiple values. The simplest type of collection is an array. An array consists of multiple individual elements, which may be of any type. For example, this expression defines an array of strings:
Using AMDL conditions, you can check whether a collection contains a given value using the contains operator, ~#.
For example, given the collection above, this condition is true:
AMDL also has a does not contain operator, !#.
For example, the following condition is true:
Arrays can be defined inline as part of your AMDL expression.
For example, the following condition would be true if the value of the merchantCategoryCode field was one of those listed:
You can also use annotations to transform AMDL expressions such as state and transient variable definitions into collections. By default, these expressions store a single value, and when the expression is evaluated the stored value is updated and overwritten with the latest value. Collections stored in a state allow you to store, for example, the values of a merchant’s 10 most recent transactions, or the unique IDs of all the mobile devices used to access an account.
Arrays
Copy section link
Annotations are used to change the way that data is stored in state and global expressions.
An array, which uses the @array annotation, turns a state or global expression from one that stores a single scalar value into one that stores multiple values.
An argument must be provided to the @array annotation, which can be either a number or a duration.
If the argument is a number, it specifies the maximum number of values to store in the collection; if the argument is a duration, values are retained for that duration in the collection.
Values that have been in the collection for longer than the specified duration are deleted when that expression is referenced or updated.
For example, the following business rule expression, defined for the consumer entity type, would store the values of the last 10 transactions for each customer entity:
the following business rule expression would store the values of all the transactions for each entity over the last 24 hours:
Sets
Copy section link
While an array is an ordered collection of any values, a set is an unordered collection of unique values—for example, the collection of all unique IP addresses from which an account has been logged into.
A set is declared in exactly the same way as arrays, except using the @set annotation.
Sets have several use cases. For example, a set could be used to store the different payment methods used by a customer over the last 30 days, the last 10 mobile device IDs that have logged into a particular account, or all the IP addresses that a device has logged in from over the course of an hour.
Using collections
Copy section link
A single value can be checked against the elements of a collection using the ~ and ! operators, or using other collection operators described in Operators in the AMDL Reference.
This allows you to check whether a value is or is not contained in an array or set, or check all the elements in a collection against a single value.
For example, you could check if all the elements in an array are greater than 100, or if all the elements are less than the value of the current transaction.
The collection methods described in Methods in the AMDL Reference can also be used to extract data from a collection. For example, there are methods for calculating the number of elements in a collection, the total value, the mean, median and mode, and other statistical measures, as well as for sorting collections alphabetically or numerically, or combining collections.
There are also more complex types of collection, including histograms and maps. For more on these types, see Histograms and Maps.
Business rule basics
Copy section link
Business rules are a way to implement fraud and financial risk detection logic by breaking down that logic into units—that is, AMDL expressions—that each serve a different purpose.
These expressions are defined under different scopes such as state and var.
When creating or editing business rules, each AMDL expression is edited within its own tab inside the AMDL Editor.
There cannot be more than one AMDL expression in a tab.
The examples in this section use the TRANSACTION_RT event type described through the example in AMDL expressions, as well as additional event types described later in this section.
Rules
Copy section link
A rule is an AMDL expression defined in the rules scope, and consists of a boolean expression that is evaluated for each event that passes through the system.
When creating business rules, this the most important type of expression.
Each AMDL expression applies to one particular entity type, such as customer accounts or merchants, and is evaluated for each entity of that type in each event. If the expression evaluates to true, the rule has triggered on that event. The general syntax of a rule is as follows:
Example 1: A simple rule
Copy section link
As an example, you might want to create a rule that triggers when the value of a transaction exceeds 100 units of your base currency. This rule could be written as follows:
In general it is good practice to give the AMDL definitions descriptive names.
In this case, the rule is named greaterThan100.
You could include additional conditions in the rule—for example, you could create a rule that triggers only for an accepted transaction where the amount is over 100 and the currency is USD or GBP:
Adding annotations
Copy section link
You can limit the events for which a rule is evaluated using the eventType annotation.
For example, to specify a rule that is only evaluated on the TRANSACTION_RT event type:
You can define a rule that is evaluated for multiple event types by adding additional eventType annotations.
For example:
If no event type annotations are specified on a rule, the rule is evaluated on every event; however, with no event type annotations, even when the rule triggers, it does not produce any effect.
In business rule AMDL, annotations are used to add a tag to the output when a rule triggers. For instance, you could add tags to the rule as follows:
This would produce a tag with the namespace action and value BLOCK, and another that would simply display the text High value transaction or account transfer in the dashboard.
The general form of the tag annotation is @tag(<namespace>="<value>").
You can add multiple tags by adding the @tag annotation multiple times, or you can add multiple tags in the same namespace using the following syntax:
If no namespace is provided, a default namespace (_tag) is used so, if the namespace is unimportant, you can simply write @tag("<value>").
This tag displays without a namespace in the dashboard.
Example 2: Storing and using state
Copy section link
In addition to detecting large transactions, you might want to detect test transactions, which are low-value transactions intended to test the payment method immediately before a large transaction. To detect a low-value transaction followed by a high-value one, you need to store some information about the first transaction.
To store the value of the most recent transaction, you can write a state expression:
The @eventType annotation is used here to limit the execution of this expression to transaction events.
The entity type against which you write this expression is important, because the state is stored for this entity type.
For the following event, if the above expression were written against the customer entity type, the value 100 would be stored for this variable in the state of the entity Customer1:
You can then reference this state expression in a rule, provided that rule is written against the same entity type. This state expression is evaluated for each transaction event, and the transaction value stored against the relevant entity. However, this evaluation is performed after the evaluation of rules and other expressions, so when a rule is evaluated the value of the state expressions it references are the values as of the most recent event previously seen for that entity.
For example, this rule triggers if a customer makes a low-value transaction (value less than 10) and their next transaction event is a high-value transaction (value over 100):
You can also write state expressions based on one event type, which are referenced in rules written against another event type. For example, you might want to store some information about a customer when they register, and use that information in the rule.
For example, suppose the registration event contains the following data:
You can store any of this against the customer entity type—for example, you might want to store the customer segment code to determine whether they are a VIP customer when the rules are executed:
This might be used to ensure that the test transaction rule does not apply to VIP customers:
Example 3: Timestamps and elapsed time
Copy section link
The previous example includes a rule that triggers if a customer makes a low-value transaction followed by a high-value one, but that rule makes no reference to the time elapsed between those two transactions. With a typical test transaction, a high-value transaction is usually made shortly afterwards; a transaction with a low value is unlikely to be a test transaction if the following high-value transaction takes place a week later. Times and durations are therefore important in business rules. For more information, see Dates, times, and durations. To store the timestamp of each transaction for each customer, use a state expression:
You can then refer to this state expression to determine the elapsed time since the previous transaction. This means that you can modify the rule from example 2 to take into account the elapsed time since the low-value transaction—for example, to make the rule trigger only if it has been less than 2 hours since the previous transaction:
Here, state.previousTransactionTime and event.eventTime are both fields that contain datetimes, and subtracting one from the other gives a duration representing the elapsed time between the two events.
Example 4: Conditional state
Copy section link
In the example above, a rule was created that checks the value and time of the last transaction and triggers if a low-value transaction is followed by a high-value transaction within 2 hours. However, this could fail to detect a test transaction if a customer’s card has been compromised and the card details are being used by both the criminal and the genuine cardholder.
Consider the following sequence of events:
| Time | Cardholder or criminal? | Transaction value | Previous transaction value | Time since previous transaction | Result |
|---|---|---|---|---|---|
10:00 |
Criminal |
$5 |
N/A |
N/A |
Does not trigger |
10:30 |
Cardholder |
$90 |
$5 |
30 mins |
Does not trigger |
10:45 |
Criminal |
$1000 |
$100 |
15 mins |
Does not trigger |
There are no events in this sequence that match all the conditions in the rule. A better approach might be to keep track of the time since the last low-value transaction, and trigger the rule if a high-value transaction occurs within 2 hours of a low-value one, regardless of whether there were any intervening transactions.
You can use conditional assignment to create state expressions that only store a value when a certain condition is true. This allows you to create an expression that stores the time of the last low-value transaction for each customer:
This expression stores the date and time only if the value of the transaction is less than or equal to 10; if the value is greater than 10, the state expression is not updated. You can then create a version of the rule that refers to this state:
Given the following sequence of events, the rule now triggers:
| Time | Cardholder or criminal? | Transaction value | Time since previous transaction | Result |
|---|---|---|---|---|
10:00 |
Criminal |
$5 |
N/A |
Does not trigger |
10:30 |
Cardholder |
$90 |
30 mins |
Does not trigger |
10:45 |
Criminal |
$1000 |
45 mins |
Triggers |
Note the N/A in the first row.
This is because on the first transaction for a particular customer, the state expression has not yet been evaluated and therefore returns a null value.
This causes the execution of the rule to stop when it reaches the reference to state.previousLowValueTransactionTime, meaning that the rule fails to execute and does not generate any output for this event.
Whenever an expression refers to a state variable that has no value for the current entity, or references an event field that is not present in the current event, that undefined value can cause expression execution to stop.
In reality, many state and other variable references return a null value. In most cases, this is the intended behavior. It makes no sense for the test transaction rule to trigger on the first transaction a customer makes, so when the rule execution stops this does not cause any issues. However, in some cases, it may be important to deal with or prevent references to state variables, event fields, or other variables from returning null and causing the execution of the referring expression to halt.
The default value operator is one approach.
You can also use the @defaultValue annotation to provide a default value when defining state and global expressions, so that when an entity or entity type is first seen, the state variable already has a value.
Example 5: Global state and more annotations
Copy section link
Consider the case where you want to create a rule that identifies anomalously high spending. If normal behavior indicated an average entity spend of about $100, you might be inclined to specify a static threshold to trigger a rule if a transaction has a value over 500, for example. However, this might cause problems during times when average spend tends to be higher—during periods of time such as the run up to Christmas or Black Friday. It might be better in this case to track the average entity spending and compare the single transaction amount to this moving threshold.
To do this, you can define a global expression that stores state from all customers' transactions—for example:
Here, use the @rollingAverage annotation to make the expression calculate an average value instead of storing a value that is overwritten each time a transaction is processed.
You can then define a rule that generates a tag message for any transaction with a value more than five times the rolling average:
Example 6: Collections
Copy section link
As described in Collections, you can also create collections in an AMDL expression. These can be useful for storing multiple values or comparing a field in the event data against a list of possible values.
For example, you might want to generate a tag to decline a high-value transaction at a merchant within a high-risk category as indicated by the Merchant Category Code (MCC):
This rule uses the using the contains operator (~#) to generate a tag message if the transaction value exceeds five times the global average and the MCC is one of those listed in the array.
To use this list of high-risk MCCs in several expressions or to separate this definition from the business logic of the rule itself, you could store it as a static value using the values scope.
You could then reference this static array in the rule to achieve the same effect as above:
Note that the values expression above must be defined for the same entity type as this rule.
You can also create expressions that store collections in entity or global state.
For example, you might want to store a list of all the payment methods used with a customer account over the last year, and use that to trigger a rule if you see a large transaction using a payment method not seen before over that time.
To store that information, you would write a state expression using the @set annotation:
You can then write a rule that checks whether the payment method being used in the current transaction event is contained in that set:
Here you can use the does not contain operator (!#) so the rule triggers if the value is over 500 and the payment method is not contained in the collection of payment methods for the current entity.
Other business rule effects
Copy section link
Other business rule effects are available for suppressing tags and for rule scores.
Suppressing tags
Copy section link
In some cases, you may want to suppress the generation of tags for specific events—for example, to prevent customers on a VIP list from having their transactions held or blocked.
The @suppressTag annotation suppresses tag emission from all sources, including AMDL rules, models, risk score thresholds and aggregators.
Suppressed tags are never shown in the dashboard, and are removed from the outputTags key in the JSON response.
However, any tags assigned by models are still be listed under tags within the models section of the response.
Because business rules are defined for a specific entity type, and tag suppression only work within an entity type—that is, if you write an expression with the @tag annotation for one entity type, and an expression with the @suppressTag annotation against a different entity type, the second expression does not suppress the tag generated by the first, even if they both trigger on the same event.
Rule scores
Copy section link
Rules can be configured to output a score using the @score annotation.
The score for each rule is supplied as an argument.
Scores may be positive or negative.
The total of all the scores from triggered rules for a specific event is output as the businessrules model score.
This score is rescaled in the same way that model scores are—a total score of 1 is equal to 100%.
For example, for an event with an MCC of 5678 and an amount of $200, the following rules would output a total score of 0.3 (30%):
If the businessrules model score exceeds the overall risk score generated by any other models for an event, it is displayed as the overall risk score in the dashboard for that event.
This may result in risk score bars with a score in excess of 100%, or a negative score.
While this is displayed appropriately in the dashboard, it may or may not be appropriate to assign events a risk above 100%.
Workflow basics
Copy section link
Analytical workflows provide features to help create, update, and manage complex sets of rules. Each analytical workflow consists of a list of rule sets which are evaluated sequentially and a set of customizable parameters and support variables that pass state between rule sets in the analytical workflow.
Only one analytical workflow ever executes for a single event. Multiple analytical workflows may exist, but the system must decide which one to run based on the event type. A single analytical workflow can be mapped to one or more event types, but the system does not allow multiple analytical workflows to be mapped to the same event type.
Components of a workflow
Copy section link
An analytical workflow consists of rule sets, scorecards, support variables, and parameters.
Rule sets
Copy section link
A rule set consists of an AMDL block, which is where the main logic of the rule set is defined, and one or more scorecards, which allow the rule set to create effects such as tags. A rule set can also have an attached condition, which is an AMDL expression that evaluates to true or false. The rule set is evaluated only for events where the condition is true.
Scorecards
Copy section link
Rule sets create effects such as adding tags by using scorecards.
Each scorecard references a transient variable (var) defined in the AMDL block of the rule set.
Based on the value of this variable, different effects can be triggered.
Different match conditions can be defined as part of a scorecard, and these conditions can have one or more associated effects. An effect can be one of the following types:
-
Generate a tag.
-
Suppress a tag.
-
Set an overall risk score.
-
Add to data list override.
-
Update a support variable.
Support variables
Copy section link
Support variables enable values to be passed from one rule set to another in a workflow, and to determine if and how rule sets get executed and what the outcomes are. The variables may be boolean expressions, strings or numerical values, which can be integers or decimals.
They can be referenced in the conditions, AMDL block, match conditions, and effects of each rule set. For each variable, an initial value is set up at the beginning of each analytical workflow. This initial value may be modified by scorecards in each rule set using effects. That change is then passed on to the next rule set and can affect how it executes and the resulting output. An effect can add to, subtract from, or set a support variable before passing it on to the next rule set or scorecard.
Parameters
Copy section link
Parameters are named, immutable values that can be referred to in workflows, but not updated. They can be used to define thresholds for risk levels as well as to check for a keyword in the event data, or to check that an AMDL expression evaluates to true or false in a given situation. Within workflow execution, parameters cannot be modified by AMDL execution or by effects.
Example 1: A simple rule set
Copy section link
This first example demonstrates the simplest way to implement an effect, such as add a tag, using an analytical workflow. This workflow generates an explanatory tag for any transaction with a value over 1000. In the AMDL block, create a transient variable that stores a value from the relevant event. Then create a scorecard that uses this transient variable, and within that scorecard, define match conditions that trigger effects. This workflow consists of a single rule set and is mapped to the transaction event type.
AMDL block
Copy section link
The expression in this AMDL block simply takes the value of the transaction (in the baseValue field within the event data) and stores it as a transient variable (var.amount) that can then be used in the scorecard:
var.amount: event.amount.baseValue
Scorecard
Copy section link
The scorecard makes use of the transient variable to decide whether to trigger effects—in this case, when the transaction value is over 1000, add a tag for the customer entity type.
Name: transactionAmount
Expression variable: var.amount
| Condition | Output Effect | ||
|---|---|---|---|
Operator |
Matcher |
Type |
Details |
Greater than |
1000 |
Tag |
Entity type: Customer |
Additional effects could be added to the scorecard.
For example, if you want to generate a tag for a value over 1000, but if the value exceeds 5000, add the tag ACTION="decline" to trigger a decline, you could modify the scorecard as follows:
| Condition | Output Effect | |||
|---|---|---|---|---|
Operator |
Matcher |
Type |
Details |
|
Between |
1000 |
5000 |
Tag |
Entity type: Customer |
Greater than or equal to |
5000 |
Tag |
Entity type: Customer |
Example 2: Multiple scorecards
Copy section link
A single rule set can include more than one variable used to trigger effects, and some effects can override others.
In this example, add another condition—if the customer making the transaction is a VIP, the transaction should not be declined (the ACTION="decline" tag should be suppressed).
AMDL block
Copy section link
The first expression in this AMDL block should be familiar from example 1.
The second expression (highlighted below) defines another transient variable—a boolean variable that is true if the customer is a VIP, and false otherwise.
This expression uses a condition that is true if the value in the field customerSegmentCode is "V".
You can then create a second scorecard using this variable (see the scorecards below).
Scorecard 1
Copy section link
This scorecard is the same as the one described in Example 1.
Name: transactionAmount
Expression variable: var.amount
| Condition | Output Effect | |||
|---|---|---|---|---|
Operator |
Matcher |
Type |
Details |
|
Between |
1000 |
5000 |
Tag |
Entity type: Customer |
Greater than or equal to |
5000 |
Tag |
Entity type: Customer |
Scorecard 2
Copy section link
This scorecard is based on the transient variable var.VIPcustomer, so if that variable is true—that is, the customer is a VIP—the ACTION="decline" tag should be suppressed.
An explanatory tag reading VIP customer is added to indicate why this transaction was not declined.
Note that in this example, the High value transaction tag is not suppressed, so it is still added to transactions from VIP customers if the value exceeds 1000.
| Condition | Output Effect | ||
|---|---|---|---|
Operator |
Matcher |
Type |
Details |
Equal |
true |
Suppress tag Tag |
Entity type: Customer Entity type: Customer |
Example 3: Storing and using state
Copy section link
Analytical workflows can store and make use of entity state—that is, behavioral profile data—for entities of all types. This enables data about individual entities to be stored on one execution of a workflow and used in a subsequent execution of the same workflow, or even by a different workflow. See State for more information about creating and referring to state expressions in analytical workflows.
In this example, a state expression is used to store the value of each transaction event in the behavioral profile of the customer entity. This means that when the rule set runs for a transaction, it can access the value of the previous transaction made by the same customer. This allows test transactions, which use a very small transaction followed by one with a larger value, to be detected.
AMDL block
Copy section link
The AMDL logic for this example includes two additional expressions.
The first is a state expression, which stores the value of each transaction.
This is similar to the transient variable defined in the first example, but in the state definition you need to include the entity type.
In this case, you would need to store the state against the customer entity type to be able to access the customer’s previous transaction value each time they make a transaction.
Note
You can find out the names of your entity types by looking at the Business Rule Profile area in Analytics > Status in the dashboard. For more information, see Real-Time Decisioning in the Dashboard.The second new expression is a transient variable that uses this stored value. Because state data is only updated at the end of the execution of a rule set, this variable always uses the value of the previous transaction for this customer entity.
Note the use of .single() to ensure that the state expression reference returns a single value (see State in workflows).
Scorecard 1
Copy section link
This scorecard is unchanged from the previous example.
Scorecard 2
Copy section link
This scorecard implements a very simple logic—if the transient variable testTransaction is true, a tag is added.
In this way, the workflow triggers a tag message if the transaction amount is between 1000 and 5000 (as defined in scorecard 1) or if the current transaction amount is over 500 and the previous transaction amount was less than 10.
Name: testTransaction
Expression variable: var.testTransaction
| Condition | Output Effect | ||
|---|---|---|---|
Operator |
Matcher |
Type |
Details |
Equal |
true |
Tag |
Entity type: Customer |
Scorecard 3
Copy section link
This scorecard, which suppresses tags if the customer is a VIP, is unchanged from scorecard 2 in the previous example. Note that this scorecard has been moved to the end of the list, to ensure that it still overrides any tags generated by other the scorecards from this rule set.
Example 4: Introducing conditions and durations
Copy section link
The previous example showed a rule set that added a tag if a customer made a small transaction (value less than 10) followed by a large one (value over 500). This misses an important consideration when looking for test transactions—typically the higher-value transaction happens shortly after the small-value test transaction. A small transaction followed, days later, by a large one, is typically not a risk indicator, but the previous example does not take this into account.
This consideration could be incorporated in a several ways. One way is to split the previous rule set into two.
Rule set 1: transactionValue
Copy section link
This rule set implements the transaction value logic created in Example 1 and is the same as that example, except for the use of the following condition described below.
Condition
Copy section link
This condition is only true if the customer is not a VIP. This means that this rule set only executes for non-VIP customers, and therefore tags are not generated on VIP customers' transactions, nor are any of those transactions declined. This replaces the following in the previous example:
event.customerSegmentCode != "V"
AMDL block
Copy section link
The VIP customer logic has been removed from this AMDL block, and from the corresponding scorecard.
Scorecards
Copy section link
Scorecard 1 is included unchanged from the previous example. The logic in scorecard 2 has been moved to the rule set below. Scorecard 3 is no longer included, because the logic for suppressing tags for VIP customers is no longer needed.
Rule set 2: testTransaction
Copy section link
This rule set implements the test transaction logic introduced in Example 3, but with the addition of the same condition used in rule set 1.
Condition
Copy section link
event.customerSegmentCode != "V"
AMDL block
Copy section link
This AMDL block has been modified to test whether the previous transaction took place within the last six hours.
This is implemented using durations.
For more information, see Dates, times, and durations.
The new condition added to the definition of var.testTransaction is true if the timestamp of this event (event.eventTime) is less than six hours after the timestamp of the previous transaction that involved the same customer entity (state.customer.previousTransactionTime).
This state expression is also defined in the AMDL block.
Scorecard
Copy section link
This scorecard implements the same logic as Scorecard 3 in Example 3—if the transient variable testTransaction is true, a tag is triggered.
Name: testTransaction
Expression variable: var.testTransaction
| Condition | Output Effect | ||
|---|---|---|---|
Operator |
Matcher |
Type |
Details |
Equal |
true |
Tag |
Entity type: Customer |
Example 5: using support variables, risk scores, and final assessment
Copy section link
Support variables are used in analytical workflows to calculate risk scores and other values that need to be updated or referenced by multiple rule sets. For example, support variables might be used to calculate risk scores for the various entities involved in an event, and then a final rule set might combine those individual risk scores into an overall risk score and trigger actions based on that score.
Support variables are defined when you create your workflow, and you must assign each one a default value.
Support variables can then be referenced in your workflow AMDL using the support scope, but AMDL expressions cannot change the value of a support variable.
To update a support variable, you must use effects in your scorecard.
In this example, two support variables are used to calculate a risk score for the customer and the merchant in a card transaction. These risk scores are then combined into an overall risk score in the final rule set of the workflow, and the final effects—that is, tags—are based on the value of this overall risk score.
Support variables
Copy section link
| Name | Description | Type | Initial Value |
|---|---|---|---|
customerRiskScore |
Risk score based on the details of the transaction itself and the customer involved. This contributes to the total risk score for a transaction. |
Numeric |
0 |
merchantRiskScore |
Risk score based on the history of the merchant in the transaction and their status. This contributes to the total risk score for a transaction. |
Numeric |
0 |
Rule set 1: customerRisk
Copy section link
This rule set looks at transaction value and the risk associated with the customer. If the value is over 5000, it adds 400 to the customer risk score, or if the value is between 1000 and 5000, it adds 200. If the previous transaction for this customer was a low-value transaction within the last six hours (possibly a test transaction), it adds 300. It combines the transaction value and test transaction logic from the previous example (using two separate scorecards—see below). Note that the logic that checks whether this is a VIP customer has been moved to Rule set 4:_finalRiskScore.
Scorecard 1
Copy section link
Name: transactionValue
Expression variable: var.amount
| Condition | Output Effect | ||
|---|---|---|---|
Operator |
Matcher |
Type |
Details |
Greater than |
1000 |
Tag |
Entity type: Customer |
Greater than |
1000, 5000 |
Update support variable |
Variable: customerRiskScore |
Greater than |
5000 |
Update support variable |
Variable: customerRiskScore |
Scorecard 2
Copy section link
Name: testTransaction
Expression variable: var.testTransaction
| Condition | Output Effect | ||
|---|---|---|---|
Operator |
Matcher |
Type |
Details |
Equals |
true |
Tag Update variable |
Entity type: Customer *Variable*: customerRiskScore + *Action*: Add + *Value*: 300 |
Rule set 2: merchantRisk
Copy section link
This rule set looks at the merchant category of the merchant involved in this transaction, and checks whether it is contained in a list of high-risk or medium-risk merchant categories.
These lists are provided as arrays of strings within the expression itself, and each condition uses the contains (~#) operator to check whether the MCC of this merchant is contained in each list.
For more information, see Collections).
This expression also uses the conditional assignment syntax described in Conditional assignment.
If the merchant category code (MCC) is in a list of high-risk MCCs, it sets the merchant risk score to 375, or if the MCC is on a list of medium-risk MCCs, it sets it to 250.
A simpler version of this check could be implemented by only checking against a single list of high-risk MCCs, as in the following AMDL block:
This contains a single conditional operator, and simply assigns the transient variable the value 375 if the MCC is on the list at the beginning of the condition, and 0 otherwise. However, the AMDL block shown below enables you to set the merchant risk score to a different value depending on whether the MCC is high- or medium-risk, using nested conditional operators. If the first condition is true the variable is set to 375; if it is false, the part of the expression after the second colon is evaluated. This starts with another condition, and if this condition is true, the variable is set to 250; otherwise, it is set to 0.
Scorecard 1
Copy section link
This scorecard simply sets the merchant risk score support variable to the value of the transient variable set in the AMDL block.
Name: mccRiskScore
Expression variable: var.MCCRiskFactor
| Condition | Output Effect | ||
|---|---|---|---|
Operator |
Matcher |
Type |
Details |
Greater than |
0 |
Update support variable |
Variable: merchantRiskScore |
Tag |
Entity type: Merchant |
Rule set 3: finalRiskScore
Copy section link
This rule set calculates the final risk score for a transaction by adding the customer risk score and the merchant risk score, with the merchant score multiplied by a weighting of 0.8. If the total risk score is over 700, it adds a tag to trigger a decline; if the total score is between 300 and 700 it also generates a tag. The rule set also outputs the total score for display in the dashboard, using the Overall Score effect.
Scorecard 1
Copy section link
Name: finalRiskScore
Expression variable: var.finalRiskScore
| Condition | Output Effect | ||
|---|---|---|---|
Operator |
Matcher |
Type |
Details |
Greater than |
1000 |
Tag |
Entity type: Customer |
Greater than |
1000, 5000 |
Update support variable |
Variable: customerRiskScore |
Greater than |
5000 |
Update support variable |
Variable: customerRiskScore |
Rule set 4: VIPcustomer
Copy section link
This rule set prevents VIP customers' transactions from generating a tag or being declined, but only if the customer risk score is low. It has been moved to the end of the list of rule sets to ensure that it overrides decline tags from all other rule sets in the workflow, and so that it can reference the customer risk score support variable set by rule set 1.
Condition
Copy section link
This condition prevents the rule set from being executed if the customer risk score is greater than 400.
Note the use of the support scope to refer to the current value of the customer risk score support variable, which was updated by rule set 1 earlier in the execution of the workflow.
support.customerRiskScore <= 400
Scorecard 1
Copy section link
This scorecard works as before, suppressing the decline tag if the customer is a VIP, and adding an explanatory tag.
Name: VIPcustomer
Expression variable: var.VIPcustomer
| Condition | Output Effect | ||
|---|---|---|---|
Operator |
Matcher |
Type |
Details |
Equal |
true |
Suppress tag |
Entity type: Customer |
Suppress tag |
Entity type: Customer |
||
Update support variable |
Entity type: Customer |
||
Unit tests
Copy section link
A vital part of writing any code is testing, and the Real-Time Decisioning analytics dashboard provides a useful offline tester that you can use to ensure AMDL expressions are behaving as expected before applying them to a live system. Multiple tests can be attached to any AMDL expression to test your business rules, and tests can also be created for any rule set in a workflow, to test the AMDL expressions in the rule set condition and in the main AMDL block.
A test consists of a test event, plus optionally some initial state, and optionally one or more tests that check the final state once the AMDL has been executed.
In both business rules and analytical workflows, the tests panel looks like this:
All of the tests attached to the AMDL expression or rule set are listed in the left panel. The initial state, input event, and expectations are entered in the middle left, middle right, and far right panels respectively.
Basics of AMDL unit tests
Copy section link
The basic principle of AMDL unit tests is to check what the results would be of running a specific AMDL expression (in business rules) or AMDL block (in analytical workflows) against your test event.
You can create positive test cases—that is, cases where an event should cause a rule to trigger, or a state variable or support variable to be set to a certain value—and negative test cases where a rule should fail to trigger or a variable should not be set or updated.
Tests are saved alongside the expressions for which they are written, and can be exported and imported alongside your business rules and workflows.
The input event
Copy section link
The most important component of any unit test is the input event.
This is the event that your AMDL will be evaluated against.
The input event panel must contain JSON event data, and the event must have an eventType field and be of a type that is defined in your schema.
However, the event is not validated against the schema, enabling you to use minimal events containing only the necessary fields, and even test potential schema changes.
The dashboard event generator tool that forms part of the AMDL unit tester enables you to generate events that match your schema, and set the values of optional and mandatory fields to create appropriate test cases.
Select the Lightning button at the top of the Input Event panel to generate a test event. This is the best way to create routine unit tests for your AMDL code, as it ensures that the test event closely approximates the real events that your AMDL will be evaluated against in production.
Expectations
Copy section link
Each test must define the expected outcome or outcomes, so that the result of the test evaluation can be compared to the expected result to determine if the test passed or failed. These are set in different ways depending on whether you are testing rules or other kinds of expression in business rules or testing analytical workflows. See Testing business rules and Testing analytical workflows.
Initial state
Copy section link
The Initial State panel allows you to define values for any other expressions referenced by the AMDL you are testing. Your unit tests have no access to real entities or live state, or the values of any other AMDL expressions. Therefore, you must define values for any referenced state or global variables, transient variables, static values, data lists, support variables, or workflow parameters that you want to have a value in your test.
State and other variables are defined in exactly the same way they are in normal AMDL code, but instead of an expression that evaluates to a value, in the Initial State panel you must set any variable to a fixed value. For example, you could set the value of a transient variable as follows:
For static values (the values scope), you can simply copy and paste the values expression into the Initial State panel.
For more information, see Static values.
You can set the value of multiple different variables in different scopes by simply placing each expression on a separate line—for example:
Entities in unit tests
Copy section link
If you have expressions that refer to the state of multiple different entities or entity types, you can set values for these state expressions in the Initial State panel using an annotation specific to unit tests, @entitytype.
This annotation takes two named arguments, the entity type (type) and the entity ID (id), although the entity ID is optional.
Any state expressions defined below such an annotation in the Initial State panel will belong to the entity specified in the @entitytype annotation.
For example, to specify the value of the state expression avgTxValue for a merchant entity with the entity ID merchant1:
This is useful for cross-entity state references in business rules and for testing analytical workflows. For more information, see Cross-entity state references and Testing analytical workflows.
In business rule tests, the entity the expression is being evaluated for always has the ID testEntity.
For example, if a business rule is written against the customer entity type, and an event contains two consumer entities, the one the rule is being evaluated for would be identified by having the ID testEntity.
Unless you explicitly set the appropriate field in the input event, this entity ID does not match any entity ID fields in the event data, so to test an AMDL expression that checks the value of an entity ID field against the ID of the entity being evaluated against, you must set the value of the appropriate field to testEntity.
See Entity meta-variables for an example of expressions where this might be necessary.
In unit tests of workflows, one entity of each type is assigned the ID testEntity.
As with business rules, other test entities can be defined using the @entitytype annotation.
Testing business rules
Copy section link
For a positive test case, consider the following rule, which is similar to one introduced in Example 3: Timestamps and durations and conditional state.
There are a number of test cases you might want to create to ensure that this rule works correctly. The simplest is the positive case—a transaction event where the value is greater than 100, the previous transaction value was less than 10, and the previous transaction was less than two hours ago.
The first thing to do when creating a test is to create the input event.
For the positive test case, this means a transaction event where the value of amount.baseValue is greater than 100—for example, 150.
The event is automatically generated with an eventTime equal to the current time.
This example uses placeholder values in all the other fields, providing a test event shown below.
To create the positive test case, you also need to specify the values of the state expressions referenced in the rule. You can do this in the Initial State panel.
The correct value of state.previousTransactionValue is simple—it just needs to be less than 10.
For this example, state.previousTransactionTime is set to a datetime less than two hours ago.
The simplest way to do this is to copy the datetime from the input event, which is 9:55 am on 13th Apr 2022, represented in ISO 8601 format as 2022-04-13T09:55:56.922Z.
By reducing the hour by one, you can set the previous transaction time to 08:55 on the same day, and create a case in which the rule should trigger.
The final step in creating this test is to set the expected outcome. In this case, the rule should trigger, which is set by selecting Check triggers from the Check rule dropdown at the top-right of the tests pane. You can now name this test and run it.
For negative test cases, while it’s important to test that your rule triggers when it should, it’s perhaps more important to test that a rule doesn’t trigger in conditions where you believe it shouldn’t generate a decline tag. This is why negative test cases are important. While it’s often difficult to cover all the possible cases where the rule shouldn’t trigger, it’s important to test some of the more likely, and to identify edge cases that could potentially cause unintended behavior.
For the rule above, you can identify three obvious cases to test:
-
Transaction value is less than 100.
-
Previous transaction value is greater than 10.
-
Previous transaction time is more than 2 hours ago.
The simplest way to create a negative test case, once you’ve created your first positive test case, is to duplicate that first test and then edit it. By duplicating the positive test case described above, you can create each of the three negative test cases by:
-
Changing the amount in the input event to 90.
-
Changing the value of
state.previousTransactionValueto 11. -
Changing the time of
state.previousTransactionTimeto06:55(for example, change the datetime to2022-04-13T09:55:56.922Z).
For each of these tests, you also need to change the Check rule setting to Check does not trigger.
One other kind of situation you might want to test is the case where one or more of the state expressions referred to in the rule has no value—for example, when this is the first transaction for this particular entity, or, in the case of an optional field, where the event data field referred to is not present.
You can create a simple test case of this kind by duplicating the initial positive test case, and deleting the expressions state.previousTransactionValue and state.previousTransactionTime from the Initial State panel.
This simulates the rule’s behavior on the very first transaction for any given customer entity.
Again, the Check rule setting needs to be changed to Check does not trigger, since the rule should not trigger when these state variables are undefined.
In this case, the rule stops executing when it reaches the reference to an undefined state.
This results in a case where the rule doesn’t trigger, and hence the test passes, but the test results also contain a warning that the rule did not execute.
While this is expected behavior, a rule that fails to evaluate returns null when referred to in another expression.
For more about this behavior, see Rule references.
Testing state and global expressions
Copy section link
It is also possible to test that state and global expressions are updating as expected. For instance, you might want to test the state expression introduced in Example 3: Timestamps and durations and conditional state, which stores the datetime of the most recent low-value transaction:
There are two cases to test with this expression—does it store the datetime of the transaction if the value is less than 10, and does the stored value remain the same if the value is greater than or equal to 10?
As with rule testing, some initial state and an event are provided. When testing state and global expression, however, rather than using the Check rule dropdown list, you can instead use the Expectations panel. The Expectations panel must contain one or more AMDL rules, which are evaluated after the input event is processed. These rules are used to check that the final state is equal to the expected value - if all the rules in the Expectations panel trigger, the test passes; if any of these rules fail to trigger, the test fails. The names of these rules must be unique within the scope of a specific test, but can be repeated across multiple tests for the same entity.
For the first test case, you might set an initial value for the state you are testing, or simply leave the Initial State blank. The input event should be a transaction with a value less than 10.
In the Expectations panel, you then write a rule that tests whether the final value of the state is what you expect it to be—in this case, the same as the datetime of the input event. You can copy the datetime directly from the input event JSON when creating the rule, giving something like this:
In the test results, the name of this rule is displayed, and if the rule triggers, a green checkmark is shown next to it. The final value of the state you are testing is also shown. Using a single expectation rule is by far the most common case, but it is possible to add additional rules to this panel to check other conditions to ensure the state was updated correctly.
You can also add another test to test the second case, where the transaction value exceeds 10. You can do this by duplicating the first test, changing the transaction value in the input event, and changing the expectation rule to check that the value of the state remains unchanged.
Testing analytical workflows
Copy section link
Unit tests of analytical workflows are used to test the outputs from a rule set. Each rule set can have one or more associated tests, each of which can contain multiple test expressions that check various different outputs and results of evaluating the rule set, including the values of transient variables used in scorecards and values stored in entity and global state.
Like all AMDL unit tests, each test must have an input event. The event must be of a type that is mapped to the workflow you are testing.
As an example, you can create tests to ensure that rule set 1 in the workflow described in Example 6: More complex state and using overridable parameters functions as intended. There are a number of positive and negative test cases to address, and for each, you might want to test a number of outcomes. These are described in the table below, with the highlighted portions indicating where a test is different from the first test case:
| Initial state | Transaction event | Outcomes |
|---|---|---|
|
Mobile device ID not seen before |
|
|
Mobile device ID not seen before |
|
|
Mobile device ID seen before |
|
The rest of this section walks through the process of creating the last of these test cases.
First, an input event must be created.
This is a transaction that has a non-zero value (for example, 100), a datetime of 12:00 on 11th April 2022 (2022-04-11T12:00:00.000Z), and for which the mobile device ID field (deviceData.deviceId) is device1.
Initial state
Copy section link
In tests of analytical workflows, the Initial State panel allows you to set the initial values of state and global expressions and static values defined in other rule sets that are referred to in the rule set you are testing. You must also define the initial values of any support variables or parameters referenced in the rule set, as the actual values are not accessible to the unit test.
For the example test, the values are set for the two state expressions used to identify test transactions (the previous transaction value and time), and the collection of unique device IDs. State definitions in the Initial State panel are written in the same way as in the body of a rule set—for example, you could write:
However, you also need to define the values of the parameters that are referenced in this rule set, using the param scope used to reference parameter values:
Expectations
Copy section link
The Expectations panel in tests of analytical workflows uses an additional scope not used anywhere else—the tests scope.
Expressions in this scope are boolean conditions—they must evaluate to true or false.
A test that evaluates to true has passed, and if all the test expressions evaluate to true, that unit test has passed.
This is indicated with a green checkmark in the Results part of the test dashboard.
The Results section also shows which test expressions evaluated to true and which evaluated to false, so that if a test fails, you can identify which test condition evaluated to false.
For the example tests, test the following outcomes:
-
var.amountis equal to transaction amount (100). -
var.testTransactionisfalse. -
var.uniqueDeviceIds_countis 3. -
state.uniqueDeviceIdsfor the customer entity contains the device ID from the transaction and still has a size of 3 (because the device ID in the event has previously been seen). -
state.lastTransactionAmountfor the customer entity is equal to the transaction amount (100). -
state.lastTransactionTimefor the customer entity is equal to transaction datetime (12:00 on 11th April 2022).
The first test needs to check that var.amount is equal to 100.
This is just a simple boolean expression, as with a rule, but using the tests scope, defined in the Expectations panel:
You can add the other test conditions as further tests expressions:
Check condition
Copy section link
The Check condition dropdown menu at the top right of the tests area enables you to test the AMDL condition for your rule set, if it has one.
-
The default setting is
Check matches, which means the unit test checks whether the rule set condition evaluates to true, given the input event you have provided. If it does, the rule set is evaluated and the test expressions in the Expectations panel is evaluated. If it does not, the rule set is not evaluated and the test fails. -
You can select Check does not match from the dropdown menu. This means that the unit test passes if the condition evaluates to false. This disables the Expectations panel. If the condition evaluates to false, the rule set is not evaluated and therefore no test expressions are required; if the condition evaluates to true, the test has failed and therefore there is no reason to evaluate the test expressions.
Advanced syntax
Copy section link
AMDL provides functionality for more complex usage. The following set of AMDL features provides a greater degree of functionality and makes your rules easier to maintain:
Switch case operator
Copy section link
In a situation where a variable can take multiple values, you might want to create an expression that evaluates differently depending on the value. You could do this by chaining multiple conditional statements together (see Conditional assignment). However, more readable expressions can be created using the AMDL switch case operator, as in this example:
Here, the ~? operator initiates the switch on the value of the country field.
Each case is terminated by a semicolon.
The values switched on must be either a fixed value or default.
The default value is optional.
The default keyword means that if the rule evaluation gets to the end of a switch case execution without detecting a match, it uses the default case.
If no default is included and a matching case is not found, AMDL execution of the expression stops.
Exists operator
Copy section link
You can use the exists operator ~ to query whether the result of an expression exists or not.
This allows you to create conditions that are true if an optional event field exists within the specific event being processed, or if a specific state variable, transient variable, or values expression has a value.
For instance, you can check whether a field exists in the event by using something like this:
The inverse—a condition that is true if a field does not exist—can be constructed using the does not exist operator, !~.
Default value operator
Copy section link
If a reference within an AMDL expression cannot be found—for example, an event field which is not present, or a state that hasn’t been updated yet—it returns a null value. If the null value is not handled in the logic of the expression, AMDL execution of the expression stops and the value of the expression being evaluated also becomes null. If the expression is a rule, it does not trigger. If the expression is a variable update expression such as a state, global or transient variable, that variable is not updated. This may be the desired behavior; however, it is important to be careful how you handle such possibilities in some cases. In particular, be aware that AMDL execution does not short-circuit boolean operators and always evaluates all references; if any of these references evaluate to null, expression execution stops.
For example, even if the field accepted has a value in the event being processed, if var.acceptedTransaction is undefined, evaluation of this rule stops and does not trigger:
The default value operator ?? automatically replaces a preceding reference with the value following it, if that reference is found to be null during execution.
For example, to ensure that the rule above executes even if the accepted field is not present:
This ensures that the evaluation of the field accepted always returns a value. If the field is missing, the default value false is returned. The parentheses in this example ensure that the components of the expression are evaluated in the correct order. For more information, see Operator precedence.
First value annotation
Copy section link
Often you might want a state to capture something once and not be updated again.
For instance, you might want to store the first login date of a user. This can be done using the exists operator and the ternary operator, but because it is a relatively common occurrence a special annotation is provided, @firstValue.
When this is applied to an AMDL expression, once the expression evaluates to a value, it maintains that value and is not updated any more.
Histograms
Copy section link
Arrays and sets store all values they are updated with for the duration. When a large number of values is expected to update the collection in the period of the duration, this could produce excessive storage requirements and adversely affect engine performance. For instance, globally storing the value of all transactions over a month could result in a very large collection.
To facilitate analysis of such time-series data, AMDL provides a histogram type. A histogram consists of a group of buckets, each of which represents a group of updated values that occurred during a specific time window. It is granular on the level of the buckets, not the values, so can be a powerful approximation to expedite performance. Any state-like AMDL expression can be given a histogram annotation which has the form:
@histogram(historyLength=<duration>, bucketSize=<bucket duration>)
The duration indicates the period for which the histogram should store buckets before expiring them. The second argument determines the number of buckets in the histogram and is optional. If left unspecified, a default is chosen to ensure at least 10 buckets are used. More buckets mean more accurate summary figures but a greater storage and performance impact. Bucket size is limited to a series of acceptable durations. If the specified bucket size is not one of those listed below, the nearest acceptable bucket size is used to ensure that at least the expected number of buckets are used. The following are acceptable bucket sizes:
-
Monthly buckets: 12 months, 6 months, 4 months, 3 months, 2 months, 1 month
-
Fixed-length buckets: 7 days, 1 day, 12 hours, 6 hours, 3 hours, 2 hours, 1 hour, 30 mins, 20 mins, 15 mins, 10 mins, 5 mins, 1 min, 30 seconds, 5 seconds, 1 second.
Bucket durations expressed in months—for example, 6 months, 12 months—use M as their unit.
For example, 6 months is represented as 6M.
Histogram buckets align with calendar time periods. Monthly buckets align with calendar months, so the first relevant event that occurs after midnight (UTC) on the first day of the month contributes the first data point to that month’s bucket. Buckets with a shorter duration—expressed in days, hours, minutes or seconds—align with calendar days. All non-monthly histogram buckets align at midnight (UTC) each Monday, so buckets with a duration of 7 days start at midnight on Monday; buckets with a duration of 1 day start at midnight each day; and buckets with a duration of 6 hours start at midnight, 6am, noon, and 6pm each day.
As with arrays, you can call the size, total, and mean methods on a histogram state.
For instance, if you define a global histogram that stores transactions over the last week as follows:
Then you could write a rule that triggers if an entity performs a transaction of amount greater than 1000, but normalized for the day’s global average transaction amount in relation to the week’s:
In this case, you have added a duration argument to the mean method.
This limits the values of the histogram to those within that elapsed period, so far as the granularity of the bins allow.
In the example above, the mean(1d) method returns the mean of values from today; the mean(7d) method returns the mean of values from today and the previous six days.
Monthly durations—for example, 1 month, 6 months—use calendar months, which may be of different durations.
To avoid issues when comparing histogram data between months of different lengths, normalized versions of these methods are also provided that normalize the total, size, or mean to a month length of 30 days: normalisedSize, normalisedTotal, and normalisedMean.
Used in this way, these methods can access data from the most recent bucket, or from that bucket and previous buckets. However, for comparison purposes or for defining a baseline, you may want to refer to previous buckets. This can be done in one of two ways.
Providing a datetime as an argument to the size(), mean(), or total() method allows you to access the bucket that contains that datetime.
This datetime can be derived from event data through subtraction of a duration, to return data from the bucket from a specified duration in the past.
For example, the expressions below enable you to compare the total value of a customer’s transactions this week with the total value from last week.
This example rule triggers a decline tag if the total value of a customer’s transactions so far this week is greater than three times the total value of the customer’s transactions last week.
Data is returned for last week’s bucket instead of for the latest bucket by passing the datetime exactly 7 days ago (event.eventTime - 7d) as the argument to the total() method.
However, this approach only allows you to access data from a single bucket.
To access data from multiple previous buckets, you can use the atTime() method.
This method only applies to histograms, and takes a datetime as an argument.
Again, this datetime can be derived by subtracting a duration from a datetime.
For example, you can modify the rule in the previous example to generate a decline tag when the total value of a customer’s transactions this week is greater than the total value of a customer’s transactions over the previous two weeks.
In this example, the atTime() method with an argument seven days in the past, in combination with total(14d), is used to access data—in this case, the total value—in two buckets, not including the current one. These are the bucket that includes the datetime supplied as an argument (seven days ago) and the previous bucket, since the bucket size is seven days and the duration supplied as an argument to the total() method is twice that.
Time field
Copy section link
By default, histograms use the event data field eventTime to determine the time at which the current event occurred and to determine which buckets to update or access.
The @histogram annotation can take an optional argument, timeField, that allows you to define a non-default time field to use.
This can be an event data field or a transient variable, specified as a string—for example, to use an event data field called realTime, you would write timeField="event.realTime.
Particularly with batch processed events, the eventTime might not signify the time that the original transaction took place, so you might want to use another field as the time field:
Alternatively, you might want to use a derived time calculated in a transient variable.
For example, if the transient variable newTime contains the datetime to be used as the histogram time field:
Testing histograms
Copy section link
To test an expression that refers to a histogram, you must specify a simulated version of the histogram. For example, the following rule shows part of an accompanying test:
The state definition in the Initial State test panel must begin with a copy of the histogram annotation from the state being referred to.
The histogram is defined as a JSON map with a single key, data, within which is a series of JSON objects representing the buckets in the histogram. Each bucket has a key, which is the start datetime of that bucket, and the value is a JSON object containing two properties, size and total.
Maps
Copy section link
A map is a lookup table—given a value, it maps that value to another and returns the other value. This enables you to store key-value states. For instance, in a transactional system, every entity may have several different payment methods, and you might want to keep track of the number of transactions made using each method. Each method identifier could be used as the key, and the count of the number of transactions would be the value—that is, what each key is mapped to. The data in this map might be represented as something like this:
Here, given method1 as a key, transactionCountByPaymentMethod would return the value 10.
AMDL can be used to define both static maps (using the values scope) and dynamic maps that can be updated and modified with each incoming event, stored in entity or global state.
Static maps can be used to store lookup tables such as a currency conversion table, or a dictionary of country codes mapped to country names. For example, you might want to store a list of threshold values for different merchant categories, and raise a decline tag when the value of a transaction exceeds the threshold for the appropriate MCC. This could be implemented using nested conditional statements, although this would be very long and difficult to read, or a switch case statement. For more on switch case statements, see Switch case operator.
However, using a static map can simplify this kind of expression by removing the thresholds into a different expression from the business logic of the rule itself. The map would be defined as follows:
A rule (in AMDL business rules) or a transient variable (in analytical workflows) can then reference this map using the syntax <scope>.<map expression>[<key>]—for example, values.MCCSpecificThresholds["7999"] would return 300.
This rule would generate a decline tag if the value of a transaction exceeds the threshold for the relevant MCC:
This expression could be written using a switch case statement, which would allow you to provide a default value.
A map does not provide a way of specifying a default, but you can use the default value operator.
This enables you to provide a default value, which is returned if the key referenced is not present in the map.
For example, to specify a default threshold of 500:
Maps can also be stored dynamically in state or global expressions.
For example, you might want to store the timestamp of the last time a customer used the transaction method in the current transaction—a specific payment card or alternative payment method.
If each payment method has a unique ID in the event field event.paymentMethod.methodId, you could define a state expression that would store the timestamp of the last transaction for each unique method ID:
state.lastTimeMethodSeen[ event.paymentMethod.methodId ]: event.eventTime
In this example, each time a transaction is received with a particular method ID, the timestamp is stored as the value, with that method ID as the key.
After receiving the transactions shown in the table below, the map is stored in state in method2 as shown in the example below the table:
| Method ID | Timestamp |
|---|---|
Method 1 |
1st Apr 2022 10:01:24 |
Method 2 |
5th Apr 2022 08:17:54 |
Method 3 |
10th Apr 2022 17:26:12 |
If you then receive another transaction with the method ID method2, at 15:26:41 on 11th Dec 2021, that overwrites the existing value for that key, and the resulting map changes as follows:
You could then write a rule or other expression referencing this state.
You might want to reference the time a particular payment method was last seen, or check whether a payment method has been seen within a certain period of time.
The following transient variable stores true if the payment method in the current event has not been seen in the last 180 days: