/
80 minute read
July 6, 2022

Using AMDL (Beta)

Hidden
Note
This feature is currently in beta and subject to change. It also requires additional activation steps. To learn more about the Beta program for this feature and about activating it for your program, contact your Marqeta representative.

ARIC Model Definition Language (AMDL™) is a language used to specify rules and logic within Real-Time Decisioning. It is a declarative language used to specify state updates and executions on each event that passes through the system.

AMDL can be used to create Business Rules and Workflows. These are two different approaches to fraud and financial crime risk analytics, but both employ broadly the same syntax and rely on the same underlying concepts such as events, entities, and state. For more information including definitions of these terms, see Real-Time Decisioning terminology in Real-Time Decisioning in the Dashboard.

Real-Time Decisioning is an event-driven system, where every event contains a reference (for example, an ID field) to one or more entities of different types (for example, a merchant and a consumer). A Real-Time Decisioning event typically represents some real-world event, such as an account registration or a transaction.

For detailed information on the AMDL elements—including data types, annotations, and methods—see the AMDL Reference.

AMDL expressions

AMDL is made up of expressions, which are the basic building blocks of both Business Rules and Analytical Workflows. The general form of any AMDL expression is as follows:

JSON
Copied

Is this helpful?

Yes
No

An AMDL expression consists of a scope, name, definition, and, optionally, one or more annotations.

  • Scope: The scope of the AMDL expression, denoting the type of expression that is being defined. Different scopes are available depending on the type of AMDL you are writing (Business Rules or Workflows). Some scopes allow you to define expressions, while others only allow you to reference values.

  • Name: The name by which the definition should be referenced in output and other AMDL expressions. A name must be unique within a scope, but can be shared by AMDL expressions of different scope. For instance, state.foo and values.foo could both distinctly exist.

  • Definition: The value to which the AMDL expression evaluates if run.

  • Annotation: Additional information about the AMDL expression that can influence its evaluation behavior. Annotations are optional, and an AMDL expression can have zero, one, or many annotations.

Event data

The Real-Time Decisioning system is event driven. Events within the system must match a defined JSON schema. If a posted event fails to validate against this schema it is rejected by Real-Time Decisioning. The following example event illustrates the structure of the JSON data and provides some examples of data types that can occur in an event, including strings, numbers, datetimes, dates, and booleans.

JSON
Copied

Is this helpful?

Yes
No

Any AMDL expression can refer to data contained in the event payload, using the event.fieldName syntax; for example, the value for the eventTime field in the current event could be referenced as event.eventTime. If there are several nested levels of JSON objects within the event, they can be accessed in a similar way to JSON objects, using periods to denote level transitions. For example, to reference the issuer field in the example transaction event above:

JSON
Copied

Is this helpful?

Yes
No

Alternatively, you can use the square-bracket map accessor syntax (for example, event.topLevelField["level2field"]["level3field"]) to access one or more levels. You may need to do this if the property names are reserved AMDL keywords such as state or values, for example, or if the field names begin with numbers:

JSON
Copied

Is this helpful?

Yes
No
Note
This cannot be used for root-level fields within the event data, as the AMDL parser expects event. at the beginning of any reference to event data.

Tags in output

The purpose of creating AMDL expressions is to trigger effects and actions when an event or a series of events matches a certain set of criteria.

  • Tags are used for two purposes:

    • They appear in the user interface associated with the event they were added to.

    • They are contained in the output produced in response to a real-time event. This means tags can be used to trigger automated decisions such as an automated decline, step-up authentication, dynamic 3D Secure, or other automated response.

Business Rules can trigger tags using annotations. Analytical Workflows can trigger the same actions through scorecards, which can be used to implement a variety of effects. For more information, see Basics of workflows.

Business Rules and Analytical Workflows can also produce other outputs, including scores, and can suppress tag output. For more information on other outputs from Business Rules, see Other effects of business rules.

Comments

Comments can be written in AMDL using //…​ and /. These are ignored by the parser. // causes everything remaining on that line to be interpreted as a comment. / causes everything to be treated as a comment until a closing / is found.

JSON
Copied

Is this helpful?

Yes
No

Conditions

In AMDL, a condition is a boolean expression that evaluates to either true or false. Conditions can be used in various types of AMDL expression. A condition consists of an operator and one or more operands. For a full list of AMDL operators, see Operators in the AMDL Reference.

An example of a simple condition is the following, which is true if the value in the country field in the event is US:

JSON
Copied

Is this helpful?

Yes
No

Conditions can also be numerical comparisons, such as event.amount.baseValue > 100. A boolean variable, or something that returns a boolean value, can also be used in place of a condition. For example, using the example event from the Event data section, this condition can be rewritten to check if a transaction was accepted:

JSON
Copied

Is this helpful?

Yes
No

Or simply write:

JSON
Copied

Is this helpful?

Yes
No

You can link conditions using the boolean operators AND, OR, and NOT. In AMDL, these are represented by && (AND), || (OR), and ! (NOT). For example, this condition is true if the transaction event took place in Duluth, US (the country is "US" and the city is "Duluth"):

JSON
Copied

Is this helpful?

Yes
No

The following condition is true if the event took place in a city called Duluth, or in the US (or both):

JSON
Copied

Is this helpful?

Yes
No

This condition is only true if the event neither took place in the US, nor in a city called Duluth:

JSON
Copied

Is this helpful?

Yes
No

Parentheses ensure that comparisons are executed in the correct order. Here they ensure that the combination of conditions using OR is evaluated first, then the NOT. Parentheses are also needed when you combine different boolean operators, such as AND and OR, to ensure the logic is evaluated correctly. For example, this condition is true if the city is Duluth and the country is US or UK:

JSON
Copied

Is this helpful?

Yes
No

However, this condition is true if the city is Duluth, US (city is Duluth, country is US), or the country is UK (if the country is the UK, the city has no effect):

JSON
Copied

Is this helpful?

Yes
No

In Business Rules AMDL, rules are usually written using conditions. A rule is a condition that, if it evaluates to true, can trigger actions such as tags. For an example, see Example 1: A simple rule.

In Analytical Workflows, a Rule Set can be executed or not executed based on whether an AMDL condition evaluates to true. For an example, see Example 4: Introducing conditions and durations.

Conditions can also be used in the main AMDL Block of a Rule Set, and using Scorecards these can be used to trigger effects such as tags. For more information, see Basics of workflows.

State

State of variables affects their persistence during expression execution: either entity or global.

Entity state

Every entity that is processed by Real-Time Decisioning in an event has its own AMDL state, which is that entity’s AMDL-specific storage area within the Real-Time Decisioning behavioral profile database. This state persists between events. For example, you could store a consumer’s registration time, which is then accessible when processing all future events with that consumer. Alternatively, you could build a history of all the transaction amounts of a merchant over the past month, and access that information in future events. All state variables can be updated by values in an event and in other state variables. AMDL state is automatically initialized by the system the first time it is needed per entity.

State variables are specified using an update expression - an expression providing a single value that is then used to update that state variable. The state variable is updated with the value of the update expression on every event.

Every state definition requires a single update expression, but how the value of that expression is stored depends upon the type of the state variable. The default type of a state variable is a single value - the variable contains the last value it was updated with, and each time the state is updated, that value is overwritten with a new one. However, annotations can be used to modify the type of the variable, as described in Annotations.

JSON
Copied

Is this helpful?

Yes
No
Global state

Comparing an entity’s behavior to fixed threshold values is one way to detect anomalous events and assess fraud and financial crime risk. However, sometimes you need to compare against a background or average value that describes the behavior of the population as a whole. For example, in many cases there might be seasonal variations in average transaction values and volumes, so that comparing transaction values to a fixed threshold might generate a much higher volume of false positives during periods of increased spending such as the run-up to Christmas, Black Friday, or Singles' Day in China.

This is where global state becomes useful. A global state expression defines a single variable that is updated by events for any entity of a given type, so stores information derived from the whole population of entities. One of the main uses of global state is comparing the activity of one entity with a threshold or comparison value derived from the behavior of the whole population.

Note
Global state is not updated as frequently as entity state - for this reason, when referring to a global expression, the value might be up to several seconds out of date. For this reason, globals can be used very effectively to track rolling averages and similar aggregate values, but not to track values or collections that need to be completely up to date at all times. Therefore, it is highly recommended that all global variables are declared as an aggregate type of some sort—for example, a collection or a rolling average declared as an aggregate type.

Syntax

Entity state uses the state scope, and global state uses the globals scope. The exact syntax for state and global expressions depends on whether you are defining this state in the context of Business Rules or Workflows.

State in business rules

For business rules, the syntax for entity and global state is the same as for any other AMDL expression. F or example, to store the timestamp of the last event for each entity:

JSON
Copied

Is this helpful?

Yes
No

This stores state for entities of the type this expression is defined for (remember that AMDL Business Rules expressions are all defined under a specific entity type).

To refer to the stored value of this state expression for the entity of the relevant type in the current event, you can simply use the scope state and the expression name. For example, to refer to the time of the last event for the current entity:

JSON
Copied

Is this helpful?

Yes
No
Important
In Business Rules, when an event is being processed, state is updated after other AMDL expressions have been evaluated. This means that a rule that references state uses the value of the state as it was before the current event was processed. In fact, even when referenced within another state update, a state variable is accessed as it was before the current event. This is necessary to preserve well-defined expression evaluation that it is not dependent upon microsecond variations in the processing speed of each rule. However, this does not diminish the computation power of AMDL because the set of prior state plus the event contents comprises the complete known information about that entity.

Global state expressions are defined in a similar way—for example, to store a rolling average of transaction value across all merchants, this expression could be defined under the merchant entity type:

JSON
Copied

Is this helpful?

Yes
No

This expression uses the @rollingAverage annotation. See Annotations for more information on the use of annotations to modify how state is stored.

This annotation, applied to entity- or global- state variables, keeps a track of the rolling average with a specified time period. This time period must be specified. It cannot keep track of a mean value over all time because it is implemented as an exponential moving average for performance and storage purposes. In general, with global state you want to know about overall trends across the entity population.

State in workflows

For Workflows, which each relate to one or more event types, you can define state expressions for different entity types, enabling each Workflow to update the state for any of the entities present in the relevant event type or types.

This means that you need to define the entity type that a state update expression relates to in the definition of the expression itself. The basic syntax for defining and referring to entity state in Workflows is state.<entity type>.<state expression name>. For example, to store the timestamp of each event in the state for the customer entity type:

JSON
Copied

Is this helpful?

Yes
No

To refer to this state expression, you can use the same syntax, including the entity type:

JSON
Copied

Is this helpful?

Yes
No

Note the use of .single(). This is a method that takes a collection with a single element and returns that element as a single value. For a complete list of methods, see Methods in the AMDL Reference.

This is necessary because state.<entity type>.<state expression name> returns a collection containing the value of the state expression for each entity of the specified type in the event being evaluated. Usually there is only one entity of each type, so you can use the .single() method to flatten that collection to a single value. If there are multiple entities of a given type present in a particular event type, you may need to use other strategies to select the state of the appropriate entity. See Rule references.

Global state works differently. Because the globals scope returns a single value, there is no need to use the .single() method to select one value from the output. However, it is still necessary to define the entity type which the expression relates to within the expression itself - global state is referenced using the syntax globals.<entity type>.<name> to store a global variable for the specified entity type. For example, to store a rolling average of transaction values across all merchants:

JSON
Copied

Is this helpful?

Yes
No

This could then be referenced as globals.merchant.averageAmount.

Important
In Analytical Workflows, state (entity state and global state) is only updated at the very end of each Rule Set evaluation. That means that if you refer to a state expression that is defined in the same Rule Set, the value used in the evaluation of the current event will be that which has been stored from previous events. If you want the state value to be updated by the current event before you reference it in the same Analytical Workflow, you must define the state in one Rule Set and then use it in the following Rule Set in the sequence. This allows the state to be updated. The following Rule Set then receives these updates and executes accordingly.

Annotations

There are a variety of annotations that can be used in AMDL to change when an expression is evaluated, or the effects of evaluating an expression.

Many annotations are available in both Business Rules AMDL and Analytical Workflows, but some annotations are only used in Business Rules. The introductory examples in Basics of business rules and Basics of workflows introduce some of the most important annotations for Business Rules and Workflows respectively, and Annotations gives a complete list of all the annotations that can be used in AMDL.

Annotations can be used to:

  • Change the way a state or global expression stores data This enables you to create expressions that store collections of values or that store a rolling average or only stores a value the first time it is updated, or specify a default value for an expression (see @defaultValue(<defaultValue>) under Annotations in the AMDL Reference). For example, the rolling average annotation changes a state or global expression from one that stores a single value into one that stores an exponentially weighted moving average: @rollingAverage(12h)

  • Change what happens when a rule is evaluated (Business Rules only) Annotations can be used to trigger effects such as tags, to output a risk score, or to suppress tags to prevent them from being generated. These effects are achieved using annotations to AMDL rules; for Analytical Workflows, the same actions can be triggered using Scorecards and Effects.

  • Change when an expression is evaluated (Business Rules only) The @eventType annotation can be used in AMDL Business Rules expressions to limit the evaluation of an expression to a particular event type (or more than one event type). For Analytical Workflows, this action is performed when a Workflow is mapped to an event type or event types.

  • Provide additional information to other users (Business Rules only) Annotations also allow you to add commentary and descriptions to your AMDL Business Rules expressions.

Static values

You can use the AMDL values scope to define static values—constants that do not change unless a user edits the expression. These can be referenced in other AMDL expressions. For example, to define a threshold value that can be used in other expressions:

JSON
Copied

Is this helpful?

Yes
No

These values can then be referenced in other expressions, using the standard reference syntax—for example, values.<nameOfExpression>.

Typically, these are used to prevent repetition of a value across multiple AMDL definitions, and to allow it to be changed easily. For instance, one might have a list of countries that is shared by several rules, and so store it in a values definition to allow easy modification.

Transient variables

Transient variables are AMDL expressions which are re-calculated for each event, and not stored for future events. It is declared in a similar way to entity state, but with the scope var.

For instance, you could calculate the exchange rate used in each event by finding the ratio of the value in the original currency to the value in the base currency:

JSON
Copied

Is this helpful?

Yes
No

Other AMDL expressions, in the var scope and other scopes, can reference the results of these calculations using the standard reference syntax—that is, var.<nameOfVariable>. For example:

JSON
Copied

Is this helpful?

Yes
No

In Business Rules, the use of transient variables can help tidy a set of expressions which all contain similar expressions with slight variations, or which contain references to the result of a common calculation. For example, many expressions might refer to the exchange rate defined above as var.fxRate.

In Workflows, transient variables are important because they provide the easiest way to calculate risk scores and other values for use in Scorecards (see Example 1: A Simple Ruleset for a simple example). For example, to create a Scorecard that makes use of the transaction amount, you would need to create a transient variable that stores that value from the event data:

JSON
Copied

Is this helpful?

Yes
No

The values of transient variables are available to other expressions and Scorecards within the same Rule Set, but not to other Rule Sets or Workflows. Only state and global variables and Support Variables can be updated in one Rule Set and referenced in another within the same Workflow, and only state and global variables can be defined in one Workflow and referenced in another. For more information, see Basics of workflows.

Conditional assignment

For many use cases, you need to be able to conditionally assign a value or update a state expression. For example, you might want to store the value of the most recent accepted transaction, or the time of the last unsuccessful login attempt, or the beneficiary of the most recent payment over $1000 in value.

AMDL uses the ? conditional operator for conditional assignment (see Ternary operator for other uses). This operator acts like an if/then statement in other languages, providing a way to evaluate some expression only if a certain condition is true. The ? operator must be preceded by a boolean expression—that is, a condition or some other expression that evaluates to true or false. It must be followed by the expression or value to evaluate if the condition is true.

For example, to create a state expression that stores the value of the most recent card not present (CNP) transaction:

JSON
Copied

Is this helpful?

Yes
No

Note that this uses the Business Rules syntax for state expression. For more on state expressions, see State.

This state variable is updated only for events where the transactionType field is "CNP". For other events, the condition before the ? operator evaluates to false (or evaluation stops altogether if the transactionType field is missing from that event). When this happens, the expression stops being evaluated and any value currently in the state variable is therefore not overwritten.

When using the conditional operator, you can also supply an expression to evaluate if the condition is false. This enables you to implement an if/then/else logic, storing one value if the condition is true and other if it false. If the evaluation of the initial condition stops, the expression is not updated.

For example, given two event types—a deposit and a withdrawal—in some cases, such as tracking a customer’s balance, you might want to calculate a signed version of the event value, which is negative for a withdrawal and positive for a deposit. You can do this using a transient variable:

JSON
Copied

Is this helpful?

Yes
No

If the event type is deposit, this variable takes its value from event.amount.baseValue; otherwise, if the initial condition is false, it takes the value of this field multiplied by -1.

Dates, times, and durations

Datetimes and durations in AMDL make it simple to compare times and base AMDL logic on the elapsed time between two events. Datetimes in Real-Time Decisioning events, and in AMDL, must be written in ISO-8601 format, such as 2020-02-01T12:34:56Z, and must have a time zone designator. This can be Z for UTC, or an offset in the format + (for time zones head of UTC) or - (for time zones behind UTC) followed by hh for an integer number of hours, or an offset in hours and minutes expressed as hhmm or hh:mm.

Durations, which represent elapsed time, are written by appending a unit to an integer. For example, 7d represents 7 days. The duration units that can be used in AMDL are shown below.

Unit Meaning Example (all = 1 day)

d

Days

1d

h

Hours

24h

m

Minutes

1440m

s

Seconds

86400s

In AMDL, you can subtract one datetime from another to give a duration, and you can add durations to and subtract durations from a datetime to get another datetime. For example, this expression gives the datetime three hours after the datetime of the current event:

JSON
Copied

Is this helpful?

Yes
No

This expression gives the datetime 30 minutes before the current event:

JSON
Copied

Is this helpful?

Yes
No

The expression below shows the use of datetimes and durations in a condition. The condition is true if less than 60 days have elapsed between the current event and the time the customer registered for their account. The first part of the expression evaluates to a duration representing the amount of time elapsed between the eventTime and the accountOpenDateTime; this is then compared to a fixed duration of 60 days (60d).

JSON
Copied

Is this helpful?

Yes
No
Note
The order of the two fields is important. If the account open date is known to be before the time of the current event (which seems likely), this order results in a positive duration; inverting the order would result in a negative duration, which would always be less than 60 days, resulting in incorrect logic.

Collections

In AMDL, collections are data structures that contain multiple values. The simplest type of collection is an array. An array consists of multiple individual elements, which may be of any type. For example, this expression defines an array of strings:

JSON
Copied

Is this helpful?

Yes
No

Using AMDL conditions, you can check whether a collection contains a given value using the "contains" operator, ~#. For example, given the collection above, this condition is true:

JSON
Copied

Is this helpful?

Yes
No

AMDL also has a does not contain operator, !#. For example, the following condition is true:

JSON
Copied

Is this helpful?

Yes
No

Arrays can be defined in-line as part of your AMDL expression. For example, the following condition would be true if the value of the merchantCategoryCode field was one of those listed:

JSON
Copied

Is this helpful?

Yes
No

You can also use annotations to transform AMDL expressions such as state and transient variable definitions into collections. By default, these expressions store a single value, and when the expression is evaluated the stored value is updated and overwritten with the latest value. Collections stored in state allow you to store, for example, the values of a merchant’s 10 most recent transactions, or the unique IDs of all the mobile devices used to access an account.

Arrays

Annotations are used to change the way that data is stored in state and global expressions. The @array annotation is used to turn a state or global expression from one that stores a single scalar value into one that stores multiple values. An argument must be provided to the @array annotation, which can be either a number or a duration. If the argument is a number it specifies the maximum number of values to store in the collection; if the argument is a duration, values are retained for that duration in the collection. Values that have been in the collection for longer than the specified duration are deleted when that expression is referenced or updated.

For example, this AMDL Business Rules expression, defined for the consumer entity type, would store the values of that last 10 transactions for each customer entity:

JSON
Copied

Is this helpful?

Yes
No

This Business Rules expression would store the values of all the transactions seen for each entity over the last 24 hours:

JSON
Copied

Is this helpful?

Yes
No

Sets

While an array is an ordered collection of any values, a set is an unordered collection of unique values. For example, the collection of all unique IP addresses from which an account has been logged into. They are declared in exactly the same manner as arrays, except using the @set annotation.

Sets have varied use cases. For example, a set could be used to store the different payment methods used by a customer over the last 30 days, or the last 10 mobile device IDs that have logged into a particular account, or all the IP addresses a device has logged in from in the course of an hour.

Using collections

A single value can be checked against the elements of these collections using the ~ and ! operators, or using other collection operators described in Operators in the AMDL Reference. This allows you to check whether a value is contained (or not contained) in an array or set, or check all the elements in a collection against a single value—for example, are all the elements in the array greater than 100, or are all the elements in the array less than the value of the current transaction.

The collection methods described in Methods in the AMDL Reference can also be used to extract data from a collection. For example, there are methods for calculating the number of elements in a collection, the total value, the mean, median and mode, and other statistical measures, as well as for sorting collections alphabetically or numerically, or combining collections.

There are also more complex types of collection, including histograms and maps. For more on these collection types, see Histograms and Maps.

Basics of Business Rules

Business Rules AMDL is a way to implement fraud and financial risk detection logic by breaking down that logic into units—that is, AMDL expressions—that each serve a different purpose. These expressions are defined under different scopes such as state and var. When creating or editing Business Rules, each AMDL expression is edited within its own tab inside the AMDL Editor. There cannot be more than one AMDL expression in a tab.

The examples in this section use the TRANSACTION_RT event type described through the example in AMDL expressions, as well as additional event types described later in this section.

Rules

When creating Business Rules AMDL, the most important type of expression is a rule. A rule is an AMDL expression defined in the rules scope, and consists of a boolean expression which is evaluated for each event that passes through the system. Each AMDL expression applies to one particular entity type (for example, customer accounts or merchants), and is evaluated for each entity of that type in each event. If the expression evaluates to true, the rule has 'triggered' on that event. The general syntax of a rule is as follows:

Java
Copied

Is this helpful?

Yes
No
Example 1: A simple rule

As an example, you might want to create a rule that triggers when the value of a transaction exceeds 100 units of your base currency. This rule could be written as follows:

Java
Copied

Is this helpful?

Yes
No

In general it is good practice to give the AMDL definitions descriptive names. In the case, the rule has been named greaterThan100. You could include additional conditions in the rule—for example, you could create a rule that triggers only for an accepted transaction where the amount is over 100 and the currency is USD or GBP:

Java
Copied

Is this helpful?

Yes
No
Adding annotations

You can limit the events for which a rule is evaluated using the eventType annotation. For example, to specify a rule that is only evaluated on the TRANSACTION_RT event type:

Java
Copied

Is this helpful?

Yes
No

You can define a rule that is evaluated for multiple event types by adding additional eventType annotations. For example:

Java
Copied

Is this helpful?

Yes
No

If no event type annotations are specified on a rule, the rule is evaluated on every event. However, there is one problem with this rule—even when it triggers, it does not produce any effect. In Business Rules AMDL, annotations are used to add a tag to the output when a rule triggers.

For instance, you could add tags to the rule as follows:

Java
Copied

Is this helpful?

Yes
No

This would produce a tag with the namespace action and value BLOCK, and another that would simply display as the text High value transaction or account transfer in the dashboard.

The general form of the @tag annotation is @tag(<namespace>="<value>"). You can add multiple tags by adding the @tag annotation multiple times, or you can add multiple tags in the same namespace using the following syntax:

@tag(<namespace>="<value1>","<value2>"…​)

If no namespace is provided, a default namespace (_tag) is used, so if the namespace is unimportant, you can simply write @tag("<value>"). This tag is displayed without a namespace in the dashboard.

Example 2: Storing and using state

In addition to detecting large transactions, as in the previous example, you might want to detect test transactions—in that example, a low-value transaction, intended to test the payment method being used, followed by a large transaction. To detect a low-value transaction followed by a high-value one, you need to be able to store some information about the customer’s previous transaction.

To store the value of the most recent transaction each customer has made, you can write a state expression:

Java
Copied

Is this helpful?

Yes
No

Just as with rules, the @eventType annotation is used here to limit the execution of this expression to transaction events.

Again, the entity type against which you write this expression is important, as the state is stored for the entity of this type. For the following event, if the above expression were written against the customer entity type, the value 100 would be stored for this variable in the state of the entity Customer1:

Java
Copied

Is this helpful?

Yes
No

You can then reference this state expression in a rule, provided that rule is written against the same entity type. This state expression is evaluated for each transaction event, and the transaction value stored against the relevant entity. However, this evaluation is done after the evaluation of rules and other expressions, so when a rule is evaluated the value of the state expressions it references are the values as of the most recent event previously seen for that entity.

For example, this rule triggers if a customer makes a low-value transaction (value less than 10) and their next transaction event is a high-value transaction (value over 100):

Java
Copied

Is this helpful?

Yes
No

You can also write state expressions based on one event type, which are referenced in rules written against another event type. For example, you might want to store some information about a customer when they register, and use that information in the rule.

For example, suppose the registration event contains the following data:

Java
Copied

Is this helpful?

Yes
No

You can store any of this against the customer entity type—for example, you might want to store the customer segment code to determine whether they are a VIP customer when the rules are executed:

Java
Copied

Is this helpful?

Yes
No

This might be used to ensure that the test transaction rule does not apply to VIP customers:

Java
Copied

Is this helpful?

Yes
No
Example 3: Timestamps and elapsed time

The previous example included a rule that would trigger if a customer made a low-value transaction followed by a high-value one, but that rule made no reference to the time elapsed between those two transactions. With a typical test transaction, a high-value transaction is usually made shortly afterwards; a transaction with a low value is unlikely to be a test transaction if the following high-value transaction takes place a week later. Times and durations are therefore important in Business Rules. For more information, see Dates, times, and durations. To store the timestamp of each transaction for each customer, use a state expression:

Java
Copied

Is this helpful?

Yes
No

You can then refer to this state expression in a rule, to determine the elapsed time since the previous transaction. This means that you can modify the rule from example 2 to take into account the elapsed time since the low-value transaction—for example, to make the rule trigger only if it has been less than 2 hours since the previous transaction:

Java
Copied

Is this helpful?

Yes
No

Here, state.previousTransactionTime and event.eventTime are both fields that contain datetimes, and subtracting one from the other gives a duration representing the elapsed time between the two events.

Example 4: Conditional state

In the example above, a rule was created that checks the value and time of the last transaction and triggers if a low-value transaction is followed by a high-value transaction within 2 hours. However, this could fail to detect a test transaction if a customer’s card has been compromised and the card details are being used by both the criminal and the genuine cardholder.

Consider the following sequence of events:

Time Cardholder or criminal? Transaction value Previous transaction value Time since previous transaction Result

10:00

Criminal

$5

N/A

N/A

Does not trigger

10:30

Cardholder

$90

$5

30 mins

Does not trigger

10:45

Criminal

$1000

$100

15 mins

Does not trigger

There are no events in this sequence that match all the conditions in the rule. A better approach might be to keep track of the time since the last low-value transaction, and trigger the rule if a high-value transaction occurs within 2 hours of a low-value one, regardless of whether there were any intervening transactions.

You can use conditional assignment to create state expressions that only store a value when a certain condition is true. This allows you to create an expression that stores the time of the last low-value transaction for each customer:

Java
Copied

Is this helpful?

Yes
No

This expression stores the date and time only if the value of the transaction is less than or equal to 10; if the value is greater than 10, the state expression is not updated. You can then create a version of the rule that refers to this state:

Java
Copied

Is this helpful?

Yes
No

Given the following sequence of events, the rule now triggers:

Time Cardholder or criminal? Transaction value Time since previous transaction Result

10:00

Criminal

$5

N/A

Does not trigger

10:30

Cardholder

$90

30 mins

Does not trigger

10:45

Criminal

$1000

45 mins

Triggers

Note the N/A in the first row. This is because, on the first transaction for a particular customer, the state expression has not yet been evaluated and therefore returns a value of null. This causes the execution of the rule to stop when it reaches the reference to state.previousLowValueTransactionTime, meaning that the rule fails to execute and does not generate any output for this event. Whenever an expression refers to a state variable that has no value for the current entity, or references an event field that is not present in the current event, that undefined value can cause expression execution to stop.

In reality, many state and other variable references return null. In most cases, this is the intended behavior. It makes no sense for the test transaction rule to trigger on the first transaction a customer makes, so when the rule execution stops this does not cause any issues.

However, in some cases, it may be important to be able to deal with or prevent references to state variables, event fields or other variables from returning null and causing the execution of the referring expression to halt.

The default value operator is one approach. You can also use the @defaultValue annotation to provide a default value when defining state and global expressions, so that when an entity or entity type is first seen, the state variable already has a value.

Example 5: Global state and more annotations

Consider the case where you want to create a rule that identifies anomalously high spending. If normal behavior indicated an average entity spend of about $100, you might be inclined to specify a static threshold to trigger a rule if a transaction has a value over 500, for example. However, this might cause problems during times when average spend tends to be higher—during periods of time such as the run up to Christmas, Black Friday, or Singles' Day in China. It might be better in this case to track the average entity spending and compare the single transaction amount to this moving threshold.

To do this, you can define a global expression that stores state from all customers' transactions—for example:

Java
Copied

Is this helpful?

Yes
No

Here, use the @rollingAverage annotation to make the expression calculate an average value instead of storing a value which gets overwritten each time a transaction is processed.

You can then define a rule that generates a tag message for any transaction with a value more than five times the rolling average:

Java
Copied

Is this helpful?

Yes
No
Example 6: Collections

As described in Collections, you can also create collections in the AMDL expressions. These can be useful for storing multiple values or comparing a field in the event data against a list of possible values.

For example, you might want to generate a tag to decline a high-value transaction at a merchant within a high-risk category, as indicated by the Merchant Category Code (MCC):

Java
Copied

Is this helpful?

Yes
No

This rule generates a tag message if the transaction value exceeds five times the global average, and the MCC is one of those listed in the array (using the contains operator, ~#, described in Collections.

To use this list of high-risk MCCs in several expressions, or to separate this definition from the business logic of the rule itself, you could store it as a static value using the values scope (see Collections):

Java
Copied

Is this helpful?

Yes
No

You could then reference this static array in the rule to achieve the same effect as above:

Java
Copied

Is this helpful?

Yes
No

Note that the values expression above must be defined for the same entity type as this rule.

You can also create expressions that store collections in entity or global state. For example, you might want to store a list of all the payment methods used with a customer account over the last year, and use that to trigger a rule if you see a large transaction using a payment method not seen before in that time. To store that information, you would write a state expression using the @set annotation:

Java
Copied

Is this helpful?

Yes
No

You can then write a rule that checks whether the payment method being used in the current transaction event is contained in that set:

Java
Copied

Is this helpful?

Yes
No

Here you can use the does not contain operator, !#, so the rule triggers if the value is over 500 and the payment method is not contained in the collection of payment methods for the current entity.

Suppressing tags

In some cases, you may want to suppress the generation of tags for specific events—for example, to prevent customers on a VIP list from having their transactions held or blocked.

The @suppressTag annotation suppresses tag emission from all sources, including AMDL rules, models, risk score thresholds and aggregators. Suppressed tags are never shown in the dashboard, and are removed from the outputTags key in the JSON response. However, any tags assigned by models are still be listed under tags within the models section of the response.

Because Business Rules are defined for a specific entity type, and tag suppression only work within an entity type—that is, if you write an expression with the @tag annotation for one entity type, and an expression with the @suppressTag annotation against a different entity type, the second expression does not suppress the tag generated by the first, even if they both trigger on the same event.

Rule scores

Rules can be configured to output a score using the @score annotation. The score for each rule is supplied as an argument. Scores may be positive or negative. The total of all the scores from triggered rules for a specific event is output as the businessrules model score. This score is rescaled in the same way that model scores are—a total score of 1 is equal to 100%. For example, for an event with an MCC of 5678 and an amount of $200, the following rules would output a total score of 0.3 (30%):

Java
Copied

Is this helpful?

Yes
No

If the businessrules model score exceeds the overall risk score generated by any other models for an event, it is displayed as the overall risk score in the dashboard for that event. This may result in risk score bars with a score in excess of 100%, or a negative score. While this is displayed appropriately in the dashboard, it may or may not be appropriate to assign events a risk above 100%.

Basics of workflows

Analytical Workflows provide features to help create, update, and manage complex sets of rules. Each Analytical Workflow consists of a list of Rule Sets which are evaluated sequentially and a set of customizable Parameters and Support Variables that pass state between Rule Sets in the Analytical Workflow.

Only one Analytical Workflow ever executes for a single event. Multiple Analytical Workflows may exist, but the system must decide which one to run based on the event type. A single Analytical Workflow can be mapped to one or more event types, but the system does not allow multiple Analytical Workflows to be mapped to the same event type.

Components of a workflow

An analytical workflow consists of rule sets, scorecards, support variables, and parameters.

Rule sets

A Rule Set consists of an AMDL Block, which is where the main logic of the Rule Set is defined, and one or more Scorecards, which allow the Rule Set to create effects such as tags. A Rule Set can also have an attached Condition, which is an AMDL expression that evaluates to true or false. The Rule Set is evaluated only for events where the Condition is true.

Scorecards

Rule Sets create effects such as adding tags by using Scorecards. Each Scorecard references a transient variable (var) defined in the AMDL Block of the Rule Set. Based on the value of this variable, different effects can be triggered.

Different Match Conditions can be defined as part of a Scorecard, and these conditions can have one or more associated Effects. An Effect can be one of the following types:

  • Generate a tag.

  • Suppress a tag.

  • Set an overall risk score.

  • Add to data list override.

  • Update a support variable.

Support variables

Support Variables enable values to be passed from one Rule Set to another in a Workflow, and to determine if and how Rule Sets get executed and what the outcomes are. The variables may be boolean expressions, strings or numerical values, which can be integers or decimals.

They can be referenced in the Conditions, AMDL Block, Match Conditions and Effects of each Rule Set. For each variable, an initial value is set up at the beginning of each Analytical Workflow. This initial value may be modified by Scorecards in each Rule Set, via Effects. That change is then passed on to the next Rule Set and can affect how it executes and the resulting output. An Effect can add to, subtract from, or set a Support Variable before passing it on to the next Rule Set or Scorecard.

Parameters

Parameters are named, immutable values that can be referred to in Workflows, but not updated. They can be used to define thresholds for risk levels as well as to check for a keyword in the event data, or to check that an AMDL expression evaluates to true or false in a given situation. Within Workflow execution, Parameters cannot be modified by AMDL execution or by Effects.

Example 1: A simple rule set

This first example demonstrates the simplest way to implement an effect, such as add a tag, using an Analytical Workflow. This Workflow generates an explanatory tag for any transaction with a value over 1000. In the AMDL Block, create a transient variable that stores a value from the relevant event. Then create a Scorecard that uses this transient variable, and within that Scorecard, define Matcher Conditions that trigger Effects. This Workflow consists of a single Rule Set and is mapped to the transaction event type.

AMDL block

The expression in this AMDL Block simply takes the value of the transaction (in the baseValue field within the event data) and stores it as a transient variable (var.amount) that can then be used in the Scorecard:

var.amount: event.amount.baseValue

Scorecard

The Scorecard makes use of the transient variable to decide whether to trigger Effects—in this case, when the transaction value is over 1000, add a tag for the customer entity type.

Name: transactionAmount

Expression variable: var.amount

Condition Output Effect

Operator

Matcher

Type

Details

Greater than

1000

Tag

Entity type: Customer
Entity type: Customer
Namespace: _tag
Value: High transaction value

Additional effects could be added to the Scorecard. For example, if you want to generate a tag for a value over 1000, but if the value exceeds 5000, add the tag ACTION="decline" to trigger a decline, you could modify the Scorecard as follows:

Condition Output Effect

Operator

Matcher

Type

Details

Between

1000

5000

Tag

Entity type: Customer
Namespace: _tag
Value: High transaction value

Greater than or equal to

5000

Tag

Entity type: Customer
Namespace: ACTION
Value: Decline

Example 2: Multiple scorecards

A single Rule Set can include more than one variable used to trigger effects, and some effects can override others. In this example, add another condition—if the customer making the transaction is a VIP, the transaction should not be declined (the ACTION="decline" tag should be suppressed).

AMDL block

The first expression in this AMDL Block should be familiar from example 1. The second expression (highlighted below) defines another transient variable—a boolean variable that is true if the customer is a VIP, and false otherwise. This expression uses a condition that is true if the value in the field customerSegmentCode is "V". You can then create a second Scorecard using this variable (see the Scorecards below).

Java
Copied

Is this helpful?

Yes
No
Scorecard 1

This scorecard is the same as the one described in Example 1.

Name: transactionAmount

Expression variable: var.amount

Condition Output Effect

Operator

Matcher

Type

Details

Between

1000

5000

Tag

Entity type: Customer
Namespace: _tag
Value: High transaction value

Greater than or equal to

5000

Tag

Entity type: Customer
Namespace: ACTION
Value: Decline

Scorecard 2

This scorecard is based on the transient variable var.VIPcustomer, so if that variable is true—that is, the customer is a VIP—the ACTION="decline" tag should be suppressed. An explanatory tag reading VIP customer is added to indicate why this transaction was not declined. Note that in this example, the High value transaction tag is not suppressed, so it is still added to transactions from VIP customers if the value exceeds 1000.

Condition Output Effect

Operator

Matcher

Type

Details

Equal

true

Suppress tag

Tag

Entity type: Customer
Entity type: Customer
Namespace: ACTION
Value: Decline

Entity type: Customer
Namespace: _tag
Value: VIP customer

Example 3: Storing and using state

Analytical Workflows can store and make use of entity state—that is, behavioral profile data—for entities of all types. This enables data about individual entities to be stored on one execution of a Workflow and used in a subsequent execution of the same Workflow, or even by a different Workflow. See State for more information about creating and referring to state expressions in Analytical Workflows.

In this example, a state expression is used to store the value of each transaction event in the behavioral profile of the customer entity. This means that when the Rule Set runs for a transaction, it can access the value of the previous transaction made by the same customer. This allows test transactions, which use a very small transaction followed by one with a larger value, to be detected.

AMDL block

The AMDL logic for this example includes two additional expressions. The first is a state expression, which stores the value of each transaction. This is similar to the transient variable defined in the first example, but in the state definition you need to include the entity type. In this case, you would need to store the state against the customer entity type to be able to access the customer’s previous transaction value each time they make a transaction.

Note
You can find out the names of your entity types by looking at the Business Rule Profile area in Analytics > Status in the dashboard. For more information, see Real-Time Decisioning in the Dashboard.

The second new expression is a transient variable that uses this stored value. Because state data is only updated at the end of the execution of a Rule Set, this variable always uses the value of the previous transaction for this customer entity.

Java
Copied

Is this helpful?

Yes
No

Note the use of .single() to ensure that the state expression reference returns a single value (see State in workflows).

Scorecard 1

This scorecard is unchanged from the previous example.

Scorecard 2

This scorecard implements a very simple logic—if the transient variable testTransaction is true, a tag is added. In this way, the Workflow triggers a tag message if the transaction amount is between 1000 and 5000 (as defined in Scorecard 1) or if the current transaction amount is over 500 and the previous transaction amount was less than 10.

Name: testTransaction

Expression variable: var.testTransaction

Condition Output Effect

Operator

Matcher

Type

Details

Equal

true

Tag

Entity type: Customer

Scorecard 3

This Scorecard, which suppresses tags if the customer is a VIP, is unchanged from Scorecard 2 in the previous example. Note that this Scorecard has been moved to the end of the list, to ensure that it still overrides any tags generated by other the Scorecards from this Rule Set.

Example 4: Introducing conditions and durations

The previous example showed a Rule Set that added a tag if a customer made a small transaction (value less than 10) followed by a large one (value over 500). This misses an important consideration when looking for test transactions—typically the higher-value transaction happens shortly after the small-value test transaction. A small transaction followed, days later, by a large one, is typically not a risk indicator, but the previous example does not take this into account.

This consideration could be incorporated in a several ways. One way is to split the previous Rule Set into two.

Rule Set 1: transactionValue

This rule set implements the transaction value logic created in Example 1 and is the same as that example, except for the use of the following condition described below.

Condition

This condition is only true if the customer is not a VIP. This means that this Rule Set only executes for non-VIP customers, and therefore tags are not generated on VIP customers' transactions, nor are any of those transactions declined. This replaces the following in the previous example:

event.customerSegmentCode != "V"

AMDL block

The VIP customer logic has been removed from this AMDL Block, and from the corresponding Scorecard.

Java
Copied

Is this helpful?

Yes
No
Scorecards

Scorecard 1 is included unchanged from the previous example. The logic in Scorecard 2 has been moved to the Rule Set below. Scorecard 3 is no longer included, because the logic for suppressing tags for VIP customers is no longer needed.

Rule Set 2: testTransaction

This Rule Set implements the test transaction logic introduced in Example 3, but with the addition of the same condition used in Rule Set 1.

Condition

event.customerSegmentCode != "V"

AMDL block

This AMDL Block has been modified to test whether the previous transaction took place within the last six hours. This is implemented using durations. For more information, see Dates, times, and durations. The new condition added to the definition of var.testTransaction is true if the timestamp of this event (event.eventTime) is less than six hours after the timestamp of the previous transaction that involved the same customer entity (state.customer.previousTransactionTime). This state expression is also defined in the AMDL Block.

Java
Copied

Is this helpful?

Yes
No
Scorecard

This Scorecard implements the same logic as Scorecard 3 in Example 3—if the transient variable testTransaction is true, a tag is triggered.

Name: testTransaction

Expression variable: var.testTransaction

Condition Output Effect

Operator

Matcher

Type

Details

Equal

true

Tag

Entity type: Customer

Example 5: using support variables, risk scores, and final assessment

Support Variables are used in Analytical Workflows to calculate risk scores and other values that need to be updated or referenced by multiple Rule Sets. For example, Support Variables might be used to calculate risk scores for the various entities involved in an event, and then a final Rule Set might combine those individual risk scores into an overall risk score and trigger actions based on that score.

Support Variables are defined when you create your Workflow, and you must assign each one a default value. Support Variables can then be referenced in your Workflow AMDL using the support scope, but AMDL expressions cannot change the value of a Support Variable. To update a Support Variable, you must use Effects in your Scorecard.

In this example, two Support Variables are used to calculate a risk score for the customer and the merchant in a card transaction. These risk scores are then combined into an overall risk score in the final Rule Set of the Workflow, and the final effects—that is, tags—are based on the value of this overall risk score.

Support variables
Name Description Type Initial Value

customerRiskScore

Risk score based on the details of the transaction itself and the customer involved. This contributes to the total risk score for a transaction.

Numeric

0

merchantRiskScore

Risk score based on the history of the merchant in the transaction and their status. This contributes to the total risk score for a transaction.

Numeric

0

Rule Set 1: customerRisk

This rule set looks at transaction value and the risk associated with the customer. If the value is over 5000, it adds 400 to the customer risk score, or if the value is between 1000 and 5000, it adds 200. If the previous transaction for this customer was a low-value transaction within the last six hours (possibly a test transaction), it adds 300. It combines the transaction value and test transaction logic from the previous example (using two separate Scorecards—see below). Note that the logic that checks whether this is a VIP customer has been moved to Rule set 4:_finalRiskScore.

AMDL block
Java
Copied

Is this helpful?

Yes
No
Scorecard 1

Name: transactionValue

Expression variable: var.amount

Condition Output Effect

Operator

Matcher

Type

Details

Greater than

1000

Tag

Entity type: Customer
Namespace: _tag
Value: High transaction value

Greater than

1000, 5000

Update support variable

Variable: customerRiskScore
Action: Add
Value: 200

Greater than

5000

Update support variable

Variable: customerRiskScore
Action: Add
Value: 400

Scorecard 2

Name: testTransaction

Expression variable: var.testTransaction

Condition Output Effect

Operator

Matcher

Type

Details

Equals

true

Tag

Update variable

Entity type: Customer
Namespace: _tag
Value: Preceded by test transaction

 *Variable*: customerRiskScore +
*Action*: Add +
*Value*: 300
Rule Set 2: merchantRisk

This Rule Set looks at the merchant category of the merchant involved in this transaction, and checks whether it is contained in a list of high-risk or medium-risk merchant categories. These lists are provided as arrays of strings within the expression itself, and each condition uses the contains (~#) operator to check whether the MCC of this merchant is contained in each list. For more information, see Collections).

This expression also uses the conditional assignment syntax described in Conditional assignment.

If the merchant category code (MCC) is in a list of high-risk MCCs, it sets the merchant risk score to 375, or if the MCC is on a list of medium-risk MCCs, it sets it to 250.

A simpler version of this check could be implemented by only checking against a single list of high-risk MCCs, as in the following AMDL Block:

Java
Copied

Is this helpful?

Yes
No

This contains a single conditional operator, and simply assigns the transient variable the value 375 if the MCC is on the list at the beginning of the condition, and 0 otherwise. However, the AMDL Block shown below enables you to set the merchant risk score to a different value depending on whether the MCC is high- or medium-risk, using nested conditional operators. If the first condition is true the variable is set to 375; if it is false, the part of the expression after the second colon is evaluated. This starts with another condition, and if this condition is true, the variable is set to 250; otherwise, it is set to 0.

AMDL block
Java
Copied

Is this helpful?

Yes
No
Scorecard 1

This scorecard simply sets the merchant risk score Support Variable to the value of the transient variable set in the AMDL Block.

Name: mccRiskScore

Expression variable: var.MCCRiskFactor

Condition Output Effect

Operator

Matcher

Type

Details

Greater than

0

Update support variable

Variable: merchantRiskScore
Action: set
Value: var.MCCRiskFactor

Tag

Entity type: Merchant
Namespace: _tag
Value: Risky MCC

Rule Set 3: finalRiskScore

This Rule Set calculates the final risk score for a transaction by adding the customer risk score and the merchant risk score, with the merchant score multiplied by a weighting of 0.8. If the total risk score is over 700, it adds a tag to trigger a decline; if the total score is between 300 and 700 it also generates a tag. The Rule Set also outputs the total score for display in the dashboard, using the Overall Score effect.

AMDL block
Java
Copied

Is this helpful?

Yes
No
Scorecard 1

Name: finalRiskScore

Expression variable: var.finalRiskScore

Condition Output Effect

Operator

Matcher

Type

Details

Greater than

1000

Tag

Entity type: Customer
Namespace: _tag
Value: High transaction value

Greater than

1000, 5000

Update support variable

Variable: customerRiskScore
Action: Add
Value: 200

Greater than

5000

Update support variable

Variable: customerRiskScore
Action: Add
Value: 400

Rule set 4: VIPcustomer

This Rule Set prevents VIP customers' transactions from generating a tag or being declined, but only if the customer risk score is low. It has been moved to the end of the list of Rule Sets to ensure that it overrides decline tags from all other Rule Sets in the Workflow, and so that it can reference the customer risk score Support Variable set by Rule Set 1.

Condition

This condition prevents the Rule Set from being executed if the customer risk score is greater than 400. Note the use of the support scope to refer to the current value of the customer risk score Support Variable, which was updated by Rule Set 1 earlier in the execution of the Workflow.

support.customerRiskScore <= 400

AMDL block
Java
Copied

Is this helpful?

Yes
No
Scorecard 1

This scorecard works as before, suppressing the decline tag if the customer is a VIP, and adding an explanatory tag.

Name: VIPcustomer

Expression variable: var.VIPcustomer

Condition Output Effect

Operator

Matcher

Type

Details

Equal

true

Suppress tag

Entity type: Customer

Suppress tag

Entity type: Customer
Namespace: ACTION
Value: decline

Update support variable

Entity type: Customer
Namespace: _tag
Value: VIP customer

Unit Tests

A vital part of writing any code is testing, and the Real-Time Decisioning analytics dashboard provides a useful offline tester that you can use to ensure AMDL expressions are behaving as expected before applying them to a live system. Multiple tests can be attached to any AMDL expression to test your business rules, and tests can also be created for any Rule Set in a Workflow, to test the AMDL expressions in the Rule Set Condition and in the main AMDL Block.

A test consists of a test event, plus optionally some initial state, and optionally one or more tests that check the final state once the AMDL has been executed.

In both Business Rules and Analytical Workflows, the tests panel looks like this:

Test panel

Is this helpful?

Yes
No

All of the tests attached to the AMDL expression or Rule Set are listed in the left panel. The initial state, input event, and expectations are entered in the middle left, middle right, and far right panels respectively.

Basics of AMDL unit tests

The basic principle of AMDL unit tests is to check what the results would be of running a specific AMDL expression (in Business Rules) or AMDL Block (in Analytical Workflows) against your test event.

You can create positive test cases—that is, cases where an event should cause a rule to trigger, or a state variable or Support Variable to be set to a certain value—and negative test cases where a rule should fail to trigger or a variable should not be set or updated.

Tests are saved alongside the expressions for which they are written, and can be exported and imported alongside your Business Rules and Workflows.

The input event

The most important component of any unit test is the input event. This is the event that your AMDL will be evaluated against. The Input Event panel must contain JSON event data, and the event must have an eventType field and be of a type that is defined in your schema.

However, the event is not validated against the schema, enabling you to use minimal events containing only the necessary fields, and even test potential schema changes.

The dashboard event generator tool that forms part of the AMDL unit tester enables you to generate events that match your schema, and set the values of optional and mandatory fields to create appropriate test cases.

Select the Lightning button at the top of the Input Event panel to generate a test event. This is the best way to create routine unit tests for your AMDL code, as it ensures that the test event closely approximates the real events that your AMDL will be evaluated against in production.

Expectations

Each test must define the expected outcome or outcomes, so that the result of the test evaluation can be compared to the expected result to determine if the test passed or failed. These are set in different ways depending on whether you are testing rules or other kinds of expression in AMDL Business Rules or testing Analytical Workflows. See Testing business rules and Testing analytical workflows.

Initial state

The Initial State panel allows you to define values for any other expressions referenced by the AMDL you are testing. Your unit tests have no access to real entities or live state, or the values of any other AMDL expressions. Therefore, you must define values for any referenced state or global variables, transient variables, static values, data lists, support variables, or workflow parameters that you want to have a value in your test.

State and other variables are defined in exactly the same way they are in normal AMDL code, but instead of an expression that evaluates to a value, in the Initial State panel you must set any variable to a fixed value. For example, you could set the value of a transient variable as follows:

Java
Copied

Is this helpful?

Yes
No

For static values (the values scope), you can simply copy and paste the values expression into the Initial State panel. For more information, see Static values.

You can set the value of multiple different variables in different scopes by simply placing each expression on a separate line—for example:

Java
Copied

Is this helpful?

Yes
No
Entities in unit tests

If you have expressions that refer to the state of multiple different entities or entity types, you can set values for these state expressions in the Initial State panel using an annotation specific to unit tests, @entitytype.

This annotation takes two named arguments, the entity type (type) and the entity ID (id), although the entity ID is optional. Any state expressions defined below such an annotation in the Initial State panel will belong to the entity specified in the @entitytype annotation.

For example, to specify the value of the state expression avgTxValue for a merchant entity with the entity ID merchant1:

Test panel

Is this helpful?

Yes
No

This is useful for cross-entity state references in Business Rules and for testing Analytical Workflows. For more information, see Cross-entity state references and Testing analytical workflows.

In Business Rules AMDL tests, the entity the expression is being evaluated for always has the ID testEntity. For example, if a Business Rule is written against the customer entity type, and an event contains two consumer entities, the one the rule is being evaluated for would be identified by having the ID testEntity. Unless you explicitly set the appropriate field in the input event, this entity ID does not match any entity ID fields in the event data, so to test an AMDL expression that checks the value of an entity ID field against the ID of the entity being evaluated against, you must set the value of the appropriate field to testEntity. See Entity meta-variables for an example of expressions where this might be necessary.

In unit tests of Workflows, one entity of each type is assigned the ID testEntity. As with Business Rules, other test entities can be defined using the @entitytype annotation.

Testing business rules

For a positive test case, consider the following rule, which is similar to one introduced in Example 3: Timestamps and durations and conditional state.

Java
Copied

Is this helpful?

Yes
No

There are a number of test cases you might want to create to ensure that this rule works correctly. The simplest is the positive case—a transaction event where the value is greater than 100, the previous transaction value was less than 10, and the previous transaction was less than two hours ago.

The first thing to do when creating a test is to create the input event. For the positive test case, this means a transaction event where the value of amount.baseValue is greater than 100—for example, 150. The event is automatically generated with an eventTime equal to the current time. This example uses placeholder values in all the other fields, providing a test event shown below.

Test business rule

Is this helpful?

Yes
No

To create the positive test case, you also need to specify the values of the state expressions referenced in the rule. You can do this in the Initial State panel.

The correct value of state.previousTransactionValue is simple—it just needs to be less than 10. For this example, state.previousTransactionTime is set to a datetime less than two hours ago. The simplest way to do this is to copy the datetime from the input event, which is 9:55 am on 13th Apr 2022, represented in ISO 8601 format as 2022-04-13T09:55:56.922Z.

By reducing the hour by one, you can set the previous transaction time to 08:55 on the same day, and create a case in which the rule should trigger.

The final step in creating this test is to set the expected outcome. In this case, the rule should trigger, which is set by selecting Check triggers from the Check rule dropdown at the top-right of the tests pane. You can now name this test and run it.

For negative test cases, while it’s important to test that your rule triggers when it should, it’s perhaps more important to test that a rule doesn’t trigger in conditions where you believe it shouldn’t generate a decline tag. This is why negative test cases are important. While it’s often difficult to cover all the possible cases where the rule shouldn’t trigger, it’s important to test some of the more likely, and to identify edge cases that could potentially cause unintended behavior.

For the rule above, you can identify three obvious cases to test:

  • Transaction value is less than 100.

  • Previous transaction value is greater than 10.

  • Previous transaction time is more than 2 hours ago.

The simplest way to create a negative test case, once you’ve created your first positive test case, is to duplicate that first test and then edit it. By duplicating the positive test case described above, you can create each of the three negative test cases by:

  • Changing the amount in the input event to 90.

  • Changing the value of state.previousTransactionValue to 11.

  • Changing the time of state.previousTransactionTime to 06:55 (for example, change the datetime to 2022-04-13T09:55:56.922Z).

For each of these tests, you also need to change the Check rule setting to Check does not trigger.

One other kind of situation you might want to test is the case where one or more of the state expressions referred to in the rule has no value—for example, when this is the first transaction for this particular entity, or, in the case of an optional field, where the event data field referred to is not present.

You can create a simple test case of this kind by duplicating the initial positive test case, and deleting the expressions state.previousTransactionValue and state.previousTransactionTime from the Initial State panel. This simulates the rule’s behavior on the very first transaction for any given customer entity. Again, the Check rule setting needs to be changed to Check does not trigger, since the rule should not trigger when these state variables are undefined.

Negative result

Is this helpful?

Yes
No

In this case, the rule stops executing when it reaches the reference to an undefined state. This results in a case where the rule doesn’t trigger, and hence the test passes, but the test results also contain a warning that the rule did not execute. While this is expected behavior, a rule that fails to evaluate returns null when referred to in another expression. For more about this behavior, see Rule references.

Testing state and global expressions

It is also possible to test that state and global expressions are updating as expected. For instance, you might want to test the state expression introduced in Example 3: Timestamps and durations and conditional state, which stores the datetime of the most recent low-value transaction:

Java
Copied

Is this helpful?

Yes
No

There are two cases to test with this expression—does it store the datetime of the transaction if the value is less than 10, and does the stored value remain the same if the value is greater than or equal to 10?

As with rule testing, some initial state and an event are provided. When testing state and global expression, however, rather than using the Check rule dropdown list, you can instead use the Expectations panel. The Expectations panel must contain one or more AMDL rules, which are evaluated after the input event is processed. These rules are used to check that the final state is equal to the expected value - if all the rules in the Expectations panel trigger, the test passes; if any of these rules fail to trigger, the test fails. The names of these rules must be unique within the scope of a specific test, but can be repeated across multiple tests for the same entity.

For the first test case, you might set an initial value for the state you are testing, or simply leave the Initial State blank. The input event should be a transaction with a value less than 10.

Input event

Is this helpful?

Yes
No

In the Expectations panel, you then write a rule that tests whether the final value of the state is what you expect it to be—in this case, the same as the datetime of the input event. You can copy the datetime directly from the input event JSON when creating the rule, giving something like this:

Java
Copied

Is this helpful?

Yes
No

In the test results, the name of this rule is displayed, and if the rule triggers, a green checkmark is shown next to it. The final value of the state you are testing is also shown. Using a single expectation rule is by far the most common case, but it is possible to add additional rules to this panel to check other conditions to ensure the state was updated correctly.

Results

Is this helpful?

Yes
No

You can also add another test to test the second case, where the transaction value exceeds 10. You can do this by duplicating the first test, changing the transaction value in the input event, and changing the expectation rule to check that the value of the state remains unchanged.

Testing analytical workflows

Unit tests of Analytical Workflows are used to test the outputs from a Rule Set. Each Rule Set can have one or more associated tests, each of which can contain multiple test expressions that check various different outputs and results of evaluating the Rule Set, including the values of transient variables used in Scorecards and values stored in entity and global state.

Like all AMDL unit tests, each test must have an Input Event. The event must be of a type that is mapped to the Workflow you are testing.

As an example, you can create tests to ensure that Rule Set 1 in the Workflow described in Example 6: More complex state and using overridable parameters functions as intended. There are a number of positive and negative test cases to address, and for each, you might want to test a number of outcomes. These are described in the table below, with the highlighted portions indicating where a test is different from the first test case:

Initial state Transaction event Outcomes
  • Default parameter values

  • Previous transaction amount below test transaction threshold

  • Time since previous transaction falls within time threshold

  • No device IDs in state

Mobile device ID not seen before

  • var.amount: equal to transaction amount

  • var.testTransaction: true

  • var.uniqueDeviceIds_count: 1

  • state.uniqueDeviceIds for the customer entity: contains the device ID from the transaction

  • state.lastTransactionAmount for the customer entity: equal to transaction amount

  • state.lastTransactionTime for the customer entity: equal to transaction date- time

  • Default parameter values

  • Previous transaction amount above test transaction threshold

  • Time since previous transaction falls within time threshold

  • Three device IDs in state

Mobile device ID not seen before

  • var.amount: equal to transaction amount

  • var.testTransaction: false

  • var.uniqueDeviceIds_count: 4

  • state.uniqueDeviceIds for the customer entity: contains the device ID from the transaction, has a size of 4

  • state.lastTransactionAmount for the customer entity: equal to transaction amount

  • state.lastTransactionTime for the customer entity: equal to transaction datetime

  • Default parameter values

  • Previous transaction amount below test transaction threshold

  • Time since previous transaction falls within time threshold

  • Three device IDs in state

Mobile device ID seen before

  • var.amount: equal to transaction amount

  • var.testTransaction: false

  • var.uniqueDeviceIds_count: 3

  • state.uniqueDeviceIds for the customer entity: contains the device ID from the transaction, has a size of 3

  • state.lastTransactionAmount for the customer entity: equal to transaction amount

  • state.lastTransactionTime for the customer entity: equal to transaction datetime

The rest of this section walks through the process of creating the last of these test cases. First, an input event must be created. This is a transaction that has a non-zero value (for example, 100), a datetime of 12:00 on 11th April 2022 (2022-04-11T12:00:00.000Z), and for which the mobile device ID field (deviceData.deviceId) is device1.

Initial state

In tests of Analytical Workflows, the Initial State panel allows you to set the initial values of state and global expressions and static values defined in other Rule Sets that are referred to in the Rule Set you are testing. You must also define the initial values of any Support Variables or Parameters referenced in the Rule Set, as the actual values are not accessible to the unit test.

For the example test, the values are set for the two state expressions used to identify test transactions (the previous transaction value and time), and the collection of unique device IDs. State definitions in the Initial State panel are written in the same way as in the body of a Rule Set—for example, you could write

Java
Copied

Is this helpful?

Yes
No

However, you also need to define the values of the Parameters that are referenced in this Rule Set, using the param scope used to reference Parameter values:

Java
Copied

Is this helpful?

Yes
No
Expectations

The Expectations panel in tests of Analytical Workflows uses an additional scope not used anywhere else—the tests scope. Expressions in this scope are boolean conditions—they must evaluate to true or false. A test that evaluates to true has passed, and if all the test expressions evaluate to true, that unit test has passed. This is indicated with a green checkmark in the Results part of the test dashboard. The Results section also shows which test expressions evaluated to true and which evaluated to false, so that if a test fails, you can identify which test condition evaluated to false. For the example tests, test the following outcomes:

  • var.amount is equal to transaction amount (100).

  • var.testTransaction is false.

  • var.uniqueDeviceIds_count is 3.

  • state.uniqueDeviceIds for the customer entity contains the device ID from the transaction and still has a size of 3 (because the device ID in the event has previously been seen).

  • state.lastTransactionAmount for the customer entity is equal to the transaction amount (100).

  • state.lastTransactionTime for the customer entity is equal to transaction datetime (12:00 on 11th April 2022).

The first test needs to check that var.amount is equal to 100. This is just a simple boolean expression, as with a rule, but using the tests scope, defined in the Expectations panel:

Expectations panel

Is this helpful?

Yes
No

You can add the other test conditions as further "tests" expressions:

Java
Copied

Is this helpful?

Yes
No
Check condition

The Check condition dropdown menu at the top right of the tests area enables you to test the AMDL Condition for your Rule Set, if it has one.

  • The default setting is Check matches, which means the unit test checks whether the Rule Set Condition evaluates to true, given the Input Event you have provided. If it does, the Rule Set is evaluated and the test expressions in the Expectations panel is evaluated. If it does not, the Rule Set is not evaluated and the test fails.

  • You can select Check does not match from the dropdown menu. This means that the unit test passes if the Condition evaluates to false. This disables the Expectations panel. If the Condition evaluates to false the Rule Set is not evaluated and therefore no test expressions are required; if the Condition evaluates to true the test has failed and therefore there is no reason to evaluate the test expressions.

The example Rule Set does not have a condition, but the subsequent Rule Set from the same example Workflow in Example 7: Autopass/Autofail. This condition ensures the Rule Set does not execute if either the auto-pass or auto-fail Support Variable is set to true, or if the customer risk score Support Variable exceeds 400. To test this condition, you would need to define the values of these Support Variables in the Initial State panel of the test.

For the positive test case, where the condition should be true and the Rule Set should be evaluated, you need to add appropriate values to the Initial State panel:

Java
Copied

Is this helpful?

Yes
No

There are a number of negative cases you might want to test, where the condition would evaluate to false and the Rule Set would not be evaluated—for example:

Java
Copied

Is this helpful?

Yes
No

Advanced syntax

AMDL provides functionality for more complex usage. The following set of AMDL features provides a greater degree of functionality and makes rules easier to maintain:

Switch case operator

In a situation where a variable can take multiple values, you might want to create an expression that evaluates differently depending on the value. You could do this by chaining multiple conditional statements together (see Conditional assignment). However, more readable expressions can be created using the AMDL switch case operator, as in this example:

Switch case operator

Is this helpful?

Yes
No

The ~? operator initiates the switch on the value of the country field. Each case is terminated by a semicolon. The values switched on must be either a fixed value or default.

The default value is optional. The default keyword means that if the rule evaluation gets to the end of a switch case execution without detecting a match, it uses the default case. If no default is included and a matching case is not found, AMDL execution of the expression stops.

Exists operator

You can use the exists operator ~ to query whether the result of an expression exists or not. This allows you to create conditions that are true if an optional event field exists within the specific event being processed, or if a specific state variable, transient variable or values expression has a value. For instance, you can check whether a field exists in the event by using something like this:

Exists operator

Is this helpful?

Yes
No

The inverse—a condition that is true if a field does not exist—can be constructed using the does not exist operator, !~ (which is really a combination of the exists operator and the NOT boolean operator).

Default value operator

If a reference within an AMDL expression cannot be found (for example, an event field which is not present, or a state which hasn’t been updated yet), it returns a null value. If the null value is not handled in the logic of the expression, AMDL execution of the expression stops and the value of the expression being evaluated also becomes null. If the expression is a rule, it does not trigger; if the expression is a variable update expression such as a state, global or transient variable, that variable is not updated. This may be the desired behavior; however, it is important to be careful you handle such possibilities when needed. In particular, it is important to be aware that AMDL execution does not short-circuit boolean operators and always evaluates all references; if any of these references evaluates to null, expression execution stops.

For example, even if the field accepted has a value in the event being processed, if var.acceptedTransaction is undefined, evaluation of this rule stops and does not trigger:

Java
Copied

Is this helpful?

Yes
No

The default value operator ?? automatically replaces a preceding reference with the value following it, if that reference is found to be null during execution. For example, to ensure that the rule above executes even if the accepted field is not present:

Java
Copied

Is this helpful?

Yes
No

This ensures that the evaluation of the field accepted always returns a value. If the field is missing, the default value false is returned. Note the parentheses. These ensure that the components of the expression are evaluated in the correct order. For more information, see Operator precedence.

First value annotation

Often you might want a state to capture something once and not be updated again. For instance, you might want to store the first login date of a user. This can be done using the exists operator and the ternary operator, but because it is a relatively common occurrence a special annotation is provided, @firstValue.

When this is applied to an AMDL expression, once the expression evaluates to a value, it maintains that value and is not updated any more.

First value annotation

Is this helpful?

Yes
No
Histograms

Arrays and sets store all values they are updated with for the duration. When a large number of values is expected to update the collection in the period of the duration, this could produce excessive storage requirements and adversely affect the engine performance. For instance, globally storing the value of all transactions over a month could result in a very large collection.

To facilitate analysis of such time-series data, AMDL provides a histogram type. A histogram consists of a group of buckets, each of which represents a group of updated values that occurred during a specific time window. It is granular on the level of the buckets, not the values, so can be a powerful approximation to expedite performance. Any state-like AMDL expression can be given a histogram annotation which has the form:

@histogram(historyLength=<duration>, bucketSize=<bucket duration>)

The duration indicates the period for which the histogram should store buckets before expiring them. The second argument determines the number of buckets in the histogram and is optional. If left unspecified, a default is chosen to ensure at least 10 buckets are used. More buckets mean more accurate summary figures but a greater storage/performance impact. Bucket size is limited to a series of acceptable durations. If the specified bucket size is not one of these (see below), the nearest acceptable bucket size is used (ensuring that at least the expected number of buckets are used). The following are acceptable bucket sizes:

  • Monthly buckets: 12 months, 6 months, 4 months, 3 months, 2 months, 1 month

  • Fixed-length buckets: 7 days, 1 day, 12 hours, 6 hours, 3 hours, 2 hours, 1 hour, 30 mins, 20 mins, 15 mins, 10 mins, 5 mins, 1 min, 30 seconds, 5 seconds, 1 second.

Bucket durations expressed in months—for example, 6 months, 12 months—use M as their unit—for example, 6 months is represented as 6M). Histogram buckets align with calendar time periods. Monthly buckets align with calendar months, so the first relevant event that occurs after midnight (UTC) on the first day of the month contributes the first data point to that month’s bucket. Buckets with a shorter duration (expressed in days, hours, minutes or seconds) align with calendar days. All non-monthly histogram buckets align at midnight (UTC) each Monday, so buckets with a duration of 7 days start at midnight on Monday; buckets with a duration of 1 day start at midnight each day; buckets with a duration of 6 hours start at midnight, 6am, noon, and 6pm each day.

As with arrays, you can call the size, total, and mean methods on a histogram state. For instance, if you define a global histogram that stores transactions over the last week as follows:

Histogram state

Is this helpful?

Yes
No

Then you could write a rule that triggers if an entity performs a transaction of amount greater than 1000, but normalized for the day’s global average transaction amount in relation to the week’s:

Histogram state normalized

Is this helpful?

Yes
No

In this case, you have added a duration argument to the mean method. This limits the values of the histogram to those within that elapsed period, so far as the granularity of the bins allow. In the example above, the mean(1d) method returns the mean of values from today; the mean(7d) method returns the mean of values from today and the previous six days.

Monthly durations—for example, 1 month, 6 months—use calendar months, which may be of different durations. To avoid issues when comparing histogram data between months of different lengths, normalized versions of these methods are also provided (normalisedSize, normalisedTotal, normalisedMean), which normalize the total, size, or mean to a month length of 30 days.

Used in this way, these methods can access data from the most recent bucket, or from that bucket and previous buckets. However, for comparison purposes or for defining a baseline, you may want to refer to previous buckets. This can be done in one of two ways.

Providing a datetime as an argument to the size(), mean(), or total() method allows you to access the bucket that contains that datetime. This datetime can be derived from event data through subtraction of a duration, to return data from the bucket from a specified duration in the past. For example, the expressions below enable you to compare the total value of a customer’s transactions this week with the total value from last week.

Histogram total value

Is this helpful?

Yes
No

This example rule triggers a decline tag if the total value of a customer’s transactions so far this week is greater than three times the total value of the customer’s transactions last week. Data is returned for last week’s bucket instead of for the latest bucket by passing the datetime exactly 7 days ago (event.eventTime - 7d) as the argument to the total() method.

However, this approach only allows you to access data from a single bucket. To access data from multiple previous buckets, you can use the atTime() method. This method only applies to histograms, and takes a datetime as an argument. Again, this datetime can be derived by subtracting a duration from a datetime.

For example, you can modify the rule in the previous example to generate a decline tag when the total value of a customer’s transactions this week is greater than the total value of a customer’s transactions over the previous two weeks.

Histogram previous two weeks

Is this helpful?

Yes
No

In this example, the atTime() method with an argument seven days in the past, in combination with total(14d), is used to access data. In this case, the total value) in two buckets, not including the current one - the bucket that includes the datetime supplied as an argument (seven days ago), and the previous bucket, since the bucket size is seven days and the duration supplied as an argument to the total() method is twice that.

Time field

By default, histograms use the event data field eventTime to determine the time at which the current event occurred and hence to determine which bucket to update and/or which bucket or buckets to access.

The @histogram annotation can take an optional argument, timeField, which allows you to define a non-default time field to use. This can be an event data field or a transient variable, specified as a string (for example, to use an event data field called realTime, you would write timeField="event.realTime). For example, particularly with batch processed events, the eventTime might not signify the time that the original transaction took place, so you might want to use another field as the time field:

Java
Copied

Is this helpful?

Yes
No

Alternatively, you might want to use a derived time calculated in a transient variable. For example, if the transient variable newTime contains the datetime to be used as the histogram time field:

Java
Copied

Is this helpful?

Yes
No
Testing histograms

To test an expression that refers to a histogram, you must specify a simulated version of the histogram referred to. For example, the following rule is shown with part of an accompanying test.

Testing histograms

Is this helpful?

Yes
No

The state definition in the Initial State test panel must begin with a copy of the histogram annotation from the state being referred to. The histogram is defined as a JSON map with a single key, data, within which is a series of JSON objects representing the buckets in the histogram. Each bucket has a key, which is the start date(time) of that bucket, and the value is a JSON object containing two properties, size and total.

Maps

A map is a lookup table—given a value, it maps that value to another and returns the other value. This enables you to store key-value states. For instance, in a transactional system, every entity may have several different payment methods, and you might want to keep track of the number of transactions made using each method. Each method identifier could be used as the key, and the count of the number of transactions would be the value—that is, what each key is mapped to. The data in this map might be represented as something like this:

Java
Copied

Is this helpful?

Yes
No

Here, given method1 as a key, transactionCountByPaymentMethod would return the value 10.

AMDL can be used to define both static maps (using the values scope) and dynamic maps that can be updated and modified with each incoming event, stored in entity or global state.

Static maps can be used to store lookup tables such as a currency conversion table, or a dictionary of country codes mapped to country names. For example, you might want to store a list of threshold values for different merchant categories, and raise a decline tag when the value of a transaction exceeds the threshold for the appropriate MCC. This could be implemented using nested conditional statements, although this would be very long and difficult to read, or a switch case statement. For more on switch case statements, see Switch case operator.

However, using a static map can simplify this kind of expression by removing the thresholds into a different expression from the business logic of the rule itself. The map would be defined as follows:

Java
Copied

Is this helpful?

Yes
No

A rule (in AMDL Business Rules) or a transient variable (in Analytical Workflows) can then reference this map using the syntax <scope>.<map expression>[<key>]—for example, values.MCCSpecificThresholds["7999"] would return 300. This rule would generate a decline tag if the value of a transaction exceeds the threshold for the relevant MCC:

Java
Copied

Is this helpful?

Yes
No

This expression could have written using a switch case statement, which would allow you to provide a default value. A map does not provide a way of specifying a default, but you can use the default value operator. This enables you to provide a default value, which is returned if the key referenced is not present in the map. For example, to specify a default threshold of 500:

Java
Copied

Is this helpful?

Yes
No

Maps can also be stored dynamically in state or global expressions. For example, you might want to store the timestamp of the last time a customer has used the transaction method used in the current transaction—a specific payment card or alternative payment method). If each payment method has a unique ID in the event field event.paymentMethod.methodId, you could define a state expression that would store the timestamp of the last transaction for each unique method ID:

state.lastTimeMethodSeen[ event.paymentMethod.methodId ]: event.eventTime

In this example, each time a transaction is received with a particular method ID, the timestamp is stored as the value, with that method ID as the key. After receiving the transactions shown in the table below, the map is stored in state in method2 as shown in the example below the table:

Method ID Timestamp

Method 1

1st Apr 2022 10:01:24

Method 2

5th Apr 2022 08:17:54

Method 3

10th Apr 2022 17:26:12

Java
Copied

Is this helpful?

Yes
No

If you then receive another transaction with the method ID method2, at 15:26:41 on 11th Dec 2021, that overwrites the existing value for that key, and the resulting map looks as follows:

Java
Copied

Is this helpful?

Yes
No

You could then write a rule or other expression referencing this state. You might want to reference the time a particular payment method was last seen, or check whether a payment method has been seen within a certain period of time. The following transient variable stores true if the payment method in the current event has not been seen in the last 180 days:

Java
Copied

Is this helpful?

Yes
No

You can also write an expression that updates values for multiple map keys for a single event, using syntax similar to the following:

Java
Copied

Is this helpful?