How to Create
a Fluent API DSL
with Lambda Builders
BOF 5129

Welcome to the Furets!

@ogunalp – Ozan Gunalp

  • PhD in Computer Science, Jenkins Pipeline Unit author
  • Software Architect at LesFurets.com


@gdigugli – Gilles Di Guglielmo

  • Designer of sweet cooked software since 1999
  • Software Architect at LesFurets.com


@geofberard – Geoffrey Berard

  • Software Developer since 2012
  • Team Lead, React JS Evangelist

  • 1 website, 5 Insurance Products : Car, Health, Home, Bike, Loan
  • 1 codebase, 450k lines of code, 60k unit tests, 150 selenium tests
  • 20 Developers, 2 DevOps, 4 Architects
  • 20 production servers including Load balancers, Frontend, Backend, Databases, BI
  • 1 release per day
  • 9 years of code history
  • 3M quotes/year, 40% of market share, 4M of customers

Introduction

LesFurets service orchestration

LesFurets service orchestration

Insurers partners

We have 100 live insurers, on 5 products, each with business validation rules, that filter prospects based on their profile

Insurer exclusions

Hierarchy of 492 legacy classes, no governance or auditability

Use case: insurer exclusions

Insurer exclusions based on object model (legacy code)

                
public void check(FieldContext context, FormuleMoto formule, Conducteur conducteur,
                  Vehicule vehicule, Void unused, Besoins besoins,
                  Set<EAbTestingScenario> scenarios)
                  throws ExclusionException {
    if (besoins == null) {
        return;
    }
    if (besoins.getDateDebutContrat() == null) {
        return;
    }
    if (!DateHelper.isAfter(besoins.getDateDebutContrat(),
                    DateHelper.ajouteJoursADate(DateHelper.getToday(), NBR_JOURS),
                    DateHelper.EPrecision.jour)) {
        throw new ExclusionException(DATE_EFFET_PLUS_60_JOURS);
    }
}
                
              

Use case: goal

  • compliance: the rules correspond to the specification documents
  • auditability: understand a rule without looking at the code
  • governance: maintenance of the rules catalogue
  • clarity: productivity for developers

Same rule, more fluent:

                
public ExclusionRule exclusionRule() {
    return DOOV.when(dateContrat().after(todayPlusDays(60)))
               .exclusionRule();
}
                
              

Meet dOOv!

Domain Object Oriented Validation

dOOv is a fluent API for typesafe domain model validation

Live code

                
public static boolean validateAccount(User user,
                Account account, Configuration config) {
  if (config == null) {
      return false;
  }
  if (user == null || user.getBirthDate() == null) {
      return false;
  }
  if (account == null || account.getCountry() == null ||
      account.getPhoneNumber() == null) {
      return false;
  }
  if (YEARS.between(user.getBirthDate(), LocalDate.now()) >= 18
          && account.getEmail().length() <= config.getMaxEmailSize()
          && account.getCompany() == Company.LES_FURETS
          && account.getPhoneNumber().startsWith("+33")) {
      return true;
  }
  return false;
}
                
              
                
public static SampleModel sample() {
  User user = new User();
  user.setId(1);
  user.setFirstName("Foo");
  user.setLastName("BAR");
  // ...
            
  Account account = new Account();
  account.setCompany(Company.LES_FURETS);
  account.setId(9);
  // ...
                
              
                
public class Account extends Identity {

  @SamplePath(field = SampleFieldId.LOGIN, readable = "account.login")
  private String login;     
  // ...

  @SamplePath(field = SampleFieldId.COMPANY, readable = "account.company")
  private Company company;
  // ...
                
              
                
SampleModelResourceBundle_en_US.properties

account.login = account login
account.password = account password
account.country = account country
account.company = account company
...

SampleModelResourceBundle_fr_FR.properties

account.login = l'identifiant de connection
account.country = le pays
account.company = la soci\u00e9t\u00e9
...

                
              
                
> ./gradlew -p sample build

BUILD SUCCESSFUL in 23s
10 actionable tasks: 3 executed, 7 up-to-date
                
              
                
public class RulesCodeOneTest {
  SampleModelRule demoRule = DslSampleModel
          .when(DOOV.matchAll(
                  userBirthdate.ageAt(today()).greaterOrEquals(18),
                  accountEmail.length().lesserOrEquals(configurationMaxEmailSize),
                  accountCompany.eq(Company.LES_FURETS),
                  accountPhoneNumber.startsWith("+33")))
          .validate();
}
                
              
                
@Test
public void test_account() {
    SampleModel sample = SampleModels.sample();
              
    Result result = demoRule.executeOn(sample);
              
    Assertions.assertThat(result).isTrue();
    System.out.println(demoRule.readable());
    System.out.println(demoRule.markdown(Locale.FRANCE));
}
                
              
                
rule when match all [user birthdate age at today >= 18, //
  account email length is <= configuration max email size, //
  account company = LES_FURETS, account phone number starts with '+33'] validate
* règle
  * lorsque
    * correspond à tous
      * la date de naissance âge à la date du jour >= 18
      * l'émail a une longueur <= la taille maximum de l'émail
      * la société = LES_FURETS
      * le numéro de téléphone commence par '+33'
  * valider
                
              
                
@Test
public void test_account_failure_cause() {
    SampleModel sample = SampleModels.sample();
    sample.getAccount().setPhoneNumber("+1 12 34 56 78");
              
    Result result = demoRule.executeOn(sample);
              
    Assertions.assertThat(result).isTrue();
}
                
              
                
java.lang.AssertionError: Expected result to be true 
  (invalidated nodes: [account phone number starts with '+33'])
                
              
                
@Test
public void test_account_failure_cause() {
    SampleModel sample = SampleModels.sample();
    sample.getAccount().setPhoneNumber("+1 12 34 56 78");
              
    Result result = demoRule.executeOn(sample);
              
    Assertions.assertThat(result)
            .isFalse()
            .hasFailureCause("account phone number starts with '+33'");
}
                
              
                
@Test
public void test_account_failure_cause_2() {
    SampleModel sample = SampleModels.sample();
    sample.getAccount().setPhoneNumber("+1 12 34 56 78");
    sample.getAccount().setCompany(Company.BLABLACAR);
              
    Result result = demoRule.withShortCircuit(false).executeOn(sample);
              
    Assertions.assertThat(result)
            .isFalse()
            .hasFailureCause("match all [account company = LES_FURETS, " +
                    "account phone number starts with '+33']");
}
                
              

A few predicate reductions examples

Multiple 'and' operator

Validate that a profile
has an email with less than 20 characters
has at least 18 years when their country is France
their country is France when their phone number starts with '+33'

                  
  DOOV.when(accountEmail.length().lesserThan(20)
        .and(userBirthdate.ageAt(today()).greaterThan(18)
          .and(accountCountry.eq(Country.FR)))
        .and(accountCountry.eq(Country.FR)
          .and(accountPhoneNumber.startsWith("+33"))))
      .validate();
                  
                

Multiple 'and' operators

                  
  model.getAccount().setEmail("test@test.org");
  model.getAccount().setCountry(Country.FR);
  model.getUser().setBirthDate(LocalDate.now().minusYears(19));
  
  ValidationRule rule = DOOV
      .when(accountEmail.length().lesserThan(20)  
        .and(userBirthdate.ageAt(today()).greaterThan(18)
            .and(accountCountry.eq(Country.FR)))
        .and(accountCountry.eq(Country.FR)
            .and(accountPhoneNumber.startsWith("+33"))))
      .validate();
  
  Result result = rule.withShortCircuit(false).executeOn(wrapper);
  System.out.println("> " + result.getFailureCause());
                  
                

                  
  > account phone number starts with '+33'
                  
                

'count' operator for predicates

Validate that a profile match at least two conditions :

  • profile has at least 18 years
  • country is France
  • phone number starts with '+33'

                  
  DOOV.when(count(
        userBirthdate.ageAt(today()).greaterThan(18),
        accountCountry.eq(Country.FR),
        accountPhoneNumber.startsWith("+33")).greaterThan(1))
      .validate();
                  
                

'count' operator for predicates

                  
  model.getUser().setBirthDate(LocalDate.now().minusYears(19));
  model.getAccount().setCountry(Country.CAN);
  model.getAccount().setPhoneNumber("1 23 45 67 89");
  
  ValidationRule rule = DOOV
      .when(count(
              userBirthdate.ageAt(today()).greaterThan(18),
              accountCountry.eq(Country.FR),
              accountPhoneNumber.startsWith("+33"))
          .greaterThan(1))
      .validate();
  
  Result result = rule.withShortCircuit(false).executeOn(wrapper);
  System.out.println("> " + result.getFailureCause());
                  
                

                  
  > account country = FR and account phone number starts with '+33'
                  
            

Beyond bean validation

Beyond Bean Validation: types

Bean Validation rules are not strongly typed since it's annotation based. This code will fail at compile time, but your IDE won't be able to tell you why.

                
public class Account {

    @NotNull @Email
    private Email email;

}
                
              

Beyond Bean Validation: complex validation

Because Bean Validation constraints are based on field annotation, cross validation between fields are only available through the extension mechanism.

                
public class Account {

    @Pattern(regexp = "(FR)|(UK)")
    private String country;

    @Pattern(regexp = "???")
    private String phoneNumber;

}
                
              

Beyond Bean Validation: natural language

Bean Validation rules are not written with a natural language syntax and does not provide a syntax tree

                
@Size(min = 10, max = 200,
      message = "About Me must be between 10 and 200 characters")
private String aboutMe;
                
              

DSL

                
userAboutMe().length().between(10, 200).validate().readable()
> When user about me length is between 10 and 200, validate
                
              

Visit our blog for details: https://beastie.lesfurets.com

Beyond Bean Validation: syntax versus consistency

 

Bean Validation allows syntax validation
 
 
Bean Validation rules are written on the model
 
 
Bean Validation uses reflection
dOOv allows consistency checks
 
 
dOOv rules are writtent outside of the model, allowing complex checks
 
 
dOOv uses code generation

Going further

Validate that the company of an account should NOT be
Dailymotion or Blablacar

              
DOOV.when(accountCompany.noneMatch(DAILYMOTION, BLABLACAR)).validate();
              
            

Writting with noneMatch

                
model.getAccount().setCompany(DAILYMOTION);

ValidationRule rule = DOOV
      .when(accountCompany.noneMatch(DAILYMOTION, BLABLACAR))
      .validate();
Result result = rule.withShortCircuit(false).executeOn(wrapper);
System.out.println("> " + result.getFailureCause());
                  
                

                  
  > account company match none  : DAILYMOTION, BLABLACAR
                  
            
                  
DOOV.when(accountCompany.noneMatch(DAILYMOTION, BLABLACAR)).validate();
                  
                

rewritten with notEq + and

                
DOOV.when(accountCompany.notEq(DAILYMOTION)
            .and(accountCompany.notEq(BLABLACAR))).validate();
            
          

              
model.getAccount().setCompany(DAILYMOTION);
Result result = rule.withShortCircuit(false).executeOn(wrapper);
System.out.println("> " + result.getFailureCause());
              
            

                    
  > account company != DAILYMOTION
                    
              
                    
DOOV.when(accountCompany.noneMatch(DAILYMOTION, BLABLACAR)).validate();
                    
                  

rewritten with not + or

                  
DOOV.when(accountCompany.eq(DAILYMOTION)
          .or(accountCompany.eq(BLABLACAR)).not())
             .validate();
              
            

                
model.getAccount().setCompany(DAILYMOTION);
Result result = rule.withShortCircuit(false).executeOn(wrapper);
System.out.println("> " + result.getFailureCause());
                
              

                      
    > not (account company = DAILYMOTION or account company = BLABLACAR)
                      
                
              
DOOV.when(accountCompany.noneMatch(DAILYMOTION, BLABLACAR)).validate();
              
            

rewritten with anyMatch

                    
DOOV.when(accountCompany.anyMatch(LES_FURETS, CANAL_PLUS, MEETIC, OODRIVE))
        .validate();
                
              

                  
model.getAccount().setCompany(DAILYMOTION);
Result result = rule.withShortCircuit(false).executeOn(wrapper);
System.out.println("> " + result.getFailureCause());
                  
                

                
> account company != DAILYMOTION
                
              
                
DOOV.when(accountCompany.noneMatch(DAILYMOTION, BLABLACAR)).validate();
                
              

rewritten with eq + or

                      
DOOV.when(
    accountCompany.eq(LES_FURETS).or(accountCompany.eq(CANAL_PLUS)
            .or(accountCompany.eq(MEETIC).or(accountCompany.eq(OODRIVE)))))
                    .validate();
                  
                

                    
model.getAccount().setCompany(DAILYMOTION);
Result result = rule.withShortCircuit(false).executeOn(wrapper);
System.out.println("> " + result.getFailureCause());
                    
                  

                  
  > account company = LES_FURETS or (account company = CANAL_PLUS 
         or (account company = MEETIC or account company = OODRIVE))
                  
                

Speed is not enough

                  
Benchmark                                     Mode  Cnt     Score      Error   Units
noneMatch                                     thrpt   20  8775.818 ± 148.951  ops/ms
notEq + and  [failure cause OK]               thrpt   20  5022.391 ± 147.550  ops/ms
not + or                                      thrpt   20  3022.433 ± 586.881  ops/ms
anyMatch     [failure cause OK]               thrpt   20  6002.415 ±  94.531  ops/ms
eq + or                                       thrpt   20  1855.837 ±  50.183  ops/ms
                  
                

                    
Benchmark                                     Mode  Cnt     Score      Error   Units
noneMatch    with short circuit               thrpt   20  6839.429 ± 262.318  ops/ms
notEq + and  with short circuit               thrpt   20  7397.586 ± 252.090  ops/ms
not + or     with short circuit               thrpt   20  5084.227 ± 450.013  ops/ms
anyMatch     with short circuit               thrpt   20  6275.185 ±  56.600  ops/ms
eq + or      with short circuit               thrpt   20  1820.198 ±  60.300  ops/ms
                    
                  

notEq + and is the overall winner
anyMatch is the winner only for enumerated values

Conclusion

Goal

We migrated our 492 business rules to dOOv,
we now have compliance, auditability, governance, clarity
(and more!)

dOOv 2.2.0 is out!

  • Gradle code generation support
  • Better failure causes
  • New operators for some types
  • Improved documentation (javadoc and wiki)
  • dOOm beta support

What is dOOm?

Next step is extending the DSL to create an object to object mapping framework, Domain Object Oriented Mapping ( dOOm). It will feature the same AST to text and statistics functionnalities.

                
DOOV.mappings(
  map(userEmail).to(insurerEmail),
  map(userFirstName, userLastName)
      .using(StringJoiner)
      .to(insurerFullName),
  when(userCountry.eq(FR))
    .map(userPhone.to(insurerPhone))
)
                
              

Stay tuned for the next versions!

Enjoy dOOv.io

http://www.dOOv.io

Appendices

DSL

"A domain-specific language (DSL) is a computer language specialized to a particular application domain. This is in contrast to a general-purpose language (GPL), which is broadly applicable across domains"

dOOv ecosystem

How to write a validation rule?

key model

                    
// Root class of model
class Model {
  User user;
}

// Add key named EMAIL
enum ModelFieldId {
  EMAIL;
}

// Annotate email field
class User {
  @Path(field = EMAIL
        readable = ...)
  String email;
}
                    
                  

code generate

                    
// dOOv typed field class
class DslModel {
  StringFieldInfo userEmail;
}
                    
                  

write rules

                    
// Create rules by using
// generated fields
// in DslModel
DslModel
  .when(userEmail.eq(...))
  .validate()
  // Optionaly add rules
  // to a registry
  .registerOn(DEFAULT);
                    
                  

How to validate a model

get model

                    
// Get model from somewhere
// or instanciate it
User user = new User();
user.setEmail("e@mail.com");

Model model = new Model();
model.setUser(user);
                    
                  

execute

                    
// Use executeOn method
DslModel.when(email.matches(...))
        .validate()
        .executeOn(model);

// Or use the registry
DEFAULT.stream()
  .map(rule -> rule.executeOn(model));
                    
                  

Why a fluent API?

Java is verbose, but you can reduce the noise and write code like natural language with a fluent API

                
// JUnit API
assertEquals(9, fellowshipOfTheRing.size());
assertTrue(fellowshipOfTheRing.contains(frodo, sam));
assertFalse(fellowshipOfTheRing.contains(sauron));

// AssertJ API (fluent)
assertThat(fellowshipOfTheRing).hasSize(9)
                               .contains(frodo, sam)
                               .doesNotContain(sauron);
                
              

Fluent API

New elements in Java 8 makes it easier to write a fluent API

                
// java.util.function (io.doov.core.dsl.impl.LogicalBinaryCondition)
left.predicate().and(right.predicate()).test(model, context)

// java.util.stream (io.doov.core.dsl.impl.LogicalNaryCondition)
steps.stream().anyMatch(s -> s.predicate().test(model, context))

// lambda and method reference (io.doov.core.dsl.impl.NumericCondition)
predicate(greaterThanMetadata(field, value),
          (model, context) -> Optional.ofNullable(value),
          (l, r) -> greaterThanFunction().apply(l, r));
                
              

Fluent API

Many popular libraries propose fluent APIs like
jOOQ, AssertJ, Apache Spark, etc.

                
Dataset<Row> averagePrice = prices
        .filter(value.<String>getAs("insurer")
                     .equals("COOL insurer"))
        .groupBy("product")
        .agg(avg("price").as("average"))
        .orderBy(desc("average"));
                
              

Syntax tree

Why a syntax tree?

Makes readable text generation possible:
we can output a multi-language rules catalog
in multiple formats (text, markdown, HTML, etc.)

Execution statistics

We make daily statistics that helps us shape the business,
by removing or tweaking rules as needed

Combined 'and' & 'or' operators

Validate that a profile
has at least 18 years when their country is France
and their phone number starts with '+33'
has at least 21 years when their country is Canadian
and their phone number starts with '+1'

                      
    DOOV.when(userBirthdate.ageAt(today()).greaterThan(18)
            .and(accountCountry.eq(Country.FR)
            .and(accountPhoneNumber.startsWith("+33")))
            .or(userBirthdate.ageAt(today()).greaterThan(21)
                  .and(accountCountry.eq(Country.CAN)
                  .and(accountPhoneNumber.startsWith("+1")))))
        .validate();
                      
                    

Combined 'and' & 'or' operators

                    
    model.getUser().setBirthDate(LocalDate.now().minusYears(22));
    model.getAccount().setCountry(Country.FR);
    
    ValidationRule rule = DOOV
        .when(userBirthdate.ageAt(today()).greaterThan(18)
            .and(accountCountry.eq(Country.FR)
            .and(accountPhoneNumber.startsWith("+33")))
            .or(userBirthdate.ageAt(today()).greaterThan(21)
                  .and(accountCountry.eq(Country.CAN)
                  .and(accountPhoneNumber.startsWith("+1")))))
        .validate();
    
    Result result = rule.withShortCircuit(false).executeOn(wrapper);
    System.out.println("> " + result.getFailureCause());
                    
                  

                    
    > account phone number starts with '+33'
        or (account country = CAN and
            account phone number starts with '+1')
                    
                  

Combined 'matchAll' & 'matchAny' operators

Validate that a profile
has at least 18 years when their country is France
and their phone number starts with '+33'
has at least 21 years when their country is Canadian
and their phone number starts with '+1'

                      
    DOOV.when(matchAny(
            matchAll(userBirthdate.ageAt(today()).greaterThan(18),
                     accountCountry.eq(Country.FR),
                     accountPhoneNumber.startsWith("+33")),
            matchAll(userBirthdate.ageAt(today()).greaterThan(21),
                     accountCountry.eq(Country.CAN),
                     accountPhoneNumber.startsWith("+1"))))
         .validate();
                     
                    

Combined 'matchAll' & 'matchAny' operators

                    
    model.getUser().setBirthDate(LocalDate.now().minusYears(22));
    model.getAccount().setCountry(Country.FR);
    
    ValidationRule rule = DOOV
        .when(matchAny(
            matchAll(userBirthdate.ageAt(today()).greaterThan(18),
                     accountCountry.eq(Country.FR),
                     accountPhoneNumber.startsWith("+33")),
            matchAll(userBirthdate.ageAt(today()).greaterThan(21),
                     accountCountry.eq(Country.CAN),
                     accountPhoneNumber.startsWith("+1"))))
        .validate();
    
    Result result = rule.withShortCircuit(false).executeOn(wrapper);
    System.out.println("> " + result.getFailureCause());
                    
                  

                    
    > match any [account phone number starts with '+33',
                 match all [account country = CAN,
                            account phone number starts with '+1']]
                    
                  

'anyMatch' operator for enumerated values

Validate that a profile country is Canadian or French

                    
    DOOV.when(accountCountry.anyMatch(Country.CAN, Country.FR)).validate();
                    
                  

'anyMatch' operator for enumerated values

                    
    model.getAccount().setCountry(Country.UK);
    
    ValidationRule rule = DOOV
        .when(accountCountry.anyMatch(Country.CAN, Country.FR))
        .validate();
    
    Result result = rule.withShortCircuit(false).executeOn(wrapper);
    System.out.println("> " + result.getFailureCause());
                    
                  

                    
    > account country != UK