matcher_old

Type Members

1. case class CommonLevelMatcherExpression(expr: Column) extends CommonLevelMatcher with Product with Serializable

   

   Specify the shared matching condition of all the levels (except the top-level exact match)

   Specify the shared matching condition of all the levels (except the top-level exact match)

   expr

   shared matching condition

   Note

   expr should be in "left === right" form so that it can really help on optimize the process by reducing searching space

2. case class ExactLevelMatcher(colName: String, exactMatchExpression: Column) extends LevelMatcher with Product with Serializable

   

   Level match with exact logic

   Level match with exact logic

   colName

   level name used in the output DF

   exactMatchExpression

   match logic colName

3. case class ExactMatchFilter(colName: String, expr: Column) extends AbstractExactMatchFilter with Product with Serializable

   

   Specify the top-level exact match

   Specify the top-level exact match

   colName

   level name used in the output DF

   expr

   match logic condition Column

4. case class FuzzyLevelMatcher(colName: String, predicate: Column, valueExpr: Column, threshold: Float) extends LevelMatcher with Product with Serializable

   

   Level match with fuzzy logic

   Level match with fuzzy logic

   colName

   level name used in the output DF

   predicate

   a condition column, no match if this condition evaluated as false

   valueExpr

   a value column, which typically return a score, higher score means higher chance of matching

   threshold

   No match if the evaluated valueExpr < this value

5. case class SmvEntityMatcher(exactMatchFilter: AbstractExactMatchFilter, commonLevelMatcher: CommonLevelMatcher, levelMatchers: List[LevelMatcher]) extends Product with Serializable

   

   SmvEntityMatcher Perform multiple level entity matching with exact and/or fuzzy logic

   SmvEntityMatcher Perform multiple level entity matching with exact and/or fuzzy logic

   exactMatchFilter

   top level exact match condition, if records matched no further tests will be performed

   commonLevelMatcher

   for all levels (except top level) shared deterministic condition for narrow down the search space

   levelMatchers

   a list of common match conditions, all of them will be tested