tresamigos

Value Members

1. package smv

   

   Main classes

   Spark Modularized View (SMV)

   • Provide a framework to use Spark to develop large scale data applications
   • Synchronize between code management and data management
   • Data API for large projects/large teams
   • Programmatic data quality management

   Main classes

   • org.tresamigos.smv.SmvDFHelper: extends DataFrame methods. E.g. smvSelectPlus
   • org.tresamigos.smv.ColumnHelper: extends Column methods. E.g. smvStrToTimestamp
   • org.tresamigos.smv.SmvGroupedDataFunc: extends GroupedData methods. E.g. smvPivot

   Main packages

   • org.tresamigos.smv.edd: extended data dictionary. E.g. df.edd.summary().eddShow
   • org.tresamigos.smv.dqm: data quality management. E.g. SmvDQM().add(DQMRule(...))