Class/Object

org.tresamigos.smv

SmvSchema

Related Docs: object SmvSchema | package smv

Permalink

class SmvSchema extends Serializable

CSV file schema definition. This class should only be used for parsing/persisting the schema file associated with a CSV file. It is no longer needed as a general purpose schema definition as spark now has that covered.

Linear Supertypes
Serializable, Serializable, AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. SmvSchema
  2. Serializable
  3. Serializable
  4. AnyRef
  5. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Instance Constructors

  1. new SmvSchema(entries: Seq[SchemaEntry], attributes: Map[String, String])

    Permalink

    entries

    sequence of SchemaEntry. One for each field in the csv file.

    attributes

    sequence of arbitrary key/value mappings (usually contains CSV file attributes such as headers, separator, etc.)

Value Members

  1. final def !=(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any

  2. final def ##(): Int

    Permalink
    Definition Classes
    AnyRef → Any

  3. final def ==(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any

  4. final def asInstanceOf[T0]: T0

    Permalink
    Definition Classes
    Any

  5. val attributes: Map[String, String]

    Permalink

    sequence of arbitrary key/value mappings (usually contains CSV file attributes such as headers, separator, etc.)

  6. def clone(): AnyRef

    Permalink
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  7. val entries: Seq[SchemaEntry]

    Permalink

    sequence of SchemaEntry.

    sequence of SchemaEntry. One for each field in the csv file.

  8. final def eq(arg0: AnyRef): Boolean

    Permalink
    Definition Classes
    AnyRef

  9. def equals(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any

  10. def finalize(): Unit

    Permalink
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )

  11. final def getClass(): Class[_]

    Permalink
    Definition Classes
    AnyRef → Any

  12. def getEntriesStr(): Array[String]

    Permalink

    Returns entries as an array of strings.

    Returns entries as an array of strings. Makes entries consumable by python side.

  13. def hashCode(): Int

    Permalink
    Definition Classes
    AnyRef → Any

  14. final def isInstanceOf[T0]: Boolean

    Permalink
    Definition Classes
    Any

  15. final def ne(arg0: AnyRef): Boolean

    Permalink
    Definition Classes
    AnyRef

  16. final def notify(): Unit

    Permalink
    Definition Classes
    AnyRef

  17. final def notifyAll(): Unit

    Permalink
    Definition Classes
    AnyRef

  18. def saveToFile(sc: SparkContext, path: String): Unit

    Permalink

  19. def saveToHDFSFile(path: String): Unit

    Permalink

  20. def saveToLocalFile(path: String): Unit

    Permalink

  21. def schemaHash: Int

    Permalink

    Used to determine if a file's schema has changed

  22. final def synchronized[T0](arg0: ⇒ T0): T0

    Permalink
    Definition Classes
    AnyRef

  23. def toString(): String

    Permalink
    Definition Classes
    SmvSchema → AnyRef → Any

  24. def toStructType: StructType

    Permalink

    convert SmvSchema to StructType

    convert SmvSchema to StructType

    Annotations
    @DeveloperApi()

  25. final def wait(): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  26. final def wait(arg0: Long, arg1: Int): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  27. final def wait(arg0: Long): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

Inherited from Serializable

Inherited from Serializable

Inherited from AnyRef

Inherited from Any

Ungrouped