Apache Avro: Data Serialization for Distributed Applications Training Course

Principles of distributed computing

• Apache Spark
• Hadoop

Principles of data serialization

• How data object is passed over the network
• Serialization of objects
• Serialization approaches
  • Thrift
  • Protocol Buffers
  • Apache Avro
    • data structure
    • size, speed, format characteristics
    • persistent data storage
    • integration with dynamic languages
    • dynamic typing
    • schemas
      • untagged data
      • change management

Data serialization and distributed computing

• Avro as a subproject of Hadoop
  • Java serialization
  • Hadoop serialization
  • Avro serialization

Using Avro with

• Hive (AvroSerDe)
• Pig (AvroStorage)

Porting Existing RPC Frameworks