protocol buffers

Package

• You can add an optional package specifier to a .proto file to prevent name clashes between protocol message types.
• In C++ the generated classes are wrapped inside a C++ namespace.
  • 即 package 指定的路径，会被转换为 namespace，来避免命名冲突。

Arena Allocation

• 功能：Arena allocation is a C++-only feature that helps you optimize your memory usage and improve performance when working with protocol buffers.
• 原因：By default, protocol buffers performs heap allocations for each message object, each of its subobjects, and several field types, such as strings. These allocations occur in bulk when parsing a message and when building new messages in memory, and associated deallocations happen when messages and their subobject trees are freed.
  • Message 的构造和释放在堆栈上进行，涉及到动态内存的生成和释放，这一步骤非常昂贵。

查看 version

protoc --version

Python

Enum

Could not serialize object: TypeError: can't pickle google.protobuf.pyext._message.EnumDescriptor objects

我在实际 spark 上处理数据时，报错 "Could not serialize object: TypeError: can't pickle google.protobuf.pyext._message.EnumDescriptor objects", 通过报错信息，很明确这是在实际 python 代码中引用 enum 导致的问题.

enum_test.protoview raw

# Add code later

参考教程

• Understand Protobuf enum in python