# API Overview --- This document explains how you use the code generated by the SwiftProtobuf plugin. The generated code relies very heavily on the associated SwiftProtobuf library. Apart from the actual properties themselves, most of the methods described below are not explicit in the generated code. Rather, most of them appear in common shared protocols in the runtime library. They're collected here to make it easier to understand. ## Message API Messages in the input proto file generate Swift structs in the result. These structs conform to `SwiftProtobuf.Message` and provide Swift properties for every field, basic information about the message, standard initializers, and serialization and deserialization methods. Here is a simple proto3 input file to motivate the example below: ```protobuf syntax = "proto3"; message Example { enum E { DEFAULT = 0; } int32 field1 = 1; repeated string field2 = 2; } ``` Here is the API for the struct generated from the above. (As mentioned above, this is not what you'll see if you open up the generated Swift code in your editor. This includes a lot of methods from extensions located in the library, and omits many details of the generated code that are intended purely for internal use by the library.) ```swift public struct Example: SwiftProtobuf.Message { // The generated struct carries constant properties reflecting // basic information about the message: public var protoMessageName: String {return "Example"} // Nested enum and message types are nested in the generated Swift public enum E: SwiftProtobuf.Enum { ... } // A public property is created for each field in the proto. public var field1: Int32 { get set } public var field2: [String] { get set } // Default Initializer public init() // A convenience factory method for constructing // immutable objects. You can use it like this: // // let e = Example.with { // $0.field1 = 7 // $0.field2 = ["foo", "bar"] // } public static with(_ configurator: (inout Example) -> ()); // Messages can be serialized or deserialized to Data objects // using protobuf binary format. // Setting `partial` to `true` will suppress checks for required fields. // An extension map may be needed when decoding nested // proto2-format messages that utilize extensions. // See below for more details. func serializedData() throws -> Data init(serializedData: Data) throws { init(serializedData: Data, extensions: ExtensionMap? = nil, partial: Bool = false) throws // Messages can be serialized or deserialized to JSON format // as either UTF8-encoded Data objects or as Strings. func jsonUTF8Data() throws -> Data init(jsonUTF8Data: Data) throws func jsonString() throws -> String init(jsonString: String) throws // Messages can be serialized or deserialized to Protobuf TextFormat: func textFormatString() -> String init(textFormatString: String) throws // These are the generated methods used internally by the // serialization and deserialization mechanisms. // You should generally not call them directly. public func decodeMessage(decoder: inout D) throws public func traverse(visitor: inout V) throws } func ==(lhs: Example, rhs: Example) -> Bool ``` ### Generated struct name The name of generated struct is based on the name of the message in the proto file. For top-level messages, the name is prefixed with the proto package name as specified in any `package` statements. The name is converted to camel case with underscore separators to preserve the structure. For example, ```protobuf syntax = "proto3"; package my_company.cool_project; message FooBar { ... message Baz { ... } } ``` will by default generate a struct named `MyCompany_CoolProject_FooBar` with another `Baz` struct nested inside it. Note that `Baz` is not prefixed. You can change the prefix with the `option swift_prefix` statement in your proto file: ```protobuf syntax = "proto3"; package my_company.cool_project; option swift_prefix="My"; message FooBar { ... } ``` will generate a struct named `MyFooBar`. (Note: `swift_prefix` is only supported by protoc 3.2 or later.) :warning: The `swift_prefix` option has proven problematic in practice. Because it ignores the `package` directive, it can easily lead to name conflicts and other confusion as your shared proto definitions evolve over time. If the resulting name would collide with a Swift reserved word or would otherwise cause problems in the generated code, then the word `Message` is appended to the name. For example, a `message Int` in the proto file will cause the generator to emit a `struct IntMessage` to the generated Swift file. ## Enum API Proto enums are translated to Swift enums in a fairly straightforward manner. The resulting Swift enums conform to the `SwiftProtobuf.Enum` protocol which extends `RawRepresentable` with a `RawValue` of `Int`. The generated Swift enum will have a case for each enum value in the proto file. If deserialization encounters an unknown value, that value is either ignored (JSON) or handled as an unknown field (protobuf binary), depending on the particular serialization. Proto3 enums have an additional `UNRECOGNIZED(Int)` case that is used whenever an unrecognized value is parsed from protobuf serialization or from other serializations that store integer enum values. Proto2 enums lack this extra case. On decode, proto2 treats unrecognized enum values as if the entire field were unknown. JSON serialization can store named or numeric enum values. Unknown named values are always dropped when deserializing. ```swift public enum MyEnum: SwiftProtobuf.Enum { public typealias RawValue = Int // Case for each value // Names are translated to a lowerCamelCase convention from // the UPPER_CASE convention in the proto file: case default case other case andMore case UNRECOGNIZED(Int) // Only in proto3 enums // Initializer selects the default value (see proto2 and proto3 // language guides for details). public init() public init?(rawValue: Int) public var rawValue: Int public var hashValue: Int public var debugDescription: String } ``` ### Enum and enum case naming The name of the Swift enum is copied directly from the name in the proto file, prefixed with the package name or the name from `option swift_prefix` as documented above for messages. If that name would conflict with a Swift reserved word or otherwise cause problems for the generated code, the word `Enum` will be appended to the name. Enum case names are converted from `UPPER_SNAKE_CASE` conventions in the proto file to `lowerCamelCase` in the Swift code. If the enum case name includes the enum name as a prefix (ignoring case and underscore characters), that prefix is stripped. If the stripped name would conflict with another entry in the same enum, the conflicting cases will have their respective numeric values appended to ensure the results are unique. For example: ```protobuf syntax = "proto3"; enum TestEnum { TEST_ENUM_FOO = 0; TESTENUM_BAR = 1; BAZ = 2; BAR = -3; } ``` becomes ```swift enum TestEnum { case foo = 0 case bar_1 = 1 case baz = 2 case bar_n3 = -3 // 'n' for "negative" } ``` Note #1: Enum aliases can potentially result in conflicting names even after appending the case numeric value. Since aliases are only supported to provide alternate names for the same underlying numeric value, SwiftProtobuf simply drops the alias in such cases. See the protobuf documentation for `allow_alias` for more information about enum case aliases. Note #2: In most cases where an enum case name might conflict with a Swift reserved word, or otherwise cause problems, the code generator will protect the enum case name by surrounding it with backticks. In the few cases where this is insufficient, the code generator will append an additional underscore `_` to the converted name. ## Message Fields Each message field in the `proto` file is compiled into a corresponding property on the generated struct. Field names are converted from `snake_case` conventions in the proto file to `lowerCamelCase` property names in the Swift file. Note: In many cases where the resulting name would cause a problem in the generated Swift, the code generator will protect the field name by surrounding it with backticks. Sometimes, this is insufficient and the code generator will append a `_p` to the converted name. Types in the proto file are mapped to Swift types as follows: ### Basic types | Proto type | Swift Type | | ------------------------- | ------------------ | | int32 | Int32 | | sint32 | Int32 | | sfixed32 | Int32 | | uint32 | UInt32 | | fixed32 | UInt32 | | int64 | Int64 | | sint64 | Int64 | | sfixed64 | Int64 | | uint64 | UInt64 | | fixed64 | UInt64 | | bool | Bool | | float | Float | | double | Double | | string | String | | bytes | Data | ### Generated Types Enums in the proto file generate Int-valued enums in the Swift code. Groups in the proto file generate Swift structs that conform to `SwiftProtobuf.Message`. Messages in the proto file generate Swift structs that conform to `SwiftProtobuf.Message`. Note: There is also a `SwiftProtobuf._MessageImplementationBase` protocol. You should not refer to that directly; use `SwiftProtobuf.Message` when you need to work with arbitrary groups or messages. ### Type modifiers **Proto3 singular fields** generate properties of the corresponding type above. These properties are initialized to the appropriate default value as specified in the proto3 specification: * Numeric fields are initialized to zero. * Boolean fields are initialize to false. * String fields are initialized to the empty string. * Bytes fields are initialized to an empty Data() object. * Enum fields are initialized to the default value (the value corresponding to zero, which must be the first item in the enum). * Message fields are initialized to an empty message of the appropriate type. Notes: For performance, the field may be initialized lazily, but this is invisible to the user. The property will be serialized if it has a non-default value. **Proto2 `optional` fields** generate properties of the corresponding type above. It also generates `has` and `clear` methods that can be used to test whether the field has a value or to reset it to it's default. If a default value was specified in the proto file, the field will be initialized to that value, and will be reset to that value when you invoke the `clear` method. If no default value was specified, the default value is the same as for proto3 singular fields above. **Proto2 `required` fields** Required fields behave the same as optional fields, except that serialization or deserialization may fail if the field is not provided. To illustrate the handling of proto2 fields, consider the following short example: ```protobuf syntax = "proto2"; message ExampleProto2 { optional int32 item_count = 1 [default = 12]; optional string item_label = 2; } ``` This will generate the following field structure in the Swift code: ```swift public struct ExampleProto2 { public var itemCount: Int32 = 12 public var hasItemCount: Bool public mutating func clearItemCount() public var itemLabel: String = ""; public var hasItemLabel: Bool public mutating func clearItemLabel() } ``` ### Compound types **Singular message fields** generate simple properties of the corresponding Swift struct type. The fields are initialized with default instances of the struct. (This initialization is usually done lazily the first time you read such a field.) Message fields generate `has` and `clear` methods as above for both proto2 and proto3. **Proto2 groups** act exactly like messages in all respects, except that they are serialized differently when they appear as a field value. **Proto `repeated` fields** generate simple properties of type `Array` where T is the base type from above. Repeated fields are always initialized to an empty array. **Proto `map` fields** generate simple properties of type `Dictionary` where T and U are the respective key and value types from above. Map fields are always initialized to an empty map. ### Oneof fields Oneof fields generate an enum with a case for each associated field. These enums conform to `ProtobufOneofEnum`. Every case has an associated value corresponding to the declared field. The message will have a read/write property named after the enum which contains the enum value; this property has an optional type and will be `nil` if no oneof field is set. It also will contain a separate read/write computed property for each member field of the enum. Here is a simple example of a message with a `oneof` structure: ```protobuf syntax = "proto3"; message ExampleOneOf { int32 field1 = 1; oneof alternatives { int64 id = 2; string name = 3; } } ``` And here is the corresponding generated Swift code. Note that the two fields `id` and `name` above share storage, thanks to the generated `OneOf_Alternatives` enum type. Also note that you can access the `alternatives` property here directly if you want to use a `switch` construct to analyze the fields contained in the oneof: ```swift public struct ExampleOneOf: SwiftProtobuf.Message { enum OneOf_Alternatives { case id(Int32) case name(String) } var field1: Int32 = 0 var alternatives: OneOf_Alternatives? var id: Int32 { get { if case .id(let v)? = alternatives {return v} else {return 0} } set { alternatives = .id(newValue) } } var name: String { get { if case .name(let v)? = alternatives {return v} else {return ""} } set { alternatives = .name(newValue) } } } ``` ## Well-Known Types For most of the proto3 well-known types, the Swift API is exactly what you would expect from the corresponding proto definitions. (In fact, the runtime library version for most of these is simply generated.) For convenience, most of these also have hand-written extensions that expand the functionality with various convenience methods. The variations from the default generated behavior are described below. | Proto Type | Swift Type | | ------------------------- | ----------------------- | | google.protobuf.Any | Google_Protobuf_Any | | google.protobuf.Api | Google_Protobuf_Api | | google.protobuf.BoolValue | Google_Protobuf_BoolValue | | google.protobuf.BytesValue | Google_Protobuf_BytesValue | | google.protobuf.DoubleValue | Google_Protobuf_DoubleValue | | google.protobuf.Duration | Google_Protobuf_Duration | | google.protobuf.Empty | Google_Protobuf_Empty | | google.protobuf.FieldMask | Google_Protobuf_FieldMask | | google.protobuf.FloatValue | Google_Protobuf_FloatValue | | google.protobuf.Int64Value | Google_Protobuf_Int64Value | | google.protobuf.ListValue | Google_Protobuf_ListValue | | google.protobuf.StringValue | Google_Protobuf_StringValue | | google.protobuf.Struct | Google_Protobuf_Struct | | google.protobuf.Timestamp | Google_Protobuf_Timestamp | | google.protobuf.Type | Google_Protobuf_Type | | google.protobuf.UInt32Value | Google_Protobuf_UInt32Value | | google.protobuf.UInt64Value | Google_Protobuf_UInt64Value | | google.protobuf.Value | Google_Protobuf_Value | For most of these types, you should refer to Google's documentation. Details are provided here to explain details of how these are implemented by SwiftProtobuf. ### Google_Protobuf_Value, Google_Protobuf_Struct, Google_Protobuf_ArrayValue These types can be used for ad hoc encoding and decoding of arbitrary JSON structures. They are particularly useful when dealing with legacy JSON formats where the bulk of the structure is well-defined and maps cleanly into protobuf JSON conventions but there are occasional fields that may contain arbitrary data. ```protobuf syntax = "proto3"; message ExampleAdHocJSON { int32 id = 1; string name = 2; google.protobuf.Struct jsonObject = 3; } ``` `Google_Protobuf_NullValue` is a simple single-value enum that corresponds to `null` in JSON syntax. In particular, `NullValue` and `Value` are the only ways to determine if a `null` appeared in JSON. (In all other circumstances, protobuf JSON decoders treat JSON `null` as either illegal or as a default value for the field.) `Google_Protobuf_Struct` contains a single `fields` dictionary mapping strings to `Google_Protobuf_Value` objects. It also conforms to `ExpressibleByDictionaryLiteral` and provides a `subscript` for directly accessing the values by name. `Google_Protobuf_ArrayValue` is similar, it conforms to `ExpressibleByArrayLiteral` and provides an integer-keyed `subscript` for accessing values by index. The `Google_Protobuf_Value` type can support any JSON type and provides a `oneof` view of the contents. ### Google_Protobuf_Any The `google.protobuf.Any` proto type is provided as `Google_Protobuf_Any`. This type serves as a general container that can store any protobuf message type. For example, suppose you have a message that contains such a field, as defined in the following proto file: ```protobuf syntax = "proto3"; import "google/protobuf/any.proto"; message ExampleAny { string message = 1; google.protobuf.Any detail = 2; } ``` If you have some other (separately-defined) message type `Foo`, you can store one of those objects in the `ExampleAny` struct by wrapping it in a `Google_Protobuf_Any` as follows: ```swift let foo = Foo() var exampleAny = ExampleAny() exampleAny.detail = Google_Protobuf_Any(message: foo) ``` You can then encode or decode the `exampleAny` as usual, even on systems that do not have the definition for `Foo`. Of course, after decoding an `ExampleAny`, you need to inspect the `detail` field and then extract the inner message yourself: ```swift let anyObject = decodedExampleAny.detail if anyObject.isA(Foo.self) { let foo = try Foo(unpackingAny: anyObject) } ``` Caveat: The inner object is not actually decoded until you call the `unpackingAny` initializer. In particular, it is possible for the outer object to decode successfully even when the inner object is malformed. You can also, of course, have `repeated` Any fields or use them in other more complex structures. When coded to JSON format, the Any field will be written in a verbose form that expands the JSON encoding of the contained object. This makes the result easier to read and easier to interoperate with non-protobuf JSON implementations, but means that you cannot translate between binary and JSON encodings without having the type information available. If you need to translate objects between binary and JSON encodings, you should carefully read the documentation comments for `Google_Protobuf_Any.register()` which explains how to make your custom types available to the decode/encode machinery for this purpose. Note: Google's C++ implementation will not decode JSON unless it understands the types of all inner objects. SwiftProtobuf can decode JSON in this case and can re-encode back to JSON. It only needs the types when translating between dissimilar encodings. Caveat: SwiftProtobuf's Text format decoding will currently ignore Any fields if the types are not registered. ### Google_Protobuf_Duration, Google_Protobuf_Timestamp The `Google_Protobuf_Duration` and `Google_Protobuf_Timestamp` structs provide standard ways to exchange durations and timestamps between systems. Following Google's specification, serializing one of these objects to JSON will throw an error if the duration is greater than 315576000000 seconds or if the timestamp is before `0001-01-01T00:00:00Z` or after `9999-12-31T23:59:59.999999999Z` in the Gregorian proleptic calendar. The `Google_Protobuf_Duration` type conforms to `ExpressibleByFloatLiteral`; it can be initialized with a double representing the number of seconds. A `Google_Protobuf_Duration` can be converted to and from a Foundation `TimeInterval`: ```swift extension Google_Protobuf_Duration { public init(timeInterval: TimeInterval) public var timeInterval: TimeInterval {get} } ``` A `Google_Protobuf_Timestamp` can be converted to and from common Foundation timestamp representations: ```swift extension Google_Protobuf_Timestamp { /// To/From a Foundation `Date` object public init(date: Date) public var date: Date /// Relative to POSIX epoch of 00:00:00UTC 1 Jan 1970 public init(timeIntervalSince1970: TimeInterval) public var timeIntervalSince1970: TimeInterval {get} /// Relative to Foundation's "reference date" of 00:00:00UTC 1 Jan 2001 public init(timeIntervalSinceReferenceDate: TimeInterval) public var timeIntervalSinceReferenceDate: TimeInterval {get} } ``` There are also overrides for simple arithmetic with durations and timestamps: ```swift func -(lhs: Google_Protobuf_Timestamp, rhs: Google_Protobuf_Timestamp) -> Google_Protobuf_Duration func -(lhs: Google_Protobuf_Duration, rhs: Google_Protobuf_Duration) -> Google_Protobuf_Duration public func +(lhs: Google_Protobuf_Duration, rhs: Google_Protobuf_Duration) -> Google_Protobuf_Duration public func -(operand: Google_Protobuf_Duration) -> Google_Protobuf_Duration public func -(lhs: Google_Protobuf_Timestamp, rhs: Google_Protobuf_Duration) -> Google_Protobuf_Timestamp public func +(lhs: Google_Protobuf_Timestamp, rhs: Google_Protobuf_Duration) -> Google_Protobuf_Timestamp ``` ## Extensions Extensions are used to add additional properties to messages defined elsewhere. They are fully supported in proto2 files. They are supported in proto3 only when extending the standard Descriptor type. Extensions are ignored when serializing or deserializing to JSON. These are defined in proto2 files as follows: ```protobuf /// File sample.proto syntax="proto2"; message CanBeExtended { extensions 100 to 200; } extend CanBeExtended { optional int32 extensionField = 100; } ``` There are several pieces to the extension support: * **Extensible Messages** (such as `CanBeExtended` above) conform to `ExtensibleMessage` and define some additional methods needed by the other components. You should not use these methods directly. * **Extension properties** use Swift's `extension` capability to add new properties to the original message. In many cases, you can simply use the extension properties without understanding any of the other extension machinery. The above example creates a Swift extension of `CanBeExtended` that defines a new property `extensionField` of type `Optional`. * **Extension objects** are opaque objects that define the extension itself, including storage and serialization details. Because proto allows extension names to be reused in different scopes, these objects appear in the scope corresponding to the context where the proto extension was defined, which generally does not correspond to that of the message being extended. In the above example, the extension object would be `Extensions_extensionField` at the file scope. * **Extension maps** are collections of extension objects indexed by the target message and field number. An extension map is generated by the code generator and included as a static global variable in every Swift source file generated from a proto file that defines extensions. Extension maps are provided to the deserialization APIs to define what extension fields the decoder should recognize. In the above example, the generated extension map would be `Sample_Extensions`. ```swift /// ExtensionMap is a protocol; you can define your own optimized /// form for special applications. The standard implementation in the /// library is `SimpleExtensionMap`. public protocol ExtensionMap { public subscript(messageType: Message.Type, protoFieldNumber: Int) -> MessageExtensionBase? {get} public func fieldNumberForProto(messageType: Message.Type, protoFieldName: String) -> Int? } ``` ## Descriptors Some other languages expose _Descriptor_ objects for messages, enums, fields, and oneof, but not all languages. The `.proto` language also allows developers to add options to messages, fields, etc. that can be looked up at runtime in those descriptors. Support for descriptors ends up requiring some amount of code, but more importantly it requires capturing a large binary blob of data for every message, enum, oneof, etc. That data has two potenial issues, it bloats the binaries, and it is something that can be extracted from the binary to help reverse engineer details about the binary. For these reasons, SwiftProtobuf does not current support anything like the Descriptor objects. It is something that could get revisited in the future, but will need careful consideration; the bloat/size issues is of the most concern because of Swift's common use for mobile applications. ## Aside: proto2 vs. proto3 The terms *proto2* and *proto3* refer to two different dialects of the proto *language.* The older proto2 language dates back to 2008, the proto3 language was introduced in 2015. These should not be confused with versions of the protobuf *project* or the protoc *program*. In particular, the protoc 3.0 program has solid support for both proto2 and proto3 language dialects. Many people continue to use the proto2 language with protoc 3.0 because they have existing systems that depend on particular features of the proto2 language that were changed in proto3.