Schema generation rules

Lots of the work carried out by go-swagger is to generate models, which can have all kinds of rules like polymorphism and validations. Go-swagger models are the go data structures used for serialization and validation.

Of course none of this is possible without a set of rules and trade offs.

About schemas

A schema is a data structure specified in a Swagger document. Loosely speaking, a swagger schema corresponds to a JSONSchema-draft4 schema (see differences below). For each schema, go-swagger will generate one or more model types in go.

NOTE: Swagger makes a distinction between schemas and "simple schemas". We use simple schemas to describe parameters and headers for operations. Simple schemas are supported by go-swagger (including validation), but are not rendered as standalone models. Rather, simple schema structures and validation methods are generated with the operation they are attached to.

Models are dedicated to the description of Swagger schemas, which are described as a parameter body, a response or a spec definition.

Interfaces

The generated models implements:

  • a serialization interface
    • MarshalJSON(), UnmarshalJSON():
      • standard structures use JSON decoder tags (serialization tags are customizable)
      • composed and extensible structures use custom marshalers (allOf, additionalProperties, tuples and polymorphic types)
    • MarshalBinary(), UnmarshalBinary() interfaces (encoding/BinaryMarshaler, encoding/BinaryUnmarshaler), which may use the fast mailru/easyjson package
  • a validation interface (go-openapi/runtime/Validatable), with a Validate(strfmt.Registry) error method

Validation methods are wired at generation time, and rely mostly on native types: this makes validation faster than a dynamic general purpose JSON schema validator.

Example of a generated structure:

// Principal principal
// swagger:model principal
type Principal struct {

    // name
    Name string `json:"name,omitempty"`

    // roles
    Roles []string `json:"roles"`
}

NOTE: if you are looking for a dynamic, fully JSONSchema compliant, general purpose validator, the go-openapi/validate package is what you want. It is fully tested against the JSONSchema-Test-Suite (supports JSON-schema-draft4). See a working example here.

Mapping patterns

The general idea is that you should rarely see interface{} in the generated code: you get a complete representation of a swagger document in somewhat idiomatic go.

There is set of mapping patterns that are applied to transform a spec into go types:

  • definition of primitive => type alias/name
  • definition of array => type alias/name
  • definition of map => type alias/name
  • definition of object with properties => struct
  • definition of $ref => type alias/name
  • object with only additional properties => map[string]T
  • object with additional properties and properties => custom serializer
  • schema with schema array in items => tuple (struct with properties, custom serializer)
  • schema with all of => struct
  • all of schema with ref => embedded value
  • all of schema with properties => properties are included in struct
  • adding an all of schema with just x-isnullable: true or x-nullable: true turns the schema into a pointer when there are only other extension properties provided
  • additional items in tuples => JSONSchema and by extension swagger allow for items that have a fixed size array with schema's describing the items at each index. This can be combined with additional items to form some kind of tuple with varargs. To map this to go it creates a struct that has fixed names and a custom json serializer.

Minimal use of go's reflection

Generated models uses no reflection except for enum and required validations. This makes validation faster.

Doc strings

All documentation items provided by the spec are integrated as godoc-friendly comments in the generated source. You may look at how a trivial example is rendered here.

The code that is generated also gets the same doc comments that are used by the scanner to generate a spec from go code (e.g. comments like // swagger:xxx). So that after generation you should be able to reverse-generate a spec from the code that was generated by your spec. It should be equivalent to the original spec but might miss some default values and examples.

Types reusability

Models can be generated independently from other components of your API. Internal model structures may thus be safely regenerated if the contract at your endpoints does not change.

Previously generated models can be reused when constructing a new API server or client (e.g. using swagger generate server --model=[my existing package]).

The generator makes every effort to keep the go code readable, idiomatic and commented: models may thus be manually customized or extended. Such customized types may be later on reused in other specs, using the x-go-type extension.

Swagger vs JSONSchema

A Swagger 2.0 schema corresponds by and large to a JSON-schema-draft4. However, there are some substantial differences (see swagger):

In JSONSchema, but not in Swagger 2.0:

  • anyOf, oneOf and not constructs are not supported (this is for OpenAPI 3)
  • the null type is not supported (the nullable keyword is defined in OpenAPI 3)
  • additionalItems are not supported (go-swagger does support it)
  • patternProperties are not supported
  • dependencies are not supported
  • multiple types, defined as type: [ ... ] are not supported

Conversely, what we have in Swagger 2.0, but not in JSON-schema:

  • the discriminator attribute controls polymorphism (see below)
  • properties may be given a readOnly attribute (same as in JSONSchema-draft7)
  • array types must have an items restriction
  • format supports more values for strings and numbers

Other minor differences:

  • default values must validate their schema
  • types array must have an items specification
  • extensions may be specified as special x-... annotations. Go-swagger defines some custom tags to customize generated code.
  • other available attributes: example, xml and externalDocs

NOTE: example and externalDocs do not currently influence models generation.

Go-swagger vs Swagger

go-swagger models implements almost all Swagger 2.0 schema features.

We also wanted to support as much JSONSchema features as possible: the model generator may be used independently to generate data structure using the Swagger specification as a serialization description language.

There are some small differences or implementation details to be aware of.

Feature JSON-schema-draft4 Swagger 2.0 go-swagger Comment
"format" Y Y Y Formats are provided by the extensible go-openapi/strfmt package. See also here
"additionalProperties": {schema} Y Y Y Rendered as map[string]T
"additionalProperties": boolean Y Y partial Rendered as map[string]interface{}
"additionalItems": {schema} Y N Y Rendered with tuple models
"additionalItems": boolean Y N partial See extensible types
empty object: { "type": "object"} Y Y Y Rendered as interface{} (anything) rather than map[string]inferface{} (any JSON object, e.g. not arrays)
"pattern" Y Y partial Speed for strictness trade-off: support go regexp, which slighty differ from JSONSchema ECMA regexp (e.g does not support backtracking)
large number, arbitrary precision Y N N
"readOnly" N Y N readOnly is currently not supported by the model validation method
"type": [ "object", ... ] Y N N JSONSchema multiple types are not supported: use Swagger polymorphism instead
implicit type from values in enum Y ? N As of v0.15, when the type is empty, the object is rendered as interface{} and the enum constraint is ignored
tuple `type: "array" items:[...] Y Y partial As of v0.15, incomplete tuples and tuples with array validation are not properly validated

JSONSchema defaults to "additionalProperties": true, go-swagger defaults to ignoring this. Same for additionalItems.

When"additionalProperties": false (resp. "additionalItems": false), uwanted properties (resp. items) do not invalidate data but they are not kept in the model.

This is the default if additionalProperties (resp. additionalItems) is not provided. It is an optimization as it makes the code simpler (and faster) for most use cases. Explicitly specifying true will produce models that retain those additional properties (resp. items).

Known limitations with go-swagger models

Recap as of release 0.15:

  • re Swagger 2.0 specification

    • no validation support for the readOnly attribute
  • re JSON-schema-draft4

    • "additionalProperties": false, "additionalItems": false do not invalidate data with extra properties. We trade strictness for speed and truncate unwanted properties or items without further validation.
    • when enum values cannot be marshalled into their schema, a runtime panic occurs - the go-openapi/validate package does not yet detect this situation
    • minProperties, maxProperties are not supported
    • patternProperties and dependenciesare not supported
    • use of additionalItems requires the --skip-validation flag (go-openapi/validate is strict regarding Swagger specification)
    • JSONSchema defaults to the "additionalProperties": true, go-swagger defaults to ignore this. Same for additionalItems.
    • use of additionalItems requires the --skip-validation flag (go-openapi/validate is strict regarding Swagger specification)
    • array validations (minItems, etc.) are not yet supported for tuples, as of v0.15
    • objects with no properties and no additional properties schema have no validation at all (e.g. passing an array is not invalid) (rendered as interface{})
    • null JSON type: the null type is not supported by Swagger - use of the x-nullable extension makes null values valid (notice that combining the use of required and x-nullable is not fully JSONSchema compliant - see below)

Custom extensions

Model generation may be altered with the following extensions:

  • x-go-name: "string": give explicit type name to the generated model
  • x-go-custom-tag: "string": add serialization tags to an object property
  • x-nullable: true|false (or equivalently x-is-nullable:true|false): accepts null values (i.e. rendered as a pointer)
  • x-go-type: "string": explicitly reuse an already available go type
  • x-class: "string": give explicit polymorphic class name in discriminator
  • x-order: number: indicates explicit generation ordering for schemas (e.g. models, properties, allOf, ...)

Primitive types

Swagger types are rendered as follows by go-swagger:

Swagger type go type
string (no format) string
boolean bool
number float64
number format double float64
number format float float32
integer int64
integer format int64 int64
integer format int32 int32
integer format uint64 uint64
integer format uint32 uint32
file io.ReadCloser(server) or io.Writer (client)
string format binary io.ReadCloseror io.Writer
string with other formats corresponding type exported by go-openapi/strfmt

The file type is exposed as a io.ReadCloser (or io.Writer) interface. The actual implementation in a runtime server or client is provided by the go-openapi/runtime/File type.

Formatted types

The go-openapi/strfmt packages provides a number of predefined "formats" for JSON string types. The full list of formats supported by this package is here

Nullability

Here are the rules that turn something into a pointer.

  • structs
  • x-nullable, x-isnullable: explicit override to accept null values (otherwise not accepted by Swagger)
  • required property
  • extending with allOf a schema with another schema with just x-nullable (or other extensions, but no new properties) turns the schema into a pointer

Primitive types (number, bool and string) are turned into pointers whenever:

  • we need to validate valid zero values vs unset (i.e. the zero value is explicitly checked against validation)

Examples:

definitions:
  myInteger:
    type: integer
    minimum: 0
  myString:
    type: string
    minLength: 0

Yields:

type MyInteger *int64
...
type MyString *string

Notice that the following equivalent does not produce a pointer:

definitions:
  myInteger:
    type: integer
    format: uint64

NOTE: read-only properties are not rendered as pointers.

API developers may use the converter utilities provided by the go-openapi/swag and go-openapi/strfmt/conv packages to manipulate pointers more easily.

Known limitations: pointers are used to distinguish in golang a zero value from no value set.

This design comes with some shortcomings:

  • it is built around the validation use case. In the general case it is not possible to know if a value has been set to a zero value when the type is not a pointer. In cases where this is important, use the x-nullable extension
  • using null as a proxy for unset, makes uneasy the explicit use of the JSON null type Swagger APIs are not supposed to carry null values. go-swagger generated APIs can, using the x-nullable extension, and it is then not possible to distinguish a field explicitly set to null from an unset field

An alternate design has been experimented but not released. For those interested in pushing forward this project again, see this pull request

Validation

All produced models implement the Validatable interface.

Exceptions:

  • file types do not support validation (however generated operations may check the maxLength of a file)
  • empty schemas (any type, rendered as interface{}) do not support validation

Therefore, type aliases constructed on either a swagger file or an empty schema does not implement this interface.

Validation errors:

  • Returned errors are supported by the go-openapi/errors/Error type, which supports errors codes and composite errors.

Validation stops assessing errors down to the property level and does not continue digging all nested strutures as soon as an error is found.

Type aliasing

A definition may create an aliased type like this:

definitions:
  myDate:
    type: string
    format: date

Rendered as:

type MyDate strfmt.Date

NOTE: go 1.9 type aliasing is not being used in generated code at the moment.

Known limitations:

Currently, re-aliasing types (i.e. a definition as a $ref on another definition) works only for objects. Map or slice realiasing still suffers some implementation bugs.

Notice that setting x-nullable: true in such an alias will not render the type itself into a pointer, but rather, all containers of his type will use it as a pointer.

Example:

definitions:
  myDate:
    type: string
    format: date
    x-nullable: true
  anArrayOfDates:
    type: array
    items:
      $ref: '#/definitions/myDate'

Yields:

type MyDate strfmt.Date
...
type AnArrayOfDates []*MyDate

Extensible types

Objects and additional properties

Additional properties in a JSON object are represented in a go struct by map[string]T.

Examples:

definitions:
  extensibleObject:
    properties:
      prop1:
        type:  integer
    additionalProperties:
      type: string
      format: date

Is rendered as:

type ExtensibleObject struct {
    Prop1 int64
    ExtensibleObjectProperties map[string]strfmt.Date
}

If there is no restriction on the additional properties:

definitions:
  extensibleObject:
    type: object
    properties:
      prop1:
        type: integer
    additionalProperties: true

We get:

type ExtensibleObject struct {
    Prop1 int64
    ExtensibleObjectProperties map[string]interface{}
}
Tuples and additional items

A tuple is rendered as a structure with a property for each element of the tuple.

Example:

definitions:
  tuple:
    type: array
    items:
    - type: integer
    - type: string
    - type: string
      format: uuid

Gives:

type Tuple struct {
    P0 *int64
    P1 *string
    P2 *strfmt.UUID
}

If we specify additional items as in:

definitions:
  extensibleTuple:
    type: array
    items:
    - type: integer
    - type: string
    - type: string
      format: uuid
    additionalItems:
    - type: number

Gives:

type ExtensibleTuple struct {
    P0 *int64
    P1 *string
    P2 *strfmt.UUID
    ExtensibleTupleItems []float64
}

NOTE: currently the P0, P1, ... names are not customizable.

Polymorphic types

Polymorphic types are swagger's flavor for inheritance (aka hierarchized composition...). The use of the discriminator keyword gives a special meaning to allOf compositions.

Base types

Whenever the special attribute discriminator is used, this means this object definition is a base type, to be used to extend other types, or subtypes.

The discriminator property indicates which subtype is used whenever an instance of the base type is found. The discriminator's possible values are the names of the subtypes (no aliasing is supported in Swagger 2.0).

NOTE: the discriminator is a required property with "type": "string". No validation attached to this property, save required, will be honored, as a discriminator property is implicitly an enum with all the subtype names found in the spec.

The base type must be a JSON-schema object (it has at least one property, the discriminator). It may define other properties than the discriminator. Like name in this example.

Example:

  Pet:
    type: object
    discriminator: petType
    properties:
      name:
        type: string
      petType:
        type: string
    required:
    - name
    - petType

A base type is rendered as an interface, with getter/setter funcs on all attributes.

// Pet pet
// swagger:discriminator Pet petType
type Pet interface {
    runtime.Validatable

    // name
    // Required: true
    Name() *string
    SetName(*string)

    // pet type
    // Required: true
    PetType() string
    SetPetType(string)
}

NOTE: an unexported reference concrete type is also generated, but not currently used by models.

Subtypes

A subtype extends a base type. It is defined by composing the base type with an allOf construct. All subtypes implement the interface of their base type. So in this examples, all instances of Dog may pretend to be a Pet.

Example:

  Dog:
    type: object
    description: A representation of a dog
    allOf:
    - $ref: '#/definitions/Pet'
    - properties:
        packSize:
          type: integer
          format: int32
          description: the size of the pack the dog is from
          default: 0
          minimum: 0
      required:
      - packSize

Yields:

// Dog A representation of a dog
// swagger:model Dog
type Dog struct {
    nameField *string

    // the size of the pack the dog is from
    // Required: true
    // Minimum: 0
    PackSize *int32 `json:"packSize"`
}

// Name gets the name of this subtype
func (m *Dog) Name() *string {
    return m.nameField
}

// SetName sets the name of this subtype
func (m *Dog) SetName(val *string) {
    m.nameField = val
}

// PetType gets the pet type of this subtype
func (m *Dog) PetType() string {
    return "Dog"
}

// SetPetType sets the pet type of this subtype
func (m *Dog) SetPetType(val string) {

}

Notice the unexported fields which correspond to the description of the base type. The properties of the base type are available with getter/setter functions.

NOTE: if you expand your spec, the allOf semantics are lost. Do not expand specs with polymorphic types for code generation.

You may define several such derived types.

Example:

  cat:
    type: object
    description: A representation of a cat
    allOf:
    - $ref: '#/definitions/Pet'
    - properties:
        huntingSkill:
          type: string
          description: The measured skill for hunting
          default: lazy
          enum:
          - clueless
          - lazy
          - adventurous
          - aggressive
      required:
      - huntingSkill
// Cat A representation of a cat
// swagger:model cat
type Cat struct {
    nameField *string

    // The measured skill for hunting
    // Required: true
    // Enum: [clueless lazy adventurous aggressive]
    HuntingSkill *string `json:"huntingSkill"`
}

// Name gets the name of this subtype
func (m *Cat) Name() *string {
    return m.nameField
}

// SetName sets the name of this subtype
func (m *Cat) SetName(val *string) {
    m.nameField = val
}

// PetType gets the pet type of this subtype
func (m *Cat) PetType() string {
    return "cat"
}

// SetPetType sets the pet type of this subtype
func (m *Cat) SetPetType(val string) {

}

Notice that the value of the discriminator field is case sensitive, e.g. "Dog" and "cat" above.

Type composition

Base types and subtypes may be used in other constructs. While subtypes are mostly handled like ordinary objects, there are special provisions taken to generate new types composing base types.

Example:

  Kennel:
    type: object
    required:
      - pets
    properties:
      id:
        type: integer
        format: int64
      pets:          # <-- this may contain Cats and Dogs
        type: array
        items:
          $ref: "#/definitions/Pet"

Yields:

// Kennel kennel
// swagger:model Kennel
type Kennel struct {

    // id
    ID int64 `json:"id,omitempty"`

    petsField []Pet
}

// Pets gets the pets of this base type
func (m *Kennel) Pets() []Pet {
    return m.petsField
}

// SetPets sets the pets of this base type
func (m *Kennel) SetPets(val []Pet) {
    m.petsField = val
}

NOTE: this representation with unexported fields for references to base types might be subject to change in the future, as it is not consistent in all cases. If you are intested to participate this design work, feel free to comment and express your views here.

Factories for base types

Subtypes and composed types have custom [un]marshallers.

Unmarshalling a base type is not carried through the standard MarshalJSON()/UnmarshalJSON() pair, but with factories created for each base type.

Example:

// UnmarshalPet unmarshals polymorphic Pet
func UnmarshalPet(reader io.Reader, consumer runtime.Consumer) (Pet, error)

// UnmarshalPetSlice unmarshals polymorphic slices of Pet
func UnmarshalPetSlice(reader io.Reader, consumer runtime.Consumer) ([]Pet, error)

Note that the marshalling of a base type into JSON is processed naturally, so there is no need for a special function.

Known limitations: As of v0.15, there are still some known limitations:

  • Unmarshalling maps of base types is not supported at the moment (e.g. UnmarshalPetMap() factory)
  • More complex constructs like [][]Pet, []map[string]Pet are not supported yet
  • composing tuples containing base types is not supported yet

Serialization interfaces

Custom serializers

Tags are generally sufficient to provide proper JSON marshalling capabilities.

Models define some custom [un]marshallers in the following situations:

  • tuples: array elements are dispatched in the tuple's struct
  • additionalProperties: additional content is dispatched in the map[string]...
  • subtypes of base types
  • types composed of base types
  • aliases on formatted types when the underlying type is not string (e.g. Date, Datetime)

results matching ""

    No results matching ""