|
Dev Essential
1.4.5
|
With DDL (Data Definition Language) a xml datamodel was introduced to describe the semantics of any data that is exchanged between arbitrary software components. The term description is used for a string or a file containing this xml datamodel.
The exchange of the data may be:
The data must have one common characteristic to describe them with the help of DDL: Each data are is a contiguously byte array within memory.
The DDL was introduced with version 1.0+, current version is 4.2.
An overview of changes in specification between the language versions is given below. For detailed information the changes and language features are explained in chapter Specification.
| DDL Version | Description of changes |
|---|---|
| 1.0 | first concept DDL version, never released |
| 1.0+ (1.01) | first released DDL version - add alignment to struct - change optional bitpos default from "1" to "0" |
| 1.02 | XML specification type changes: change some tags from CHAR to STRING and STRING to CLASS |
| 2.0 | - introduce support for dynamic arrays ) - add constant value attribute for element |
| 3.0 | - change alignment behaviour for struct (now it influences the size of the struct in memory) - add ddl_version attribute for struct - introduce min, max, default, scale, offset to element |
| 4.0 | - rearange the element tag with new sub-tag serialized deserialized - change unit in element from mandatory to optional - introduce stream_meta_types |
| 4.1 | - add std types to the predefined datatypes ("bool", "uint8_t", "int8_t", ... "float", "double") |
| 4.2 | - introduce valid_element_count for element to extend array support (see virtual dynamic arrays ) |
The newer versions of the DDL Definition File Format are also specified by a XSD specification file. See the XSD specification for further details:
There are several types that are used in attributes and as tag data in the DDL. The following table lists the types with a description of the values and/or their formatting.
| Type | Allowed values |
|---|---|
| CHAR | single character of [a-z][A-Z][0-9] _.-+/ |
| STRING | characters of [a-z][A-Z][0-9] _.-+/ |
| TEXT | all visible ASCII characters |
| UINT | unsigned integer values |
| INT | signed integer values |
| FLOAT | signed float values |
| DATE | allowed formats: yyyymmdd, dd-mm-yyyy, yyyy-mm-dd or dd.mm.yyyy |
| CLASS | any name value of another DDL element |
| ENUM | a predefined value from the description of the attribute |
| DDLVERSION | Version of 1.0/1.01/1.0+/1.02/2.0/3.0/4.0/4.1/4.2 |
The header section contains meta information about the document and version information. Example:
| Tag | Type | Description |
|---|---|---|
| language_version | DDLVERSION | Version number of the file |
| author | STRING | Author |
| date_creation | DATE | Creation date |
| date_change | DATE | Last modification date |
| description | TEXT | Short description |
All previously mentioned header tags are mandatory.
Additional information can be added by using the ext_declaration tag. Example:
| Attribute | Type | Description |
|---|---|---|
| key | STRING | Name of the additional information |
| Value | TEXT | Value of the additional information |
The definition of units will be divided in SI-base units and own units. The SI-based units are the following ones:
| SI (see ISO 1000) | no SI, but needed for daily usage |
|---|---|
| Metre | Degree |
| Gram | Radiant |
| Second | Unitless |
| Ampere | |
| Candela | |
| Kelvin | |
| Mole |
Units are defined within the tags <units> and </units>. Example:
A concrete base unit definition will be specified by the tag <baseunit> and </baseunit>
The baseunit needs the following mandatory attributes:
| Name | Type | Required | Default | Description |
|---|---|---|---|---|
| name | STRING | mandatory | Name of the base unit e.g. "metre" | |
| symbol | STRING | mandatory | Symbol of the base unit e.g. "m" | |
| description | TEXT | optional | "" | Description of the represented base unit |
Prefixes between 10 power(-24) and 10 power(24) are predefined. A prefix can be defined by the <prefixes> tag.
Every <prefixes> tag needs the following mandatory attributes:
| Name | Value | Required | Description | changes between 1.02 and 1.0+ |
|---|---|---|---|---|
| name | STRING | mandatory | Name of the prefix | |
| symbol | STRING | mandatory | Represents a short symbol e.g. k | changed to STRING from CHAR in DDL1.02 |
| power | INT | mandatory | Defines the power of the prefix |
A self defined unit is specified within the <unit> and </unit> tag and needs the following mandatory attributes:
| Name | Value | Required | Description |
|---|---|---|---|
| name | STRING | mandatory | Name of the new unit |
The <unit> tags needs the following mandatory sub-tags:
| Name | Value | Required | Description |
|---|---|---|---|
| numerator | STRING | mandatory | containing pi/PI or a floating-point value (Numerator of the new unit related to the baseunits) |
| denominator | STRING | mandatory | containing pi/PI or a floating-point value. The value '0' is not defined. (Denominator of the new unit related to the baseunits) |
| offset | FLOAT | mandatory | Offset to the baseunits |
The new unit is able to use several base units. To represent this, it is possible to specify the related base units by the <refUnit> tag. This tag uses the following mandatory attributes:
| Name | Value | Description | changes between 1.02 and 1.0+ |
|---|---|---|---|
| name | CLASS | The referenced unit | changed to CLASS from STRING in DDL 1.02 |
| power | INT | Power of the new unit related to the base one | |
| prefix | CLASS | Reference to the prefix to use | changed to CLASS from STRING in DDL1.02 |
The newly defined unit relates to the SI base units like this:
newUnit = offset + (numerator / denominator) * Product (prefix(n) * baseUnit(n) ^ power(n))
The following base units are provided as default:
| Base Unit Name | Description | Symbol |
|---|---|---|
| Metre | Fundamental unit for length | m |
| Kilogram | Fundamental unit for mass | kg |
| Second | Fundamental unit for time | s |
| Ampere | Fundamental unit for electric current | A |
| Kelvin | Fundamental unit for thermodynamic temperature | K |
| Mole | Fundamental unit for amount of substance | mol |
| Candela | Fundamental unit for luminous intensity | cd |
| Degree | Non-SI standard unit for angle | deg |
| Radiant | Non-SI standard unit for angle | rad |
| Unitless | No SI, but needed for own unit definitions | |
| nou | No SI, but needed for no unit definitions |
The following prefixes are provided as default:
| Prefix Name | Power | Symbol |
|---|---|---|
| yotta | 24 | Y |
| zetta | 21 | Z |
| exa | 18 | E |
| peta | 15 | P |
| tera | 12 | T |
| giga | 9 | G |
| mega | 6 | M |
| kilo | 3 | k |
| hecto | 2 | h |
| deca | 1 | da |
| deci | -1 | d |
| centi | -2 | c |
| milli | -3 | m |
| micro | -6 | u |
| nano | -9 | n |
| pico | -12 | p |
| femto | -15 | f |
| atto | -18 | a |
| zepto | -21 | z |
| yocto | -24 | y |
This section describes the primitive data types which can be used within the struct elements. Example:
| Name | Type | Required | Description | changes between 2.0 and 3.0 | changes between 1.0+ and 1.0 |
|---|---|---|---|---|---|
| name | STRING | mandatory | Name of the primitive data type | attribute name changed to "name" from "type" | |
| size | UINT | mandatory | Number of bits (relevant for serialization) | ||
| description | STRING | optional | Description of the primitive data type | ||
| arraysize | UINT | ignored | = 1 -> primitive presentation> 1 -> array with number of elements. This feature is not supported within any DDL. | ||
| unit | CLASS | optional | Referenced unit of the data type | ||
| min | STRING | optional | Minimum value of the data type | introduced in DDL 3.0 | |
| max | STRING | optional | Maximum value of the data type | introduced in DDL 3.0 |
The following predefined data types are automatically used within DDL. Since DDL 4.1 also the standard data type names can be used.
| Type | also since DDL 4.1 | Description | Number of bits |
|---|---|---|---|
| tBool | bool | boolean | 8 (C++ data type) |
| tChar | char | character | 8 |
| tInt8 | int8_t | signed integer | 8 |
| tUInt8 | uint8_t | unsigned integer | 8 |
| tInt16 | int16_t | signed integer | 16 |
| tUInt16 | uint16_t | unsigned integer | 16 |
| tInt32 | int32_t | signed integer | 32 |
| tUInt32 | uint32_t | unsigned integer | 32 |
| tInt64 | int64_t | signed integer | 64 |
| tUInt64 | uint64_t | unsigned integer | 64 |
| tFloat32 | float | IEEE Float | 32 |
| tFloat64 | double | IEEE Float | 64 |
This section describes the enum type which can be used within the struct elements. Example:
Enum Attributes
| Name | Type | Required | Description |
|---|---|---|---|
| name | STRING | mandatory | Name of the enum |
| type | CLASS | mandatory | Referenced data type of the enum |
Enum Element Attributes
| Name | Type | Required | Description |
|---|---|---|---|
| name | STRING | mandatory | Name of the element |
| value | INT or UINT | mandatory | Value of the element |
Remarks:
This section describes constants which are implemented using the enum type. Example:
The definition of structs allows to build complex data types. Example:
The tag <struct> uses the following attributes:
| Name | Type | Required | Description | changes between 2.0 and 3.0 | changes between 1.0+ and 1.0 |
|---|---|---|---|---|---|
| name | STRING | mandatory | Description of the data type | ||
| version | UINT | mandatory | Version number of the specified struct type, by default is "1" | ||
| comment | TEXT | optional | Additional comments | ||
| alignment | ENUM of 1/2/4/8/16/32/64 (default is "1") | optional | Alignment value to get the whole packing of the complex data type which is important to get the calculated size of the structure (relevant for serialization) | From version 3.0 on, the alignment influences the size of the struct. The size will always be a multiple of the alignment. | introduced in DDL 1.0+ |
| ddlversion | STRING | optional | The version of the size calculation scheme, see alignment. If not specified the version from the containing definition will be used. |
Remarks:
The tag <element> uses the following attributes:
| Name | Type | Required | Description | changes between 4.2 and 4.1 | changes between 3.0 and 4.0 | changes between 2.0 and 3.0 | changes between 1.02 and 2.0 | changes between 1.02 and 1.0+ | changes between 1.0+ and 1.0 |
|---|---|---|---|---|---|---|---|---|---|
| type | CLASS | mandatory | Reference to an existing data type | ||||||
| name | STRING | mandatory | Name of the element | ||||||
| bytepos | UINT | deprecated | Byte position of the data in the serialized representation of the containing struct. This is NOT relevant for the struct layout in memory (deserialized)! Started with '0' Elements following a dynamic array must have a byte pos of '-1' | From version 4.0 on this information is specified within the <serialized> tag. | |||||
| bitpos | UINT | deprecated | Bit position of the data in the serialized representation of the containing struct. This is NOT relevant for the struct layout in memory (deserialized)! default = 0 (in the range of 0 to 7) (relevant for serialization) | From version 4.0 on this information is specified within the <serialized> tag. | default value changes from 1 to 0 in DDL1.0+ | ||||
| numbits | UINT | deprecated | Specifies the amount of bits used in the serialized representation, if not set the size of the type will be used. e.g. tInt8 with numbits of 7 (numbits can only be used to non-arrays)This is NOT relevant for the struct layout in memory (deserialized)! | From version 4.0 on this information is specified within the <serialized> tag. | |||||
| byteorder | ENUM of LE/BE/Motorola/Intel | deprecated | Defines the byte order in the serialized representation. | From version 4.0 on this information is specified within the <serialized> tag. | |||||
| alignment | ENUM of 1/2/4/8/16/32/64 | deprecated | Defines the alignment of the element in the deserialized representation. | From version 4.0 on this information is specified within the <deserialized> tag. | |||||
| description | STRING | optional | Description of the created data type. | ||||||
| unit | CLASS | optional | Referenced unit of the element | Changed to optional | |||||
| comment | TEXT | optional | Additional comments (may be used as documentation comment in code generation) | ||||||
| arraysize | UINT or STRING | optional | Defines the array size of the element. The default is "1". Starting with DDL 2.0 the name of a preceding struct element can be used to specify a dynamic array.arraysize="elementName" For a detailed explanation of dynamic arrays refer to dynamic arrays. | Dynamic array support in DDL 2.0 | |||||
| valid_element_count | STRING | optional | Defines the name of a sibling struct element that identify the valid subset of the array content. The default is empty. For a detailed explanation of valid_element_count and its usage refer to virtual dynamic arrays. | introduced in DDL 4.2 | |||||
| value | CLASS | optional | Reference to an enum element as constant value for struct element | introduced in DDL 2.0 | |||||
| min | STRING | optional | Minimum value of the element | introduced in DDL 3.0 | |||||
| max | STRING | optional | Maximum value of the element | introduced in DDL 3.0 | |||||
| default | STRING | optional | Default value of the element | introduced in DDL 3.0 | |||||
| scale | STRING | optional | Scaling value of the element | introduced in DDL 3.0 | |||||
| offset | STRING | optional | Offset value of the element | introduced in DDL 3.0 |
The tag <serialized> uses the following attributes:
| Name | Type | Required | Description | changes between 3.0 and 4.0 |
|---|---|---|---|---|
| bytepos | UINT | mandatory | Byte position of the data in the serialized representation of the containing struct. This is NOT relevant for the struct layout in memory (deserialized)!Usually start with '0'. For dynamic elements following a dynamic array must have a byte pos of "-1". | introduced within the <deserialized> tag |
| bitpos | UINT | optional | Bit position of the data in the serialized representation of the containing struct. This is NOT relevant for the struct layout in memory (deserialized)! default = 0 (in the range of 0 to 7) (relevant for serialization). | introduced within the <deserialized> tag |
| numbits | UINT | optional | Specifies the amount of bits used in the serialized representation, if not set the size of the type will be used. e.g. tInt8 with numbits of 7 (numbits can only be used to non-arrays). This is NOT relevant for the struct layout in memory (deserialized)! | introduced within the <deserialized> tag |
| byteorder | ENUM of LE/BE/Motorola/Intel | mandatory | Defines the byte order in the serialized representation. | introduced within the <deserialized> tag |
The tag <deserialized> uses the following attributes:
| Name | Type | Required | Description |
|---|---|---|---|
| alignment | Enum of 0/1/2/4/8/16/32/64 | mandatory | Defines the alignment of the element in the deserialized representation. |
Remarks:
This section will explain the different meanings of alignment used in the DDL. Since alignment is only needed for deserialized representation, this section will only consider this kind of representation.
Inside a struct, every element has a explicit alignment value. This value influences, at what position the element will be placed inside the struct. The memory placement is initially determined by the order of the element tags inside the struct. After that the alignment takes effect. An element inside a struct will only be placed at memory positions that are multiples of the alignment value. The considered memory address is always relative to the beginning of the struct (memory position 0). As an example we assume having the following struct definition:
In this case, ui8Array will be placed at address 0. Because of the order of the element tags, ui32Value has to be placed after ui8Array. The next memory position after ui8Array is 5. But since the element ui32Value has an alignment of 4 and therfore the memory address has to be a multiple of 4, ui32Value will be placed at address 8 since this is the first address to matches all requirements. The attibute bytepos is only valid for serialized representation.
The alignment of a struct does not affect the alignment of its elements. A struct's alignment only determines the positioning of this struct inside of arrays. Assuming we have the following struct:
Lets now assume we have another struct tOuterStruct, that contains an array element and the element's type is the struct tInnerStruct:
Inside the array, the array element's positions are influenced by the alignment of the type (tInnerStruct). This means that the first array element is at position 0, in relation to the beginning of the array. The next free memory address would be 2, but since the alignment of the struct is 4, the memory address has to be a multiple of 4 and the second element of the array now is situated at memory address 4. The third will be at 8, the fourth at 12 and so on.
If for example you are planning to describe data with DDL that was originally defined as a struct in a header file, you should also define the alignment values inside your DDL file accordingly. If on the other hand you are shure, that the data you are using is always interpreted using DDL, there is no need to use an alignment other than 1.
The changes in DDL 3.0 regarding alignment are meant to make it more easy to describe structs defined in C(++). Since C(++) does not use alignment but packing for its structs, the alignment in DDL now also influences the size of a struct defined in DDL (see new definition of alignment in DDL 3.0). This means that arrays and structs may now become bigger in DDL 3.0 as they were in DDL 2.x. For example the struct
has the deserialized size of 1 byte in DDL 2.x and a deserialized size of 2 byte in DDL 3.0. The struct
has a size of 5 bytes in DDL 2.x and a size of 6 Bytes in DDL 3.0. This is due to the fact, that in 2.x the array looks like this:
| Bytepos | Element |
|---|---|
| 0 | aValue[0] |
| 1 | padding |
| 2 | aValue[1] |
| 3 | padding |
| 4 | aValue[2] |
resulting in a size of 5 bytes. The padding elements are inserted, so that the following Element is correctly aligned. In DDL 3.0 the tFirstStruct has already a padding alement attached to its end so that its size matches the requirements of the alignment attribute (multiple of 2). Therefore, the array in 3.0 will look like this:
| Bytepos | Element |
|---|---|
| 0 to 1 | aValue[0] |
| 2 to 3 | aValue[1] |
| 4 to 5 | aValue[2] |
resulting in a size of 6 bytes.
Assuming you have a couple of structs defined in DDL 2.x, if you now plan to change the language version from 2.x to 3.0, that you are creating new structs, that may not be compatible to its namesakes defined in DDL 2.x. Its better to use new names for the structs to differentiate between the different versions.
Detailed explanation of the dynamic array functionality.
Dynamic arrays were introduced in DDL 2.0.
Attention:
The use of dynamic arrays will cause a severe performance drop compared to static arrays. It is therefore recommended to use static arrays whenever possible. Consider virtual dynamic arrays for more performance.
To minimize the performance impact of dynamic arrays the user should adhere to the following guidelines:
alignment="1" for all elements.Dynamic Arrays are always a performance issue for the codecs. Address information of the elements are evaluated for the dynamic part each time a decoder/codec is initialized.
The valid_element_count attribute was introduced in DDL 4.2 to support virtual dynamic arrays. This enables you to identify a sibling struct element to mark a subset of the array as valid.
Example:
In this example a static array member of 256 elements f64VirtualDynamicArray is used within tArrayStruct.
If the value of the member ui64ValidCount is i.e set to 200:
f64VirtualDynamicArray[0] and f64VirtualDynamicArray[199] are validf64VirtualDynamicArray[200] and f64VirtualDynamicArray[255] are not validat this iteration and may not be considered by any implementation.
So, the set arraysize="256" is the maximum virtual array size of your array and enables an implementation of pre allocation of the whole static array byte size while initializing.
The byte size of the whole struct will always be a static byte size!
Stream Meta Types are defined within the tags <streammetatypes> and </streammetatypes>. Example:
The tag <streammetatype> uses the following attributes:
| Name | Type | Required | Description |
|---|---|---|---|
| name | STRING | mandatory | The identifier of the stream meta type |
| version | STRING | optional | Version information of the specified stream meta type i.e. "ant", "bat", ... |
| parent | TEXT | optional | Identifier of a parent stream meta type |
The tag <property> uses the following attributes:
| Name | Type | Required | Description |
|---|---|---|---|
| name | STRING | mandatory | The name of the property |
| type | STRING | mandatory | The type of the property |
Streams are defined within the tags <streams> and </streams>. They describe named instances of streams with concrete data content and a type of stream. Usually this stream description is only necessary to describe the content of a streaming file.
The tag <stream> uses the following attributes:
| Name | Type | Required | Description |
|---|---|---|---|
| name | STRING | mandatory | The name of the stream |
| type | CLASS | mandatory | Referenced type of the stream (Is a reference to a Stream Meta Type or a Struct.) |
| description | STRING | optional | The description of the stream |
The optional tag <struct> uses the following attributes: This tag should only be used, if
| Name | Type | Required | Description |
|---|---|---|---|
| bytepos | UINT | mandatory | The data described |
| type | CLASS | mandatory | The structured type used for the data area of the stream (Is a reference to a a Struct Type.) |
| name | STRING | optional | A name for the struct |