UDP v2/3 Protocol

The UDPv2/3 protocol is a secure way to communicate between ubsub and low-level devices, such as arduino or particle.

The official C++ implementation can be found on github.

Server

The default udp server exists at: iot.ubsub.io:4001

Features

  • Bi-directional communication (Events and listeners)
  • NAT traversal via holepunching
  • Encrypted payload (salsa20) with HMACSha256 signature for secure communication
  • Automatic (optional) payload wrapping / extraction
  • Automatic renewals/re-subscribe and retrying
  • ACK support for guaranteed delivery (with timeouts)

Use Cases

  • Subscribing to events by a name
  • If topic doesn't exist, create it by event name
  • Publish events to a given "topic"
  • If topic to publish events to doesn't exist, drop event
  • Unsubscribe from a topic

The above should provide functionality for:

  • Creating a "function" (aka topic) in which others can send to
  • Creates a unique named topic and auto-subscribes to it
  • Invoking a "function"

Protocol

Payload Specification (Version 2/3)

  • Little endian encoding
  • HMAC Sha 256 MAC. Uses user/device secret
  • Salsa20 with random nonce (used as the IV)

Shape

|                              |            ENCRYPTED                |
|          CryptHeader         |        Header         |    Body     |          Signature             |
|------------------------------|-----------------------|-------------|--------------------------------|
| Version |   Nonce   | devId  | TS | CMD | LEN | Flag |    Payload  |       HMAC Sha-256 Digest      |
|---------|-----------|--------|----|-----|-----|------|-------------|--------------------------------|
| 1B: 0x2 | 8B Random |  16B   | 8B | 2B  | 2B  |  1B  |    <len>    |           32 Bytes             |

Versions:

Id Description
0x2 Unencrypted version of the payload
0x3 Encrypted version of the payload

Field Details:

Field Description Encrypted
Version The version of the protocol to use
Nonce Crypto-secure random 8 bytes of data. Used at IV and non-repeat
devId The id of the device (currently the user id)
ts The timestamp the message was sent at, unix epoch UTC. *
cmd The id of the command that will exist in the body (see commands below) *
len The length of the body *
flag Any flags associated with the command (see below) *
Body The body data of the command *
Signature HmacSha256(entire message, userSecret)

Commands

NOTE: Local port should be randomly chosen by the device within the upper-range to reduce risk of conflict with other devices on the same network. (IANA: 49152 to 65535)

Note about cstrings:

It is expected in any case that a cstr is present that it is null-terminated. If the string fills up the entire length of the field (eg a 32 character string into a 32 byte field), it is acceptable to not have a null terminator as it will be assumed that the string is simply 32 characters in length. Anything longer than the field length will be truncated.

Subscribe (0x1)

Client -> server

Ask a device to be sent a message whenever a given topic is invoked. A message will be sent via the subscription message command.

Layout:

Field Type Bytes Description
Port u16 2 Port number to reply at (local port)
TopicNameOrId cstr 32 Topic name or id to subscribe to
FunctionID u64 8 Unique id the device will use to identify the function
TTL u16 2 Number of seconds the subscription will exist before needing renewal (max: 5 minutes)

Flags:

Id Name Description
0x1 ACK Acknowledge the subscription call
0x2 UNWRAP If server should unwrap the payload before sending it to the device
0x4 MSG_NEED_ACK If the server should be expecting an ACK from the client when sending it a message
0x8 DO_NOT_CREATE Do not create the topic if it is missing

Subscription Ack (0x2)

Server -> Client

If requested, the server will respond with this acknowledge to a subscription.

Layout:

Field Type Bytes Description
Nonce u64 8 The nonce of the original subscribe request
FuncId u64 8 The FunctionID sent in the original subscribe request
TopicId cstr 16 The topicId that was subscribed to
SubscriptionId cstr 16 The subscriptionId that represents the subscription
SubscriptionKey cstr 32 The key associated with the subscription (can be used to validate it)
Expires u64 8 The unix epoch time that the subscription will expire at

Flags:

Id Name Description
0x1 DUPE The requested subscription is actually a duplicate request. Will send back details anyway
0x2 TOPIC_NOT_EXIST The topic to subscribe to does not exist, and won't be created

Unsubscribe (0x3)

Client -> Server

Request to remove device from a subscription

Layout:

Field Type Bytes Description
Port u16 2 The local port to respond to the request on
TopicId cstr 32 The topic id to unsubscribe from
SubscriptionId cstr 16 The subscription to remove

Flags:

Id Name Description
0x1 ACK Request an acknowledge message from the unsubscribe

Unsubscribe Ack (0x4)

Server -> Client

Acknowledgment of the unsubscribe call.

Layout:

Field Type Bytes Description
Nonce u64 8 The nonce of the original request

Flags:

Id Name Description
0x1 No such sub Acknowledging the request, but there was no subscription to be removed

Subscription Message (0x5)

Server -> Client

The subscription has been invoked, and a message is being delivered to the client.

If the unwrap flag was passed on the original subscription, the payload key will be extracted and its value passed as the argument. If either or of the above are not present, the raw JSON will be passed.

Layout:

Field Type Bytes Description
FuncId u64 8 The FunctionID of the origin
SubscriptionKey cstr 32 The key of the subscription
Message raw remaining The payload. The length is the all of the remaining bytes of this payload

Flags:

Id Name Description
0x1 ACK An ack is requested for this message. Only set if originally asked for. If not responded, message will be retried
0x2 UNWRAPPED This message was unwrapped

Subscription Message Ack (0x6)

Client -> Server

Acknowledging the subscription message sent to the client.

Layout:

Field Type Bytes Description
Nonce u64 8 The nonce of the message sent to the client

Flags:

Id Name Description
0x2 REJECTED The message sent to the client was rejected

Message (0xA)

Client -> Server

A message sent from the client to a topic on the server.

Layout:

Field Type Bytes Description
Port u16 2 The local port used to respond with the ACK
TopicNameOrId cstr 32 The topic name or id to publish the message to
TopicKey cstr 32 The key of the topic to publish the message to (if needed)
Message cstr remaining The message to send. If valid JSON, will be used as such, otherwise, will be wrapped like: {"payload": <msg>}

Flags:

Id Name Description
0x1 ACK An acknowledgment is requested for this message
0x2 EXTERNAL Send this message only by its topicId, not specifically to the user. Will mean you must send by id not by name
0x4 CREATE_TOPIC Create the topic if it does not exist

Message Ack (0xB)

Server -> Client

If requested, the server will send an acknowledgment to the client on the requested port.

Layout:

Field Type Bytes Description
Nonce u64 8 The nonce of the original message request

Flags:

Id Name Description
0x1 DUPE The message is a duplicate (has already been received)

Ping (0x10)

Client -> Server

Ping the server. Will be responded on the requested port with a pong. Used to keep hole-punched tunnel open, or to check connection.

Layout:

Field Type Bytes Description
Port u16 2 The local port for the pong to respond to

Flags:

None

Pong (0x11)

Server -> Client

The response to the PING command (above).

Layout:

Field Type Bytes Description
BounceTS u64 8 The timestamp of the PING command

Flags:

None

Sequence of Events

Establishing a Subscription:

  1. Device sends 'SUB' to the UDP Router with unique funcId
    1. UDP Router will ACK to the sub with the nonce, and subscription id/key
      • UDP router will use the address:funcId to de-duplicate
    2. Upon ACK received device will assume all is good, and become passive
    3. Repeats until device receives ACK, or the device "gives up"
  2. Once established, will need to re-SUB after SUB TTL is up (at expires time, which could be less than requested)
    1. Resub will update subscription as appropriate

Sending a message:

  1. Send a MSG(0x3) to the UDP router with the target event id
  2. UDPRouter will extract the message, and forward to the topic
    1. Router with MSGACK with the nonce of the message (if requested)
    2. If nonce has already been processed, will MSGACK with DUPE flag
    3. Client has right to resend after a retryTime (if any)

Periodically:

  1. Ping the UDP router
    1. If no pong received in threshold OR pong received TS is bad, assume bad connection and re-establish all subscriptions
    2. Consider only using ping/pong if subscriptions exist, otherwise it might be unnecessary

Versioning

All protocol versions will maintain backwards compatibility. New fields may be added to a command, but only postfixed, and the meaning will never be changed.

Requesting a change / fix

If you'd like to request a fix, please create an issue here.