2. The Teleport Protocol

The Teleport Protocol is a network protocol for client-server immersive applications. Essentially this means that applications can be used remotely without downloading or installing them. The protocol is open, meaning that anyone can use it, either by writing a client program or by developing a server that uses the protocol.

Teleport is a top-level application-layer protocol. It uses three transport layers in parallel: a WebSocket connection for signaling, a media/data transport (WebRTC by default, with optional SRT/EFP support) for real-time streams, and an HTTP(S) service for static asset retrieval.

Table 2.1 Network Layers

Teleport XR	Real-time client-server interface of a virtual world.
WebSocket / WebRTC / HTTP(S)	Signaling, real-time data channels, and static asset transfer.
TCP / UDP / TLS / DTLS-SRTP / SCTP	Standard transports underlying the above.
Network, Data Link, Physical Layer	Standard network layers

Teleport XR Ltd provides reference implementations of both client and server. The protocol and software should be considered pre-alpha, suitable for testing, evaluation and experimentation.

Here is the code:

Table 2.2 Reference Code

URL	Contents
https://github.com/teleportxr/Teleport/	Native C++ Client app for Windows, Linux and Quest, Native C++ server library for Unreal and Unity-based servers.
https://github.com/teleportxr/teleport-nodejs	Lightweight nodejs server library. Use npm install teleportxr.
https://github.com/teleportxr/teleport-nodejs-server-example	Lightweight nodejs server example: uses the teleportxr npm package.
https://github.com/teleportxr/teleport-web-client	Web client suitable for embedding (pre-alpha)

The protocol uses three concurrent connections between a client and server:

The Signaling connection is established first; the Server then sends a Setup Command which contains the parameters needed by the Client to attach to the Data Service and the HTTP service.

The data source may be the same server, or it may be one or more other sources.

Contents:

• 2.1. Conventions
  • 2.1.1. Endianness, alignment and packing
  • 2.1.2. Identifiers
  • 2.1.3. Coordinate systems and units
  • 2.1.4. Time base
  • 2.1.5. Versioning
• 2.2. Connection State Machine
  • 2.2.1. Phases
  • 2.2.2. Reconnection
  • 2.2.3. Connection state enumeration
  • 2.2.4. Timeouts
• 2.3. Signaling
  • 2.3.1. Connection
  • 2.3.2. Signal types
  • 2.3.3. After connection
• 2.4. Data Service
  • 2.4.1. Initialization
  • 2.4.2. Command packets (server-to-client)
  • 2.4.3. Message packets (client-to-server)
  • 2.4.4. Setup Command
  • 2.4.5. Handshake
  • 2.4.6. Update
• 2.5. Data Transfer
  • 2.5.1. Reliability requirements
  • 2.5.2. WebRTC transport (default)
  • 2.5.3. Legacy transport: SRT + EFP
• 2.6. HTTP Service
  • 2.6.1. Endpoint
  • 2.6.2. URL format
  • 2.6.3. Client behaviour
  • 2.6.4. Lifecycle
  • 2.6.5. Sequence
  • 2.6.6. Operational notes for implementers
• 2.7. Video
  • 2.7.1. Video Texture Layout
  • 2.7.2. Video Frame Structure
  • 2.7.3. Recovering from Corruption
  • 2.7.4. Minimizing Latency
  • 2.7.5. Processing of video Frame on the Client
• 2.8. Video Metadata
  • 2.8.1. Structure of Metadata
  • 2.8.2. Sending Metadata to the Client
  • 2.8.3. Processing Metadata on the Client
• 2.9. Geometry Payload
  • 2.9.1. Channel framing
  • 2.9.2. Payload types
  • 2.9.3. Node payload
  • 2.9.4. Mesh payload
  • 2.9.5. Material payload
  • 2.9.6. MaterialInstance payload
  • 2.9.7. Texture payload
  • 2.9.8. Animation payload
  • 2.9.9. Skeleton payload
  • 2.9.10. FontAtlas payload
  • 2.9.11. TextCanvas payload
  • 2.9.12. TexturePointer payload
  • 2.9.13. MeshPointer payload
  • 2.9.14. MaterialPointer payload
  • 2.9.15. RemoveNodes payload
  • 2.9.16. Resource lifecycle
  • 2.9.17. Axis conversion
  • 2.9.18. Coding conventions
• 2.10. Audio
  • 2.10.1. Codec and RTP parameters
  • 2.10.2. Topology
  • 2.10.3. AudioConfig (inside SetupCommand)
  • 2.10.4. Selection policy and caps
  • 2.10.5. AudioSourceMapping command
  • 2.10.6. AudioParticipantStateChange command
  • 2.10.7. Join and leave
  • 2.10.8. Example
  • 2.10.9. Lifecycle
• 2.11. Input
  • 2.11.1. Negotiation: SetupInputsCommand
  • 2.11.2. InputType bitfield
  • 2.11.3. Reporting: InputStatesMessage
  • 2.11.4. Reporting: InputEventsMessage
  • 2.11.5. Pose stream
  • 2.11.6. Sequence
  • 2.11.7. Operational notes for implementers