Media Server

Networking overview

Unreal Media Server supports UMS protocol for streaming to Unreal Streaming Media player on Windows OS; WebRTC protocol for streaming to HTML5 <video> in web browsers; WebSocket-video/mp4 protocol for streaming to HTML5 <video> Media Source Extensions in web browsers; RTMP and RTMPT protocols for streaming to Flash player on any Flash-enabled OS; MS Smooth and MS-WMSP protocols for streaming to Silverlight and Windows Media player; Apple HTTP Live streaming (HLS) for iOS and other HLS-enabled players and devices; MPEG2-TS protocol for streaming to Set-Top boxes.

Unreal Media Server architecture
When the server streams via HLS, MS Smooth and MPEG2-TS protocols, players do not connect to the server at all: they receive fragmented streams directly from web servers (HLS and MS Smooth streaming), or from UDP ports on their IP addresses (MPEG2-TS). In these cases the server doesn't know anything about players or number of concurrent viewers.
In contrary, when streaming via WebRTC, WebSocket-video/mp4, RTMP, MS-WMSP and UMS protocols, players initiate a direct connection to Media Server; the server can perform authentication and can keep track of concurrent players. All players connect to media server's TCP port 5119 (unsecure connections) or 443 (TLS-secured connections); these ports can be configured. The above ports also serve for WebRTC signaling, while the actual WebRTC media is streamed over port 5135.
The server supports wide range of live software and hardware encoders, accepting live streams in both push and pull modes (the only difference is which side initiates a connection). Pull-mode encoders include: RTSP software encoders and IP cameras; MS-WMSP encoders such as WME; Unreal Live Server; RTMP servers; HLS servers. Push-mode encoders include: WebRTC-enabled web browsers; RTMP publishing encoders such as FMLE, ffmpeg and mobile apps; MPEG2-TS encoding devices; Unreal Live Server. Unreal Live Server streams to Unreal Media Server via UMS protocol over TCP and RTP(UDP) transports.

Media processing engine: the universal transmuxer

Live content that arrives to media server via different streaming protocols, is found to be packaged in different container formats native to these streaming protocols. For example, RTMP protocol sends media encapsulated in FLV container format; MS-WMSP protocol wraps media in ASF container. Similarly, when Unreal Media Server streams recorded content, it deals with media files packaged in different container formats (MP4, FLV, ASF etc.). Regardless of whether media comes from disk (files) or from network (live), the Media Server instantly demultiplexes it into audio-video elementary streams, i.e. extracts these streams from the container. Streams remain in their original encoded form; the server does not perform any transcoding or decoding. When various players connect and ask for media, the server re-packages these streams to the container format needed by these players and sends these streams via streaming protocols supported by these players. For example, the same H.264 stream, coming from live encoder via RTSP or WebRTC protocol and encapsulated in RTP packaging, can be served at the same time to HTML5 <video> WebRTC players in web browsers via WebRTC protocol (RTP container), to HTML5 <video> MSE players in web browsers via WebSocket protocol (video/mp4 ISO BMFF container), to iPADs/iPhones via HLS protocol (MPEG2-TS container); to Flash players via RTMP protocol (FLV container); and similarly to Silverlight, Unreal Media Player, STBs, via appropriate protocol/container formats.

Supported standards and specifications

WebRTC W3C WebRTC specification
WebSockets IETF RFC6455
Media Source Extensions W3C MSE specification
RTMP/RTMPT Flash protocol
H264, MPEG4 Video over RTP IETF RFC3984, IETF RFC3640
AAC, MP3 Audio over RTP ISO/IEC 14496-3, IETF RFC2250
MS-WMSP Windows Media Streaming Protocol
MPEG2-TS ISO/IEC 13818-1
Apple HLS HLS format
MS Smooth streaming Smooth streaming protocol

Streaming via UMS protocol

Unreal Streaming Media Player and its browser plugin can play UMS streams in unicast mode over TCP, RTP(UDP) and HTTP(S) transports, and in multicast mode over RTP(UDP) transport. If TCP/UDP connection attempt is unsuccessful, the player automatically switches to HTTP transport.
UMS streams can also be played on Android and iOS devices with mPlayer mobile app. This mobile player supports UMS-TCP unicast mode only, and H.264/AVC1, AAC/MP3 encoded streams only.

Multicasting via UMS protocol

Unreal Media Server supports simultaneous multicasting of unlimited number of live sources and virtual folders. When a first viewer connects and requests Multicast delivery of live source, the server will start multicasting. When a last viewer disconnects, multicasting is stopped. When multicasting files, administrator of Media Server must manually start and stop multicasting of virtual folder, using server configuration program.
When requesting multicast delivery, a player first connects to the server on UMS-TCP. The server authorizes this player and sends him multicast group information for this particular broadcast. The player joins this multicast group and receives media packets via UMS-RTP multicast.
Unfortunately, multicasting will not work via the Internet in most cases, since public ISPs don't allow their routers multicast delivery.

Streaming via WebRTC protocol to HTML5 <video> in browsers

Unreal Media Server supports live low latency streaming to HTML5 <video> element via WebRTC protocol, using WebRTC-UDP and WebRTC-TCP transports. Supported browsers: Chrome and Firefox on Windows; Safari on iOS and Mac; Chrome on Android. The signaling part is handled via WebSocket (plain or secure) via main ports (5119/443 by default). To use secure WebSocket, the server must have a valid SSL certificate installed. The actual WebRTC ICE connection can be established via single predefined port in Unreal Media Server, dedicated to WebRTC connections (by default 5135), or via random port. The player has a choice whether to use a single or random port. If single port is used (recommended), service provider only needs to open port 5135 in firewall/NAT router (together with signaling ports, 5119 or 443). If random port is used, all ports will need to be open in the firewall, or Unreal Media Server should be added as an allowed app in the firewall. Unreal Media Server's WebRTC implementation is NOT a peer-to-peer solution. In contrary, it's a server-based solution, providing full interoperability between WebRTC and other streaming systems. Stun and Turn servers are not used in Unreal Media Server's WebRTC implementation. Web browser always creates a direct connection to Unreal Media Server; ICE establishment is immediate. Server-based solution enables multi-user video conferencing; low latency playback of hardware-originated live streams (IP cameras and encoders) in HTML5 <video> in web browsers; playback and ingest of WebRTC-originated live streams in conventional players and STBs. Unreal Media Server supports WebRTC streams encoded with H264, VP8, VP9 video codecs and with Opus, G.711 (PCMA and PCMU) audio codecs.

Streaming via WebSocket-video/mp4 protocol to HTML5 <video> Media Source Extensions in browsers

Live low latency streaming via plain and secure WebSockets is achieved by sending ISO BMFF video/mp4 segments for decoding and playback by Media Source Extensions of HTML5 <video> element. Unreal Media Server performs on-the-fly segmentation of a live stream for each connected WebSocket. H264 and AAC encoding is required. All major browsers on Windows, MAC and Android support HTML5 MSE playback.

Streaming via WebSocket-video/mp4 and UMS protocols over HTTPS transport

Unreal Media Server supports TLS1.2 encryption for UMS-HTTPS and WSS (secure WebSockets). The server must have a valid SSL certificate installed. HTTPS transport provides a complete stream protection on its way from the server to the player.

Streaming via MS-WMSP (MMSH) protocol

Unreal Media Server supports Unicast streaming via MS-WMSP (MMSH) protocol. Live and recorded audio/video is being encapsulated into ASF container on the fly and sent as HTTP payload to any player on any OS that is capable of playing MMSH streams. WMV(VC1), WMA or MP3 encoding is required for MMS streaming. Various players on Windows, MAC, Linux, Mobile devices are supported, such as Silverlight, Windows Media Player, QuickTime Player, VLC, MPlayer.

Streaming via RTMP/RTMPT protocols

Unreal Media Server supports Unicast streaming of live sources and media files via RTMP/RTMPT protocols to Flash player. Audio/video content is being encapsulated into FLV container on the fly and sent to Flash player on any supported OS. H264, AAC or MP3 encoding is required for Flash streaming. MKV, MP4, MP3, FLV files with H264, AAC or MP3 content are supported.

Streaming via Apple HLS protocol

Unreal Media Server supports HLS streaming of live sources (and files, using a "live playlist" type of live broadcast). The server acts as a segmenter, recording HLS chunks into specified web server folder. In addition to Unreal Media Server, you need to run a web server (any web server is OK) to serve these chunk and m3u8 playlist files to HLS players. AES-128 HLS encryption is supported. H264, AAC or MP3 encoding is required for HLS streaming.

Adaptive bitrate streaming

When live encoder uploads multi-bitrate streams to Unreal Media Server, these streams can be served to players via WebSocket-video/mp4, HLS and RTMP protocols. For HLS, the m3u8 file that is generated automatically by Unreal Media Server, will contain links to individual bitrate m3u8 files. HLS players will switch between these bitrates based on available bandwidth. For RTMP, the server supports "Play2" RTMP command. Once a player sends "Play2" command, the server will switch to a different bitrate stream. For WebSocket-video/mp4, Unreal HTML5 MSE player will send a "switch" command to the server based on network conditions and delivery performance.

Streaming via MS Smooth protocol

In addition to Unreal Media Server, you need to run IIS7 web server with Media Services extension installed. Unreal Media Server streams to Live Smooth streaming publishing points (created by IIS manager tool) via Smooth streaming protocol: stream manifests and mp4-encoded chunks are being sent in real time via HTTP Post requests. Smooth streaming players such as Silverlight player connect directly to publishing points URLs. H.264/AAC and VC1/WMA codecs are supported by MS Smooth streaming. MS Smooth streaming is a high latency protocol; the best latency you can get is 10 seconds.

Streaming via MPEG2-TS protocol

Unreal Media Server supports Unicast and Multicast streaming of live sources and media files via MPEG2-TS protocol. The Media Server broadcasts MPEG2-TS encapsulated streams over UDP or RTP transports to any specified IP addresses. MPEG2-TS players do not connect to Media Server at all; they rather connect to whatever IP address the Media Server broadcasts to.

User Authentication, access restrictions and DRM

Configuration programs of Unreal Media Server and Unreal Live Server make it possible to fully control access to media resources, allowing it only to trusted parties. Administrator can make Media Server grant access to authorized users only, and Live Server grant access to specific Media Servers only. There are 2 types of user authentication supported by Unreal Media Server: Internal Authentication and Session-based authentication. Every resource configured with Media Server Configurator (all virtual folders and live broadcasts), can be set to allow anonymous access or require one of these 2 types of authentication. Internal authentication can be used with UMS protocol only, whereas Session-based authentication can be used with UMS, MS-WMSP, WebRTC, WebSocket-video/mp4 and RTMP protocols.

Full DRM can be achieved when playing via UMS protocol, by using one these 2 types of user authentication, and also encrypting the stream by using HTTPS transport (refer to "Streaming via UMS protocol over HTTPS transport" section above).

1. Internal Authentication
Authentication and user management is handled directly by Unreal Media Server; this is our proprietary authentication mechanism. Users play with Unreal Streaming Media Player or its browser plugin. Media Server administrator needs to create users on the server side. Users can be administrated using either Media Server Configurator or Web page (Refer to useradmin page in our SDK). A user can belong to one of 3 user groups: Basic, Medium and Advanced. Resource that is set to use Internal authentication, can restrict access to a specific user group. When a user tries to access such a resource, the Streaming Media Player prompts the user to enter User name and Password, and optionally persists these credentials on the user's computer. User's credentials are bound to Media Server IP address, such that if the user accesses some resource on a different Media Server, he/she will be prompted to enter credentials again.
Security: User's credentials are encrypted with strong one-way hashing algorithm, before they are sent on the network. These encrypted credentials are encrypted once again with industry-strength symmetric algorithm when they are stored on user's computer (if the user chooses to remember them locally) and on Media Server's computer, when administrator creates users.

2. Session-based Authentication
It is to be used when a web application handles authorization by standard web methods and provides authorized users with access to media resources. When a web app authorizes a user, it needs to notify Unreal Media Server about this user, passing an authentication token such as Session ID or any other unique ID associated with the user's session. When the user logs out or the session is expired, Unreal Media Server has to be called to remove this session from the list of active sessions. Also, when the web app returns HTML page to the client, the same authentication token needs to be appended to UMS://, MMS:// or RTMP:// links, or "UseSessionID" method needs to be called in the client-side scripting for embedded Unreal Streaming Media Player's browser plugin. This way, when the player sends a request to Unreal Media Server, the same authentication token will be passed along, and Unreal Media Server will be able to recognize the user. Refer to our sample web application for reference on session-based authentication.

   2003-2019 Unreal Streaming Technologies. All rights reserved.