Node basically has two kinds of streams, a readable stream and a writable stream. An example of a stream is a TCP socket that we can read from and write to, or a file that we can append or read from sequentially.
Example 1: Let us create a HTML5 video streaming server with Node and Express
Why do we need streaming? If we won’t then the entire video file will be downloaded to the client and loaded into memory which may or may not be good.
Imagine the video size to be 100’s of MB’s, in that case the network bandwidth etc will break the user experience.
Also, take this scenario. You have 8GB RAM on the server, but you have a 20GB video file on the drive and you would like to load that file?
Welcome to streaming!
With streams, the file stays on the disk and only chunks from the whole thing is transferred to the client. For example, just 1MB at a time.
The Pipe
The Unix | (pipe) is mostly common way to pass output of one program to another one.
In Node we can use Pipes inside the code to pass the result of one function to the next one.
For example
somefile.pipe(somewritableStream).pipe(toSomeOtherWritableStream)
Combining Pipes and Streams
We will combine the pipe and stream to build our video streaming server.
Refer streams\video-streaming folder for the complete source code.
Let us first setup ExpressJs by running the below command
npm install --save express
Let us now create and index.html file that will host the <video> tag.
<!DOCTYPE html>|<html><head><meta charset="utf-8"><title>Stream Video</title><meta name="viewport"content="width=device-width,initial-scale=1"></head><body style="text-align:center"><div style="margin:0 auto;"><video src="" controlswidth="640"height="480"></video></div></body></html>
Thats it for the HTML file. Note we are using the <video> element, the src is set to empty,as we will pass it from the URL.
All the videos will be stored in a subfolder videos within our project as shown below
The entire commented source code is in the index.js
Lets understand the important part of the code and first lets setup the route to server our index.html file.
Let us start with the /video/:videoname route. The following is and example of how to request the streaming media.
http://localhost:3000/videos/datatable-reorder-columns-drag-drop.mp4
The above request will fetch the media from the videos folder and stream it.
First we grab the file name from the parameter and then use fs.state to check the status of the file. If there are any errors, we simple end the response with the message.
Next, let us grab the range the browser will be requesting and also the filesize from the stats variable that is available as a result of call to fs.stat.
Once we grab the range and the size, lets compute the start and the end of the buffer. The end will remain the same and the start will vary with each buffered request. We have to compute the chunksize.
Note in the above screenshot, the range is in the format bytes=0-100, where 0 is the start and 100 is the end just for example.
If the end value is not available, then we take the size of the file as the end value.
Once we get the start and the end we compute the chunkSize. This is the size of response that the Node server will send back every time when data is requested from the client.
Now let us set the response header with the required information to support sending chunked data as shown below and also set the status code to 206 meaning the status is set to partial content.
Let us understand this in more detail. In order to support streaming HTML5 video on server side, we need to handle the Range HTTP header from the browser.
The Range HTTP header specifies what byte rnge from the video the browser is requesting.
If the range is missing, we can send the whole video starting from byte 0. Otherwise we ill only send the range of bytes requested indicated by start and end values.
As discussed the range header format variation is given below.
Range: bytes=0-
The above means the browser is requesting the whole video or atleast this is the initial request so the size of the video can be determined by the browser from the Content-Length header in the response.
Range: bytes=2000–5000
The above values mean the browser is requesting the video starting from 2000 bytes to 10000 bytes (may be the user has skipped ahead in the video).
Also, note the response headers that the Node server is sending.
Accept-Ranges: bytes
Content-Type: video/html
Content-Length: (length)
Content-Range: bytes (start)-(end)/(total)
The Accept-Ranges tells the browser that the server supports HTML5 video streaming and can take byte ranges.
Content-Length sends the total length of the file in bytes.
Content-Range sends the range of content being returned in bytes.
The next step is to create a read stream as shown below and watch for the open event.
We open a readable stream using fs.createReadStream and pass in the optional start and end bytes ranges.
Once the stream is open we pipe the stream data to the response so that the data is available to the client/browser.
Then we handle error if any and close the fs.state function call.
And finally, lets start our server on a specific port as show below.
Let us know run our app using node index.js at the terminal and open up your browser and fire a request for a video file within the videos folder (Refer the source code).
The output of streaming video is shown below for your reference.
Example 2: Compressing files using gzip (stream)
Lets roll out a quick program to compress files using gzip. (I am not doing any error handling for brevity sake)
Lets import the fs and zlib modules (they are built in to node)
Example 2: Compressing files using gzip (stream)
Lets roll out a quick program to compress files using gzip. (I am not doing any error handling for brevity sake)
Lets import the fs and zlib modules (they are built in to node)
Let’s grab the filename from the command line argument.
Let us now use the fs.createReadStream method to read the file, then pipe the output to zlib and then create a new .gz compressed file.
If you run the below code in your terminal
You get the new compressed file as the output as shown below. I ran the the code on the file itself, but you are free to pass in any file as the second argument.
NOTE: You can use winrar or 7zip or any other standard tool to unzip or as an exercise roll your own unzip command.
