The Browser Is Where It’s At

I used to create a lot of Flash based content – be it video players, widgets, or full on applications what I built was built using the Flash Platform. I haven’t opened the Flash IDE or Flash Builder to create a SWF in a long, long…long time. What does this mean? First, technology is changing – no doubt. Second, I get to learn some new stuff. Guess what? I’m okay with that.

Users, platforms, and developers have have forced browsers to evolve. The browser is no longer just a window to view content. It is an environment that applications execute in. It used to be that you’d open a browser, search for something, or read something, then close it down and get to work. Now, what you work on is in in the browser & those browsers are open all day.

What should I learn? What should you learn, if you’re not already? Learn about the stuff that happens in the browser – Yep,  JavaScript, CSS, & HTML. Learn the existing APIs as well as the upcoming API changes and additions.

Banner of Browser Logos

The browser as a first-class citizen?

Browsers are more powerful, they are more feature rich, and are becoming first class citizens when it comes to how people use them. I don’t think that it is 100% where it needs to be, but it won’t be long until that tipping point that causes a shift in how people use and think about the browser. Consider Google services – to name a few, GMail, Drive & Docs, & Calendar. These services all run in the browser and each week new features are added to make them more comparable and sometimes better than their desktop counterparts. It used to be that everyone relied on Microsoft Office and Word to create documents, edit and track changes, Outlook to manage their email and calendar. Now, all of that and more is in the browser, of course you can still use desktop applications to manage that data, but, like I said there will be a tipping point. The point of this? Pay attention to the browser, the browser is where things are headed.

Why the browser?

Because:

  • Browsers are familiar to users
  • They exist for all major platforms
  • Browsers have established a quick and easy update path
  • Browsers will become more accepted by the enterprise
  • They take advantage of HTTP protocols
  • Browsers leverage new and existing technology

Familiarity

A browsers is an easy path to entry. A browser is a simple concept to grasp and easy to explain and learn. Although it has a low learning curve, browsers have and can be extended on to provide functionality needed for today and tomorrow’s users.

All major platforms

All major platforms have a browser. Desktop & mobile, even TVs and DVD players have browsers. For developers the headache is support different platforms. You will have to provide platform specific code. But, the main point to get across here is that HTTP, JavaScript, & CSS are will be supported by more and more platforms.

Quick and easy update path

Chrome and Firefox update at lightning speed, and for many users without them even knowing. This helps roll out new features (Web RTC, Media APIs etc) more quickly. There is a major barrier when it comes to the enterprise and government, but this is something that I think will change in the near future.

Accepted by the enterprise & government

Currently these are two areas where updates and browser versions can really hold back innovation. But, with the current direction of and additions to APIs and security, this issue should become a problem of the past as browser updates are easier, more secure and become the norm rather than the exception for the enterprise and government.

HTTP Protocol

HTTP has been around forever and for good reason. It works. It is flexible and powerful. Innovation and increased bandwidth allow for more innovative and more interesting uses of the protocol. HTTP video streaming is a great example of this. The client is responsible for managing the HTTP requests that it will need to successfully play back video served up in HTTP chunks, while still providing expected functionality to the user. We still have conversations about “chatty” applications, but these conversations will be minimised as a different perspective and different technologies emerge that leverage HTTP to a greater and more efficient degree.

Leveraging Existing & New Technology

As with HTTP, other established technologies will be accepted and leveraged by the browser. For instance, browsers are finally getting around to integrating media playback. WebRTC is another example things like WebSockets, node.js, socket.io, there are some really interesting things going on excite about the next-gen applications and tools that will be created.

What I see

All of this isn’t to say what we as developers are doing now will go away. Things certainly won’t change immediately. But, I am looking to the future, evaluating trends and technology, and emerging conversations, and what I see is the browser. Maybe not in it’s current incarnation, but the browser is what I see.

What do you see?

HDS & Bootstrap Data

inspect-hds-bootstrap

Working with HDS Bootstrap Data

I’ve always been curious about the bootstrap data for HDS content. Recently, I had the chance to find out more about it and get in some fun development with Node.js. We’ve been kicking around the idea of building a tool set for Adobe Media Server using Node.js and possibly socket.io. Last weekend we got some of the ideas going and one of those was parsing the hds bootstrap data created when content is packages for HDS delivery.

The bootstrap data can live in a couple of places:

  1. In the <metadata> node of an F4M file
  2. In an external .bootstrap file

The .bootstap file contains binary data and the F4M file contains the same binary data that has been Base64 encoded. So, getting to the data is pretty trivial – either read in the .bootstrap file or un-encode the string in that is in the F4M. Getting to the data contained in the bootstrap binary data is the fun part.

Understanding the bootstrap data

To do so, check out the F4V file format specification. This PDF gives you the details for the entire F4V file format. If you read through the PDF, you’ll  see that it is built using what are called “boxes”. These boxes are given identifiers such as “abst”,  “adaf”,  “adkm”,  “aeib”,  “afra”,  & “afrt” to name a few. Each box contains a header, that header identifies the box by its identifier and lets you know how much data is contained in the box. These boxes are also arranged into a hierarchy, so each box has some data that is specific to some part of the data contained in the file.

It is all in the boxes

The boxes that we are concerned with are “abst” or the bootstrap information box, “asrt” or the segement run table box, and “afrt” or the fragment run table box.

The abst box

The bootstrap information box contains information needed to bootstrap playing of HDS content – specifically to construct the URLs necessary to retrieve the fragments for playback. This includes information about the server, media, & segment information.

The asrt box

The segment run table box contains data about the segments for the media item. There can be multiple ASRT boxes – each representing a different quality level. There are some rules that you’ll want to pay attention to for the data in the asrt box:

  • An asrt box can represent fragment runs for several quality levels.
  • Each entry gives the first segment number for a run of segments with the same count of fragments.
    • The count of segments having this same count of fragments can be calculated by subtracting the first segment number in this entry from the first segment number in the next entry.

The afrt box

The fragment run table box is used to find the fragment corresponding to a given time. Similar to the asrt box, there are some rules that you’ll want to pay attention to:

  • Fragments are individually identifiable by the URL scheme based on segment number and fragments number.
  • Fragments may vary both in duration and in number of samples.
  • Duration of the fragments are stored in the this box.
  • A Fragment Run Table may represent fragments for more than one quality level.
  • Each fragment run table entry gives the first fragment number for a run of fragments with the same duration.
    • The count of fragments having this same duration can be calculated by subtracting the first fragment number in this entry from the first fragment number in the next entry.

Parsing the bootstrap data using Node.js

Parsing binary data in Node.js can be done using “Buffer”. For the most part parsing the bootstrap data was pretty straight forward. There is one issue that I ran into with 64bit Integers which was solved easily enough (there are node modules for just about anything) using the node-int64 module to represent the 64Bit Integers. Once that was solved it was just a matter of parsing through the box header to figure out where you are in the dataset, and then creating the appropriate data structures to represent what you want and need in from the bootstrap data.

In our case we want to be able to monitor live events across multiple servers to make sure that they are all on the same segment and fragment. We’re building a services that in the case that something happens to a server and it goes haywire, will notify another service that can then restart or shut down that particular server or let caching servers know that they need to flush or refresh cache. We’re still dreaming up things we can use this type of data for.

Just want to get to that data?

If you have a .bootstrap file you can use the f4fpackager.exe that is part of the Adobe Media Server toolset to inspect the bootstrap data. All you need to do is run the tool with the argument “–inspect-bootstrap”. So the command looks something like the following if you have a bootstrap file named mydata.bootstrap:

[shell].f4fpackager.exe –input-file=mydata.bootstrap –inspect-bootstrap[/shell]

Anyways, if you have any questions or input let me know in the comments.