Today’s Secret Deep Dive is written by Tovi Levis, Senior Software Engineer at SCRT Labs.
Hello to the Secret community!
Today we’re going to take a look at the secret sauce behind KillRoy Was Here – the first-ever NFT full-length feature film ever minted. It was a privilege for us to partner with Kevin Smith and unlock a groundbreaking new experience for users.
At Secret Network, we take pride in pioneering novel solutions and exploring bleeding edge technologies. At the same time, we want to ensure the user experience is as high quality as possible – perhaps even rivaling Web2 platforms. It was important for us to have the KillRoy Was Here release set the standard for what’s possible with Secret NFTs, and NFTs more generally!
To this end, we decided to tackle two main technical issues that we felt would make the experience deserving of the first ever release of a feature film using blockchain technology:
- NFT holders being able to stream the film
- Being able to watch the movie on mobile devices
In this blog post, I’ll share with the technical details of how we achieved each of these goals.
Video Streaming using a Secret NFT
The first goal was to play the encrypted video directly on site instead of downloading and decrypting it offline.
In order to do that, we used the HLS (HTTP Live Streaming) protocol that is industry standard for video streaming, which also has native support for playing encrypted videos. In a nutshell, HLS format contains a manifest file (m3u8 file) that describes a list of video files and their length. Each video file is a small chunk of the whole video (H.264 encoded), which optionally can be encrypted with AES-128-CBC (or SAMPLE-AES for encryption of partial content).
Here is an example of a m3u8 file:
As you can see, the intended way that this type of encryption is meant to be used is by providing the link to the encryption key in the manifest file itself. Unfortunately, it seems the creators of the HLS specification did not consider encryption keys that are stored in Secret NFTs. No matter! It just means we need to somehow break this link so the player would be able to get the encryption key from the private metadata inside the Secret NFT.
Essentially, if we want to inject a dynamic key into this file, we have two approaches. The first would be to download the m3u8 file, modify it in the browser, then provide it to the video player of choice. The second would be to override the key after the m3u8 is loaded into the player, but right before each H.264 video segment is fetched and played.
We examined one of the most popular open source browser video players – Video.js. Fortunately for us, we found that this player allowed us to “hijack” each HTTP request that the player makes before the request, check if it contains a known string and replace it with a new one. This allowed us to take the latter approach that we outlined above: we replace the encryption key URI in the manifest file with “key://0.key” and check to see if we can detect it:
The URI we replace is an echo server, whose goal is simply to echo the encryption key (plus verify and decode it), and return it back to the player. A naive implementation would be:
This is done over HTTPS to keep it secure end-to-end. You can find a full example of this implementation in the GitHub repo:
scrtlabs
Viewing Secret Content on Mobile
The second goal was to somehow allow NFT holders to view their exclusive content on their mobile devices.
Unfortunately at this current moment, Secret Network doesn’t have full support for a mobile wallet application that allows for out-of-the-box web3 application support, and so we need to get creative.
Secret.js (our 1st party Javascript SDK) can work on a mobile device and can provide us with the tools we need. We’ll make creative use of the Permit mechanism. (Permits are signed blobs that can be used to share something. Read more about it in our documentation.)
First, we took the permit that was signed in the desktop version with Keplr, and generated a QR code from it. Next, on the mobile side, we created a dedicated page for it, and used secret.js to generate a temporary wallet on-device.
Lastly, the user is prompted to scan the QR code from desktop so we can use the generated permit to query the users’ NFTs and extract the private data to be able to stream the videos on the mobile device (using the same methods described above).
This method seems to work without any issues on iOS and Android devices. The only caveat was that iOS doesn’t support our “HLS hack” since it doesn’t have MSE support.
So only in iOS, we use the native video player (that supports HLS) and provide it a source that we modified and served as a “data:application/x-mpegurl;base64,[data]”. It also turns out that this native mobile player UI works pretty well, and we didn’t have to use a different 3rd party player (thanks Apple!).
With these technical challenges sorted, we turned our attention to other key aspects of the user experience – for example, ensuring users can easily get their first SCRT and participate in the mint. (We were proud to partner with Crypto.com as a part of this effort!) We tried to ensure every piece of the UX was as good as it could be, and in the process we learned a lot about what other fundamental improvements could be made to the network, its tooling, and its resources.
Going Deeper
At SCRT Labs, we’re constantly trying to innovate on everything – from L1 privacy for smart contracts to next-generation Secret Apps. We take a deeply pragmatic approach to everything we build, because we care about driving real-world adoption by millions of users. (If you’re curious about our approach at the blockchain level, check out our recent post series on Web3 privacy!)
Guy Zyskind
We’ve thought deeply about these issues for years, and Secret Network is proud to continue leading the charge towards secure computation in the blockchain space. If these ideas interest you – and if you’re convinced of their importance – please join us in building solutions and technologies that can help us secure the decentralized web and scale Web3!
If you’re an app developer and you want to build cool things like the KillRoy Was Here streaming experience, check out our dev resources here. (Secret uses Rust!) You should especially check out our new Getting Started Guide.
If you’re anyone passionate about ensuring Web3 users have the data privacy protections they need and deserve, consider becoming a Secret Agent! It is our mission to make sure the decentralized web we’re building is one that truly empowers – and one that is accessible to all. From awareness and education to international growth and university relations, there’s many ways to help contribute to the expansion of the Secret ecosystem and the global availability of privacy technologies in Web3.
Check out the Secret Agents program and join one of the best and most committed communities in the entire blockchain space!
Thanks for reading through this Secret Deep Dive. Upwards and onwards!
