This post is all about building an Alexa Video Skill, handling the requests and expanding the PoC created previously to control your streaming services on any Android based TV device. I’ve been testing this skill out at home for a while now (my poor partner), and it’s simplified how we watch TV at home significantly. It got it’s first big test over the Christmas period when we wanted to watch some of our favourite Christmas movies, but couldn’t be bothered to check which streaming service they were on or search for them for them in each streaming service app.
This post ended up being a lot longer than I expected when I sat down to write the final part (Maybe I should have broke it down again) but if you want to see the code behind this and skip my ramblings! It’s available on GitHub.
Overview
There are quite a few moving parts in this project, data being sent between services and devices being controlled. Below is a high level overview of how this project and how it’s dependencies function.
sequenceDiagram;
Alexa ->> Lambda: Alexa Request
Lambda ->> Self Hosted Service: Forward Alexa Request
Self Hosted Service ->> Trakt: Look up IMDB ID
Self Hosted Service ->> JustWatch: Look up content with details from Trakt
JustWatch ->> Self Hosted Service: Convert content links to deeplinks
Self Hosted Service ->> ADB Server: Connect to the device and process deeplink
ADB Server ->> Android TV: Send deeplink to device
Creating the Alexa skill
The most obvious place to start in a blog post titled ‘Controlling streaming services with Alexa’ is with the Alexa skill itself. Alexa provides several prebuilt models that you can use, which makes it a lot easier for us as it provides an extensive base set of entities and intents. To get started, head to the Alexa Developer Console and select Create Skill.
To begin with, we need to provide some basic details about the skill. Start by giving the skill a sensible name (I used TV Tuner) and select the Primary Locale that is relevant for your language, which for me is English (UK)
We’re now asked a few questions about what the purpose the skill is, on this screen you’re able to select one of the pre-built models that Amazon provide. Select Movies & TV and choose the recommend model of Video.
This will automatically select the Provision your own option for hosting your Alexa skill. At the top of the page, click Next. At this point you’ll be asked to review the information provided. At the top of the page click Create Skill. And that’s it, the Alexa skill has been created (the joy of Amazon’s pre-built skills).
Configuring the Alexa Skill
Unfortunately, because of a restriction on Amazon’s part (I’m assuming this is somehow related to using a pre-built model??). We’re required to create a Lambda function (Amazon’s brand of Serverless functions) to handle the request from Alexa. Thankfully, I had already written a proxy for handling requests from Alexa to a self hosted service for another reason (controlling my local Home Assistant). That proxy is called Haaska. To save some time, I’ve modified Haaska slightly to remove the HomeAssistant specific elements and replace it with a call to the service that I had built, deployed the modified code and now the gap from Alexa to our home network (Which is where we’ll be hosting all the logic and connections for our skill) has been bridged. P.S. If you’re a Home Assistant user, I’d recommend checking out Haaska.
You’ll need to deploy the function to AWS, and complete the initial configuration. If you’re not confident with this, you can follow most of the setup instructions for Haaska to achieve this.
Handling the requests from Alexa
Phew, it feels like we’ve done a lot of work already and we’ve not even written a single line of code 😞 (or got any content playing on the TV). It’s time to change that. Let’s set up a new NodeJS project and get on with it! (Note, this code could probably be improved heavily by making use of the Alexa Skills Kit but this is just me hacking away on a personal project). To start with, we need to let Alexa know about the devices that can be controlled. In Alexa terms, this is part of the Discovery process. The same process that’s kicked off when you get a new smart light and have to tell Alexa Discover new devices. As this is a small project I’ve just gone ahead and hard coded my devices. Because let’s be honest, I’m not getting a new TV every day!
const express = require('express');
const app = express();
app.use(express.json());
const devices = [{
"id": "192.168.1.1",
"friendlyName": "Office TV",
"type": "firetv"
}]
app.post('/', async (req, res) => {
const event = req.body;
const directive = event.directive;
if (directive) {
if (directive.header.name == 'Discover') {
console.log('Discover Request received from Alexa');
const alexaEndpoints = devices.map(x => ({
displayCategories: ['TV'],
capabilities: [
{
interface: 'Alexa.RemoteVideoPlayer',
type: 'AlexaInterface',
version: '1.0',
},
],
endpointId: `tvtuner#${x.id}`,
description: `${x.friendlyName} by TVTuner`,
friendlyName: `${x.friendlyName} TVTuner`,
manufacturerName: 'tvtuner',
}));
const resp = {
event: {
header: {
messageId: directive.header.messageId,
name: 'Discover.Response',
namespace: 'Alexa.Discovery',
payloadVersion: '3',
},
payload: {
endpoints: alexaEndpoints,
},
},
};
res.send(resp);
return;
}
}
res.sendStatus(200);
});
const port = process.env.PORT || 3000;
app.listen(port, () => {
console.log(`API server listening on port ${port}`);
});
It feels like there’s a lot of code to achieve such a simple task, but the main points of interest are the mapping of our ‘devices’ to the Alexa discover schema, and returning that data. I’m being lazy and using the device ID as tvtuner#... so that I can parse the data after the # to know which device to play the content on.
Playing the requested content
Great, at this point Alexa knows about the devices that are capable of playing content. We now need to handle the request to actually play some content now. We get a Directive from Alexa SearchAndPlay (similar to the Discover directive above.) so we can handle it from there. I’m stripping the target down to just the IP address of the device and sending that to the new method playContent
if (directive.header.name == 'SearchAndPlay') {
const imdb = directive.payload.entities[0].externalIds.imdb;
const target = directive.endpoint.endpointId.replace('tvtuner#', '');
const data = playContent(imdb, target);
const resp = {
event: {
endpoint: {
scope: {
type: 'DirectedUserId',
directedUserId: 'some-Amazon-user-id',
},
endpointId: directive.endpoint.endpointId,
},
header: {
messageId: directive.header.messageId,
name: 'Response',
namespace: 'Alexa',
payloadVersion: '3',
},
payload: {},
},
};
res.send(resp);
await data;
return;
}
playContent will be the method responsible for handling translating the IMDB ID, searching JustWatch and processing the deeplink.
Identifying what we want to play
We’re slowly inching closer to the end goal, Alexa knows about our devices, and now Alexa can now tell our service to play some content somewhere. We’re now getting to the first hurdle. Alexa only provides us with the IMDB id for the content that we have requested to play. But JustWatch doesn’t support searching with that ID, only the title of the content. We could (in theory) just use the IMDB API to get the right data back. But I’m going to use a service called Trakt (Mainly because I’m already an avid user… and a big fan of their service and API). The main reason for using Trakt, is I’ll be able to filter out previously watched content, which can be handy if a TV show moves streaming services inbetween seasons. Or one service only has the latest season, but not the first x seasons.
Finding the streaming service
I’ve been kindly asked by the people over at JustWatch to include a disclaimer regarding the unofficial and unsupported usage of their API.
The work of many developers went and is still going into the development and maintenance of the data and the API. JustWatch’s main business is to operate a streaming guide with apps for iOS and Android. They offer the data for business intelligence and marketing. Therefore it is prohibited to use the API for commercial use (consumer service, data science, business intelligence, etc.). It is ok to use the API for private projects, but please be respectful with your API calls to not put too much load on the API. The API is not supported in any way and will change frequently without warning. If you would like to work with JustWatch and use the data/API please get in contact with them via [email protected]. Currently, JustWatch can only work with bigger partners and clients.
Thanks to the great folks over at JustWatch, we can query their API for which streaming services have what content. I’ve made use of the filtering that they provide (I’m not made of money unfortunately and can’t subscribe to all streaming services). You can also filter content based on the cost of it, they separate out shows that are included as part of the package and those that are paid (For example, Amazon Prime Video have content that is included in the subscription, and content you need to pay separately for). For my personal use case I’m only interested in content that is either free or included in the subscriptions that I have.
There are a few quirks to be aware of when using the JustWatch API (most likely given it’s not really been designed for people like me to use). You’ll notice that we have to filter out some data that is returned but we didn’t search for, as well as making additional requests to get the data that we need to show the content. And given we’re not making JustWatch any money it’s worth pointing out they may turn the service off, require paid API keys or similar. And that’s fine, they’ve invested the money in accumulating the data and are being fairly generous in not stopping the unofficial access.
Using the JustWatch-API package, it’s quite simple for us to query the API.
You’ll need to instantiate it with the following, making sure to provide the relevant locale.
const justwatch = require('justwatch-api');
const JustWatch = new justwatch({ locale: 'en_GB' });
The search is also relatively straightforward. You’ll need to work out the provider IDs for the services that you’re interested in (just use their website and look at the network requests made in the background).
const providers = ['al4', 'amp', 'bbc', 'dnp', 'nfx', 'itv']; //All4, Prime Video, BBC iPlayer, Disney+, Netflix, ITVX
const monetization_types = ['flatrate', 'free']; // Flatrate = included in subscription
const params = {
query: data.title,
providers: providers,
monetization_types: monetization_types,
content_types: ['show'],
};
const shows = await JustWatch.search(params);
At this point we’ll get all the results relating to our search term, their search API seems to be optimistic, for example searching for Buffy the Vampire Slayer will also return The Vampire Diaries.
const shows = await JustWatch.search(params);
const show = shows.items.find((item) => {
return item.object_type == 'show' &&
item.original_release_year == data.year
});
const episodes = await JustWatch.getEpisodes(show.id);
const offer = episodes.items[0].offers.find((item) => {
return x => providers.includes(x.package_short_name) && monetization_types.includes(x.monetization_type)
})
return offer;
In that code we’ve cleaned up the returned data, for the purposes of this post I’ve limited it to TV shows, and we’ll filter all the shows returned to any that are in the right year. The rest of it is just filtering to get the correct episode and then filtering out the providers and price points we’re not interested in. Hey Presto, we’ve got our deeplinks!
Given how infrequently this data changes, and because we’re all considerate people (we are, right?). It’s worth caching the data that is returned. It’ll prevent their services from being unnecessarily hit with requests for the same data.
Controlling the TV
For controlling the TV, we’re going to use a protocol called ADB (Android Debug Bridge), and then we’re going to use a docker container that connects to all of our TVs via ADB, and give’s us a single point of access for our application.
Depending on the devices that you want to control, depends on how you enable ADB on that device. The FireTV devices are the easiest!
For a Google TV, head to Settings > Device Preferences > About and click the Build number several times until a prompt appears confirming developer options have been enabled. Once that’s been done, head back to the menu and you should now have a Developer Options menu in the Device Preferences menu.
On a FireTV device, head to Settings > Device > Developer Options and turn on ADB Debugging. And…. that is all it takes.
Given this post isn’t about connecting your android devices to ADB, I’m not going into detail on how it’s done.
With all of our TV’s connected, it leaves one more bit of code remaining. Actually sending the deeplink to the device.
const deeplinkCommand = await deviceClient.shell(
`input keyevent 3 && am start -W ${deeplink}`
);
The code above isn’t particuarly complicated, I’m sending the home button to the device (the input keyevent part) to ensure that I’m not in any active application (if you’re in Netflix and send a deeplink for Netflix content then the command is ignored) and then I’m sending the deeplink that has been returned from JustWatch. Your TV should react fairly instantaneously and the content will open.
Linking your Echo’s to your TV’s
This is it, we’re onto the home straight 🥳!
There is one final part of the configuration that we need to set up. This will tell the skill which device you want to play the content on when interacting with the echo device. For example, when saying ‘Alexa, stream Buffy the Vampire Slayer from TV Tuner’ to my Alexa in the living room, Alexa will know to instruct our service to start playing content on the TV in the living room.
Alexa App -> More -> Settings > TV & Video -> {Skill Name} -> Link or Manage Devices
Now that you’ve got the devices linked, you can speak to each individual echo and just ask it play the content. Try out Alexa, stream Buffy the Vampire Slayer from {SkillName}. Sit back, get the popcorn and you’re sorted!
Validating the request
With Alexa skills they provide a way of validating that the request is genuinely from Amazon and not spoofed in anyway. More Details. As this just a small project, I just went ahead and implemented a pre-shared key between Haaska and this code. Checking that it exists within the header of the request. If you’re creating a project that will last longer, you should absolutely be validating the timestamp and signature of the incoming request.
Future Steps
You may have noticed that I’ve cut a certain amount of corners, fine, a lot of corners. I could have spent a long time perfecting this, but in all honesty it has worked 99% of the time that me and my partner have used it. I could have gone with proactive responses, instead of just immediately returning a success to Alexa whilst I try (and potentially fail) to find the right content on a streaming service I have. I could have implemented extra security features as above.
If you have any questions about this, reach out and I’ll try to answer them. If you take this project further, I’d love to know how you’ve used it and how it’s worked for you!
Comments