My First Twilio App
My First Twilio App
Intro and Overview
I recently started at Twilio, and as part of the onboarding process is the opportunity to earn a coveted red track jacket. You earn this jacket by building and demoing an app that you make using the twilio platform - this takes you out of your day to day role and into the customer's shoes, giving a sense of camaraderie with the customers, reading the same docs, understanding how things work, and common language to speak - all good stuff!
This walkthrough will document my journey to the track jacket and serves to help anyone following along build their own app. I will be outlining the general process of setting up a Twilio account, the steps I followed to learn and build my app, and provide some tips and tricks as I learn them.
As we kick off, here are the rough high level things we will go through:
Brainstorming app ideas
Setting up my development environment
Setting up a Twilio account
Reviewing and learning about the API's available to me
making my app
testing the app
So let's dig in!
You can also find my full example app hosted on github here
Brainstorm session
Some of my ideas for the app are:
a number guessing app, the server generates a random number between 1 and 10, and then the user guesses a number, the server returns a match or a clue
a photo a day collage app, the user is prompted to submit a picture of what they are doing at that moment. At the end of the week a montage of the photos is returned
a mad lib generating app, the user is prompted for a noun, verb, etc.. and the end result is a completed mad lib.
a photo identifier application, submit a picture and have it analyzed for content, think silicon valley tv show, hot dog or no hot dog ha!
a music recommendation app, send in a band you like and get a new music suggestion in return
App outline
For my submission, I am going to go with the idea for a music recommendation application. There are several reasons I chose this app as my demo;
First - it's actually something i want! There are so many times I search through my music library and just want something new, or i'm not quite sure what to listen to, so I think that this will actually be helpful to myself!
Second, there is an easy API provider for this exact thing. TasteDive.com actually provides a recommendation engine for a lot of things, from Music, to books, to TV shows and movies. They also offer a very friendly website that you can use, but they have a free API that we are going to integrate with for our project.
The way that users will interact with my application is via SMS - the users will send a band or artist they like to my service via a text message, and my service will text them back with a new artist to check out. Seems easy enough!
Behind the scenes this is the architecture and flow of the requests:
So let's walk through this step by step:
the user on their phone sends a text message to my applications phone number - that message is sent from their handset to the carrier AT&T for example
the carrier routes the message from their infrastructure and routing it to the SaaS hosted Twilio service
The Twilio service consumes the message request
As part of my phone number configuration, I have given it a webservice URL for it to route incoming messages to.
Here twilio takes the incoming message and routes it to my configured web service (this is the app we will be making)
The application we are going to create will consume the users SMS, and send the artist name over to the tastedrive.com API
The tastedive API will respond to our API request with a list of suggested artists if it has any
My application will then take that list of artists, randomly pick 1 from the list, and send some TwiML back to the twilio service
The twilio service will then push that back up the stack from the phone service to the carrier interfaces
The SMS with the recommended artist then goes back out over the internet to the Carrier
The Carrier SMS gateway consumes the SMS
The SMS with the fresh recommendation is delivered back to the user's handset!
Building our App
Creating a Twilio Account
Looking at our diagram above, the Twilio services are a central link between the users and our custom application. To get a phone number, and to manage what happens when users send it messages, we will need a Twilio account. So let's start with that.
Getting started with Twilio is actually very easy!
Open a web browser tab - don't close this one ;)
in the upper right hand corner hit the sign up button
You will then be presented with a quick form to create your twilio account
Once you have completed your account, you will have a fresh trial account with $15 of credits to use any of the twilio services! That is more than enough to get started with buying a phone number, and sending some test messages and ultimately some music recommendations.
Buying a phone number
Now that our account is set up, we need to get a phone number for users to text. With Twilio this is about as easy as possible.
From your account home expand the services menu and select the phone numbers feature
From the phone numbers feature select the Buy number in the upper right hand corner
From there we are going to search for a local number - in this case something in the Atlanta area
and select Buy next to a number you like
When you select a phone number it will show you the capabilities that are associated with that phone number, for example here this number supports voice, fax, SMS and MMS services.
Hit buy in the bottom corner to pick this number and add it to your account.
When the purchase is complete you should see your phone number properties and details.
Lets go ahead and do a quick test - send a text message to the phone number, at this point it does not matter what you say, so i am just going to say "hi". Since the number is unconfigured at this point you will get just a default message back.
Great! We have a number that is able to send and receive text messages and is under our control to configure and manage.
Configuring our development environment
Tools
Many people, especially experienced developers will already have their preferred tools and languages, extensions, etc.. already defined, but since I am new to this whole development thing, I am picking what I have heard around the virtual watercooler, and learning those.
For my code editor, I am going to use Visual Studio Code, its lightweight, and tools of extensions for different coding languages and lots of integrations and customizations that can be configured.
I am going to be using and writing a node.js based application, so I have installed node on my mac
While node itself has a lot of functionality as part of the core language, there are a couple of additional libraries I am going to use to make my application development a bit easier. These additional libraries are Express, bodyparser, axios, and of course the Twilio library
To make my development, testing and troubleshooting a bit easier, I am also going to use nodemon.js. This is a really handy tool that watches your code files for changes and as you update and save them, it will automatically reload the node app, so you're always running the latest code - very handy!
and finally to make my application available on the internet without having to open firewall ports, get a URL and all the mess of making network and firewall changes, another handy tool called ngrok.
API docs to keep handy
What is TwiML
TwiML is the markup language and formatting used by the twilio services Here is a link to the Twilio Docs:
TwiML for Programmable SMS
TwiML for Programmable Voice
Create a basic SMS response
Based on the documentation referenced above, we can see a simple example to get us started. Looking at this example we see the text "Hello World!" wrapped in the Message Tag, which itself is wrapped in the Response Tag. This is an example of the TwiML markup that will inform the Twilio Platform to respond with an SMS, in this case a message that simply says "Hello World!"
<Response>
<Message>Hello World!</Message>
</Response>
Creating the Application
After installing the tools mentioned above, we are going to go ahead and start writing our application. In VSCode, I am going to open a new document, and save it as recommendation.js. By saving it as a javascript file, VS Code knows the file type and will do some dynamic text colorization, auto complete, etc.. which is really handy.
The first part of our code is going to add in the required libraries for our application, and define some variables. As noted in the development environment section, we have installed a few additional libraries, so lets add them to our application. This will make the functionality of the Express, body-parser, axios and twilio libraries available to be consumed and used in our application.
This will also define the port that our application will run on - we will need this later with the ngrok app to expose the services to the internet. Here that port is defined as 3000, but you can make it whatever you want as long as it is not in use on your machine already.
const axios = require('axios');
const express = require('express');
const bodyParser = require('body-parser');
const app = express()
const port = 3000
app.use(bodyParser.urlencoded({ extended: false }));
const MessagingResponse = require('twilio').twiml.MessagingResponse;
const url = require('url');
The next block of code here will create a new server running on the defined port, and generate several endpoints.
The first endpoint it will generate is just the root of the site, so if a user went to http://localhost:3000 the server would respond with the classic "Hello world! "
The second endpoint is the /recommend route, this is where our application is going to live, and its path is http://localhost:3000/recommend - following along?
Since I am new to this whole programming thing I am going to be using the console.log function to output various things to the console window while the app is running so i can follow along with what is happening as users are submitting requests. Here I am creating a new variable called usersubmission, and setting it equal to the value of request.body.Body.
When a message is sent to twilio and it forwards it onto the configured messaging webhook, it will send a bunch of useful information, such as the phone number of the originating user, the actual text the user submitted among other info. Here the text the user sends is what is interesting to us, as it will contain the artist the user is looking to base new music off of. That data is stored in the Body field. Here we are simply storing the requested artist as a variable called usersubmission.
Finally we are outputting that value to the console log, again just so i can follow along easier. In a real application, you may have some other logging system or output for later analysis.
app.listen(port, () => {
console.log(`Example app listening at http://localhost:${port}`)
})
app.get('/', (request, response) => {
response.send('Hello World!')
})
app.post('/recommend', (request, response) => {
//collect the incoming user submission
let usersubmission = request.body.Body;
console.log('the user submitted '+usersubmission) //outputs the user submission to the console log
Our next section of code is going to do the bridge from consuming the user's requested artist to sending the query to the tastedive API. Here I am creating a new function called getrecommendedmusic
We are going to use the axios library here to build an outbound http request from our application to the tastedive api. If we look at the TasteDive API docs we can see how the HTTP messages are expected to be generated. In this case they have a parameter of "q" that should equal the query we wish to submit. So to build that, we are creating a new variable called payload and setting its value to be "q:usersubmission".
remember that earlier we set the usersubmisison variable to equal the body of the incoming twilio message, which was equal to the text of the message from the user's original SMS.
We are then bundling that up and setting the url of the tastedive api and our parameters variable.
If there are any matches, the tastedive API will return a JSON response with artists that the user may like.
Here we are going to count how many matches were returned, and store that value as the variable mycount
Just like before i am going to print that value to the console log, so i can see how many matches there were.
async function getrecommendedmusic () {
let payload = { q:usersubmission };
console.log(payload)
const params = new url.URLSearchParams(payload);
let res = await axios.get(`https://tastedive.com/api/similar?${params}`);
let data = res.data;
let mycount = data.Similar.Results.length;
console.log ('my count is '+mycount)
OK, so far so good, I am receiving the incoming message from twilio, picking it apart to get the users requested artist, sending that to tastedive and they are sending back other music i may like. I also know how many different artists they suggested.
In this next block of code we are going to define a new function that will generate a random number. In this case I don't know ahead of time how many potential results there are for any given artist, so hard coding a number range would be bad. Instead we are going to tell the random number picker to use the mycount variable to set the range of possible numbers. If tastedive returns 5 results, that will result in number guess from 0 -5, and if it returns a count of 20 for example, then the range is 0-20.
Later we will use this random number to pick from the list of returned matches to send back to the user.
Again here we are going to print the random number to the console log
and finally to confirm that the taste dive web service found a match for what I submitted, we are going to pull from the taste dive response the value it searched for and store that in the "submitted" variable. In this case it happens to be stored in the Name field of an Array.
Again printing that out to the console log (picking up on my pattern?)
// defining a variable that will be used to pick a result from all the recommended music .
randomnumber = getRandomInt(mycount);
//This is the function to generate a random number that is stored in the above variable
function getRandomInt(max)
{
return Math.floor(Math.random() * max);
}
console.log('the random number generated is '+randomnumber)
//console.log(randomnumber)
let submitted = data.Similar.Info[0].Name
console.log('submitted '+submitted)
Now there are several possible outcomes of the user's search. It may result in matches, or it may not. In this next block of code we are going to test for a search that ended with no matches and prompt the user to ensure spelling is correct or try another artist.
Recall that we set the "mycount" variable to be the count of matches returned from the tastedive service. Here we are checking to see if that value is equal to 0. If it is then this test will be true and the following logic will be triggered.
That logic includes printing out to the console log that the recommended count was 0, and that the user is being asked to try with a different artist.
We are going to use response.send to push back some TwiML to twilio with a message explaining that we got no results.
A second response.send will expand on why - and suggest they try another artist.
The 2 response.send actions will result in a pair of text messages being sent to the user.
if(mycount == '0')
{
// Send a message to the console log that the returned number of matches was 0
console.log('recommended count was 0');
console.log('asking user to retry with a different artist');
// Here we are now going to send a response back to the user who sent in the message. This is formatted using the TwiML XML Response / message tags
response.send( "<Response><Message>Sorry, but that search resulted in no results.</Message></Response>");
response.send( "<Response><Message>That is typically due to either a very niche artist, or incorrect spelling, double check your spelling or try another artist</Message></Response>");
}
This next block of code will be evaluated if the Tastedive service returned at least 1 match. Where the prior statement was looking for the "mycount" variable to equal 0, here we are checking to see if it does NOT equal 0.
We are now going to build a new variable called "result1". Recall that earlier the tastedive service sent us a list of recommended artists, that list was stored in an array and is addressable by a number. Here we are going to use the random number we generated earlier to pick one of the artists.
That random number was stored as the variable "randomnumber" so our result1 variable is going to be equal to whatever artist name is stored in the random numbers array value.
Here we are going to use the Twilio Libraries to create a Message response, instead of generating the TwiML myself.
Here I am defining the message back to the user:
if you like *the artist you submitted* we think you will also like *the randomly picked artist from the list of matching music*
So for example if you submit Led Zeppelin the message sent back to the user may look like:
If you like Led Zeppelin we think you will also like David Bowie
else if(mycount !=='0')
{
let result1 = data.Similar.Results[randomnumber].Name
console.log('You may like '+result1)
const twiml = new MessagingResponse();
twiml.message('If you like '+submitted +' we think you will also like '+result1);
response.writeHead(200, {'Content-Type': 'text/xml'});
response.end(twiml.toString());
}
And finally we are going to define a catch all in case something goes wrong.
Here a message will be logged to the console that there was an unknown entry
And a message using the twiML formatting is being generated to send back to the user that there were no matches, try another artist
and then just closing out our application
else
{
console.log('unknown entry')
response.send( "<Response><Message>I wasnt able to find any suggested matches, try another artist</Message></Response>");
}
}
getrecommendedmusic();
})
Running the Application
Now that we have all of our code written for our application lets get it running and a public URL that it can be reached by. This will allow us to configure the twilio service in the next step, to forward on the users incoming SMS to our service.
Since we have the nodemon tool installed from our previous steps of installing tools and frameworks we are going to use that to run our application.
On a mac, open the terminal application and change the directory to the folder that you saved your file in
In the Terminal window start the nodemon application with your file as an argument: nodemon filename
if you have no code errors - you should see a print out in the console that your application is running on port 3000: http://localhost:3000
If you get some errors - you will need to correct any syntax or code issues first
Once our app is running as expected, lets start up ngrok.
ngrok will create a connection to our local app running on port 3000, and give us a URL that's out on the internet to expose our app.
Open another terminal window and navigate to the /Applications folder (or whatever path you have the ngrok executable)
run the command ngrok http 3000
this tells ngrok to use standard HTTP and HTTPS ports (80 and 443 respectively) on the public interface and to map those to the localhost port 3000
When that is running you should see a pair of urls listed in your terminal window, in this case for the http / https connections as mentioned. Lets copy the HTTPS url.
Configuring the Twilio service
Now that we have our application running on our local machine and exposed on the internet, we can now configure twilio to pass on the incoming SMS messages to it.
In your twilio console, navigate to phone numbers, select the phone number we configured to edit its properties.
Under the Messaging section, configure the settings for when a message comes in to the webhook option, paste our ngrok url, and add the application route in our case that was /recommend - the full path should be something like https://ngrokurl.com/recommend
save the changes
and finally - lets rock!
Now let's give it a go!
On your phone create a new text message, sending it to the phone number we bought in Twilio, and enter a band name, in my case I am looking for music like Led Zeppelin - and send
Look at that, we get back a new artist to check out !
As a final reminder you can also find my full example app hosted on github here