Creating a Clova Skill with Elixir

Greetings from Kyoto! My name is Adam Millerchip, I’m a server-side Engineer at the LINE KYOTO office. This post is day 7 of the LINE Advent Calendar 2018. This year I’ve been working on developing a package in Elixir for developing Clova Skills, so for today’s topic, I’d like to do a walkthrough of developing a simple Clova skill using Elixir.

In terms of new programming languages, Elixir is pretty new and trendy. People often ask me “Why don’t you learn something like Go?”. Well, if you’re interested in Go, be sure to check back tomorrow, because Shunsuke Suzuki will be talking about new features and changes to drone, a Go-based CI/CD platform. 🙂

Since this is an Advent Calendar blog, I thought it would be relevant to make an Advent Calendar skill. To keep it simple, we’ll just count the number of days to Christmas. That’s a bit too simple though, so in order to demonstrate how to do something a little more complex, we’ll also extend it to calculate the number of days between today and any date specified by the user.

We’ll use the official Clova Elixir SDK, which I developed this year while learning Elixir. It is still a work in progress so it may completely change in the future. If you have any feedback, Github pull requests and issues are warmly welcomed!

When creating a skill, the first thing we need to do is to register and configure it with the Clova Developer Center. That’s a blog post in itself, so I’ll skip that for now and let’s mock the messages from the server using an HTTP debugging tool.

Before we start, we need to take care of some pre-requisites:

  1. Install Elixir. I prefer to use ASDF with the elixir plugin. This allows you easily control the version of elixir you use per project, without depending on your system’s environment.
  2. Install ngrok. This is a proxy tool that will allow us to communicate with the Clova server from our local development environment.
  3. Install Postman. This is useful for debugging HTTP requests.
  4. Clova device, to use your skill. If you don’t have a Clova device, you can still use HTTP requests and the online Test GUI to test your skill, but you won’t be able to interact with it verbally.

Getting Started

Create the app

First, let’s make a new Elixir project using Elixir’s mix tool. I’m going to call this project advent.

mix new --sup advent

The --sup option tells mix to create a supervision tree for us, which will restart our server if it crashes.

We should now have have a basic Elixir project in the advent directory. You should be able to start it and call the test function:

$ cd advent
$ iex -S mix
iex(1)> Advent.hello
:world

Add dependencies

Currently we have a minimal Elixir application. Let’s add some dependencies:

  • Clova – For communicating with the Clova server
  • PlugCowboy – An HTTP Server
  • Poison – JSON
  • Timex – time zone handling

Update the deps function in mix.exs so that it looks like this:

defp deps do
  [
    {:clova, "~> 0.5.0"},
    {:plug_cowboy, "~> 2.0"},
    {:poison, "~> 3.1"},
    {:timex, "~> 3.4.2"}
  ]
end

Then we should be able to get and compile the dependencies:

mix deps.get && mix deps.compile

Implement a Web server

First, we need to tell Elixir what to do when our application starts.

In lib/advent/application.ex, update the start function to define the child processes like this:

  def start(_type, _args) do
    children = [
      {Plug.Adapters.Cowboy, scheme: :http, plug: Advent.Router, options: [port: 4000]}
    ]

    opts = [strategy: :one_for_one, name: Advent.Supervisor]
    Supervisor.start_link(children, opts)
  end

This tells our application to start a webserver listening on port 4000, and to forward requests to the Advent.Routermodule.

After this our application will not build, because the Advent.Router module does not exist, so let’s create it.

Create lib/advent/router.ex with the following contents:

defmodule Advent.Router do
  use Plug.Router

  plug :match
  plug :dispatch

  post "/clova" do
    send_resp(conn, 200, "Hello Clova!")
  end
end

This should be enough to serve a simple HTTP request. Run the server with iex -S mix, then test with Postman by sending a POST request to http://localhost:4000/clova.

Handling Clova Requests

Obviously we need to do more than say hello. Let’s add some more code to our HTTP router:

defmodule Advent.Router do
  use Plug.Router
  use Plug.ErrorHandler

  plug Plug.Logger

  plug Clova.SkillPlug,
    dispatch_to: Advent,
    app_id: "com.example.advent",
    json_module: Poison,
    # FIXME required until we start receiving real requests from the Clova server
    force_signature_valid: true

  plug :match
  plug :dispatch

  post "/clova" do
    send_resp(conn)
  end

  match("/clova", do: send_resp(conn, :method_not_allowed, ""))
  match(_, do: send_resp(conn, :not_found, ""))

  def handle_errors(conn, %{kind: :error, reason: reason}) do
    send_resp(conn, conn.status, Exception.message(reason))
  end
end

A little more complicated now. We added some logging and error handling, so we can tell what’s happening if something goes wrong.

The main part is the Clova.SkillPlug section. This assumes that all requests will come from the Clova server. It validates the request, decodes it, and calls our application’s Advent module to generate the response.

One important point is that SkillPlug verifies the request signature, to make sure it really comes from the Clova server. Because we will be imitating the Clova server with Postman in this guide, we tell SkillPlug to assume the signature is valid with the force_signature_valid flag. Later we’ll use some config to make sure we don’t skip the signature check in a production environment.

Let’s POST a sample Clova request. We need to set up a couple of headers in Postman:

Content-Type: application/json
SignatureCEK: ZHVtbXk=

Although we are forcing the signature to be considered valid, the validator will still parse it, so we provide a dummy Base-64 SignatureCEK header.

Then we can post a dummy Launch Request. This is simulating what the Clova server will send to our skill when somebody starts it. Use Postman to send the below JSON data to our server.

{
    "version": "1.0",
    "session": {
        "sessionId": "dummy",
        "sessionAttributes": {},
        "user": {
            "userId": "dummy",
            "accessToken": "dummy"
        },
        "new": true
    },
    "context": {
        "System": {
            "application": {
                "applicationId": "com.example.advent"
            },
            "user": {
                "userId": "dummy",
                "accessToken": "dummy"
            },
            "device": {
                "deviceId": "dummy",
                "display": {
                    "size": "l100",
                    "orientation": "landscape",
                    "dpi": 96,
                    "contentLayer": {
                        "width": 640,
                        "height": 360
                    }
                }
            }
        }
    },
    "request": {
        "type": "LaunchRequest",
        "intent": null
    }
}

We should get an error saying something like function Advent.handle_launch/2 is undefined or private. This is because the Clova SDK handled the request, and tried to call our application to generate a response. We didn’t implement a request handler yet, so let’s do it now.

Handle the request

We will write our main application logic in lib/advent.ex. It currently contains the default sample code, but we can replace it with this:

defmodule Advent do
  use Clova

  def handle_launch(_req, resp) do
    resp
    |> add_speech("ハロー、ワールド!")
    |> end_session
  end
end

This module uses the Clova module, which imports functions we can use to read the contents of the Clova request, and generate a response.

We need to define callbacks to handle the Clova request. Here we defined handle_launch, which is called when the user launches our skill. We ignore the contents of the request for now, but we do manipulate the response. We perform two operations on the response:

  1. Add the text “”ハロー、ワールド!”. This is what the Clova device should read out when the skill is launched.
  2. End the session. This tells the Clova device not to prompt for any more interaction from the user.

Let’s start the server again with iex -S mix and try out the dummy POST call again with Postman.

Hopefully this time our server should return a JSON response like this:

{
    "version": "1.0",
    "sessionAttributes": {},
    "response": {
        "shouldEndSession": true,
        "reprompt": null,
        "outputSpeech": {
            "values": {
                "value": "ハロー、ワールド!",
                "type": "PlainText",
                "lang": "ja"
            },
            "type": "SimpleSpeech"
        },
        "directives": null,
        "card": null
    }
}

This is the format the Clova server expects, and in this case it should read out “ハロー、ワールド!”.

Get it working on the Clova device

Let’s try to get our server to respond to a real request from the Clova server. If you don’t have a Clova device, you can skip this section.

Let’s start ngrok, which creates a publicly accessible proxy to our local server on port 4000:

$ ngrok http 4000
ngrok by @inconshreveable

Session Status                online
Session Expires               7 hours, 59 minutes
Version                       2.2.8
Region                        United States (us)
Web Interface                 http://127.0.0.1:4040
Forwarding                    http://********.ngrok.io -> localhost:4000
Forwarding                    https://********.ngrok.io -> localhost:4000

Connections                   ttl     opn     rt1     rt5     p50     p90
                              0       0       0.00    0.00    0.00    0.00

Let’s test it locally in Postman first. Change the localhost URL in Postman we used earlier to the HTTPS URL generated by ngrok (https://********.ngrok.io/clova – don’t forget the /clova on the end). We should get the same result as before. This means the Clova server should also be able to reach our server using that URL.

Now we need to tell the Clova server to contact our skill server. To do that, we need to register it on https://clova-developers.line.biz/. This post doesn’t have room to explain how to use the Clova Developer Center, but here are some important points to keep in mind:

  1. The extension ID you select must match the app_id that we specified in lib/advent/router.ex.
  2. The 呼び出し名 determines how to start your skill. In my case, I used 「アドベントカレンダー」.
  3. You need to login to the Clova Developer Center using the same LINE account that’s linked to your Clova device, if you want to use your skill during development.
  4. You need to define and build an interaction model (対話モデル)before Clova will use your skill. This is not a simple task, but unfortunately there is not enough space to explain it here. Please try to use the official guide to define an interaction model as follows:
    1. Enable the built-in CLOVA.DATETIME slot type.
    2. Add a Custom Intent called different_day.
    3. Import this specification(advent_intent_different_day) for the different_day intent.
    4. Build the interaction model.

If this goes well, you should be able to set the ngrok URL as your skill’s URL in the Clova Developer Center, and when you say 「アドベントカレンダーを起動して」, Clova should call our skill server, and Clova should say 「ハロー、ワールド!」

Implement the skill behaviour

Count the days to Christmas

Now we have a server that communicates with the Clova server via JSON messages. Let’s update the handle_launchfunction to actually do something useful. First, we’ll add some helper functions. Place these functions inside the Adventmodule in lib/advent.ex:

  def say_days_to_christmas(0), do: "クリスマスの日です!メリークリスマス!"
  def say_days_to_christmas(days) when days < 0, do: "クリスマスはもうすぎました。来年まで楽しみましょう。"
  def say_days_to_christmas(days), do: "クリスマスまであと#{days}日です!"

We can test these in iex:

iex(1)> Advent.say_days_to_christmas(0)
"クリスマスの日です!メリークリスマス!"
iex(2)> Advent.say_days_to_christmas(-1)
"クリスマスはもうすぎました。来年まで楽しみましょう。"
iex(3)> Advent.say_days_to_christmas(1)
"クリスマスまであと1日です!"

Let’s add a few more date calculation functions:

  def today, do: Timex.now("Asia/Tokyo") |> DateTime.to_date()

  def say_date(date), do: "#{date.year}#{date.month}#{date.day}"

  def days_to(date), do: Date.diff(date, today())

  def say_days_to(date), do: say_days_to(date, days_to(date))
  def say_days_to(date, 0), do: "#{say_date(date)}は今日です!"
  def say_days_to(date, days) when days < 0, do: "#{say_date(date)}から#{-days}日すぎました。"
  def say_days_to(date, days), do: "#{say_date(date)}まであと#{days}日です。"

  def days_to_christmas do
    {:ok, xmas} = Date.new(today().year, 12, 25)
    days_to(xmas)
  end

We can test these too. This example was generated on 29 November 2018, so the results reflect that date:

iex(1)> today = Advent.today()
~D[2018-11-29]
iex(2)> Advent.say_date(today)
"2018年11月29日"
iex(3)> Advent.days_to(~D[2019-05-01])
153
iex(4)> Advent.say_days_to(~D[2019-05-01])
"2019年5月1日まではあと153日です。"
iex(5)> Advent.days_to_christmas()
26

Now we have some utility functions, we can use them to redefine our handle_launch function to something more interesting:

  def handle_launch(_req, resp) do
    resp
    |> add_speech("今日は#{say_date(today())}です")
    |> add_speech(days_to_christmas() |> say_days_to_christmas())
  end

Let’s try it in Postman. Restart the server, and post the same LaunchRequest JSON as we posted before. This time we should get JSON including the following section:

[
    {
        "value": "今日は2018年11月29日です",
        "type": "PlainText",
        "lang": "ja"
    },
    {
        "value": "クリスマスまであと26日です!",
        "type": "PlainText",
        "lang": "ja"
    }
]

If you managed to get the Hello World response working with ngrok above, then now you should also be able to get Clova to read this new response out loud, too.

Handle user reply

So far we have only implemented the handle_launch callback, which is called when our skill is launched. After that, all further interactions from the user are called intents, for which we have the handle_intent function.

The first argument to handle_intent is the name of an intent we registered in the Developer Center. We are going to handle two intents.

  1. The different_day intent we defined earlier.
  2. The built-in Clova.YesIntent – for receiving the user’s confirmation.

First, let’s update handle_launch to ask the user for an additional date:

  def handle_launch(_req, resp) do
    resp
    |> add_speech("今日は#{say_date(today())}です")
    |> add_speech(days_to_christmas() |> say_days_to_christmas())
    |> add_speech(prompt_for_another())
  end

  defp prompt_for_another(), do: "違う日まで計算したい場合、日付を言ってください"

I put prompt_for_another() in a separate function, because we will use it again later. Notice that this time we don’t call end_session(), so Clova will wait for the user to say something. Now let’s add the different_day intent hander:

  def handle_intent("different_day", req, resp) do
    with potential_date when not is_nil(potential_date) <- get_slot(req, "date"),
         {:ok, date} <- Date.from_iso8601(potential_date) do
      resp
      |> add_speech("#{say_date(date)}まで計算しますか?")
      |> put_session_attributes(%{"date" => date})
      |> add_reprompt("#{date.month}#{date.day}日まで計算したい場合、「はい」と言ってください。")
      |> add_reprompt(prompt_for_another())
    else
      _ -> add_speech(resp, "それは日付だと思いますがもっとわかりやすく言ってくださいね")
    end
  end

This handler is doing the following:

  1. Extract the date the user said from the request (if the date cannot be extracted, the else clause is executed).
  2. Ask the user to confirm the date.
  3. The add_reprompt is an instruction to remind the user we’re waiting for a response, if they do not reply quickly.
  4. Adds the user’s date to the session data, so we can look it up again later.

You can try to test this by starting the skill and saying a date to Clova. Or you can POST the JSON below to your skill:

{
    "version": "1.0",
    "session": {
        "sessionId": "dummy",
        "sessionAttributes": {},
        "user": {
            "userId": "dummy",
            "accessToken": "dummy"
        },
        "new": true
    },
    "context": {
        "System": {
            "application": {
                "applicationId": "com.example.advent"
            },
            "user": {
                "userId": "dummy",
                "accessToken": "dummy"
            },
            "device": {
                "deviceId": "dummy",
                "display": {
                    "size": "l100",
                    "orientation": "landscape",
                    "dpi": 96,
                    "contentLayer": {
                        "width": 640,
                        "height": 360
                    }
                }
            }
        }
    },
    "request": {
        "type": "IntentRequest",
        "intent": {
            "name": "different_day",
            "slots": {
                "date": {
                    "name": "date",
                    "value": "2019-05-14",
                    "valueType": "DATE"
                }
            }
        }
    }
}

You should get a response like this:

{
    "version": "1.0",
    "sessionAttributes": {
        "date": "2019-05-14"
    },
    "response": {
        "shouldEndSession": false,
        "reprompt": {
            "outputSpeech": {
                "values": [
                    {
                        "value": "5月14日まで計算したい場合、「はい」と言ってください。",
                        "type": "PlainText",
                        "lang": "ja"
                    },
                    {
                        "value": "違う日まで計算したい場合、日付を言ってください",
                        "type": "PlainText",
                        "lang": "ja"
                    }
                ],
                "type": "SpeechList"
            }
        },
        "outputSpeech": {
            "values": {
                "value": "2019年5月14日まで計算しますか?",
                "type": "PlainText",
                "lang": "ja"
            },
            "type": "SimpleSpeech"
        },
        "directives": null,
        "card": null
    }
}

You can see that as well as the text to read to the user, the date has been stored in the sessionAttributes. These session attributes are passed to our server for every request, so we can use them when implementing the Clova.YesIntent handler.

Handling a request using stored session data

Let’s add the handler for when the user says “Yes”. This arrives as the pre-defined “Clova.YesIntent”.

  def handle_intent("Clova.YesIntent", req, resp) do
    with %{"date" => iso_date} <- get_session_attributes(req),
         {:ok, date} <- Date.from_iso8601(iso_date) do
      resp
      |> add_speech(say_days_to(date))
      |> add_speech(prompt_for_another())
    else
      _ -> end_session(resp)
    end
  end

Here, the same as in the different_day handler, we parse the date. Except this time, we look it up from the session attributes. If we couldn’t find it, we just end the session in the else clause.

Now, when the user says “yes”, our server should calculate the number of days between today, and the user’s desired date, then ask Clova to say it.

If you need the JSON to test the HTTP request manually, here it is:

{
    "version": "1.0",
    "session": {
        "sessionId": "dummy",
        "sessionAttributes": {
            "date": "2019-05-14"
        },
        "user": {
            "userId": "dummy",
            "accessToken": "dummy"
        },
        "new": true
    },
    "context": {
        "System": {
            "application": {
                "applicationId": "com.example.advent"
            },
            "user": {
                "userId": "dummy",
                "accessToken": "dummy"
            },
            "device": {
                "deviceId": "dummy",
                "display": {
                    "size": "l100",
                    "orientation": "landscape",
                    "dpi": 96,
                    "contentLayer": {
                        "width": 640,
                        "height": 360
                    }
                }
            }
        }
    },
    "request": {
        "type": "IntentRequest",
        "intent": {
            "name": "Clova.YesIntent",
            "slots": null
        }
    }
}

Ensure the request is genuine

We now have a working Clova skill. We just have to take care of the force_signature_valid: true line we added earlier.

When we put our skill into production, we want to ensure we only reply to genuine requests from the Clova server. But in a local environment, it’s convenient to send our server requests from Postman or another HTTP tool. So let’s set up some config, to only verify the request signature in the production environment.

Add the following line to config/config.exs:

config :advent, force_signature_valid: Mix.env() !== :prod

This sets a configuration value called :force_signature_valid, which is true if the application was not compiled for production.

Then we can use this configuration variable in our lib/advent/router.ex file:

  plug Clova.SkillPlug,
    dispatch_to: Advent,
    app_id: "com.example.advent",
    json_module: Poison,
    force_signature_valid: Application.get_env(:advent, :force_signature_valid, false)

Here, we tell the Clova SDK to force the signature to be valid based on the :force_signature_valid configuration of the :advent application (defaulting to false).

Now we should be able to test our skill locally using Postman, but when we deploy it to a production environment (setting MIX_ENV=prod), only genuine requests from the Clova server will be accepted.

Finished Project

A completed version of this example skill is on Github. Please feel free to check it out.

Conclusion

That was a basic walkthrough of how to set up a Clova skill in Elixir using the :clova Hex package. If you have any questions or comments feel free to contact me on twitter @ExAdamu. As I said earlier, I was learning Elixir myself as I developed this, so any suggestions or feedback would be warmly welcomed! One improvement I would like to make in the future, is to remove the need to setup the basic boilerplate manually by providing a mix script that does this automatically. In the mean time I look forward to seeing some Clova skills developed in Elixir!

Related Post