LINE Engineering
Blog

Using Headless Chrome with chatbots

Kazuki Nakajima 2018.08.07

Hi again, this is Nakajima, LINE's Developer Advocate.

Be smart in using web with chatbots

I am sure you've had at least one or more encounters with the following and ended up with frustration:

  • Entering something in on mobile-unfriendly websites
  • Having to sign in every time you access a webpage
  • Entering something in on smartphones itself

LINE the app is mostly used on smartphone, and this is why we value and put much effort into UX. Having been frustrated myself, I wondered how LINE could ease up the use case I've mentioned earlier and tried out something that has never been done before. Which I'd like to share with you in this post.

Headless Chrome for using web pages with chatbots

Chrome must sound familiar even to those who are unfamiliar with IT, but have you heard of Headless Chrome? Generally we run Chrome in GUI (Graphic User Interface) mode, which means we see the UI of it with our eyes. Contrarily, Headless Chrome does not provide UI but is ran using CLI (Command Line Interface), and thus is used often by test automation which do not require UI. Similarly, chatbots do not require UI to access web pages—a perfect use case for Headless Chrome.

Here is a video clip on a chatbot booking and renting a book from a library website. Although I've stated that Headless Chrome does not come with UI, to help you understand the scenario, the video shows an ordinary web browser used. However, the actual feature uses Chrome in the headless mode.

You can see how the website responds as the conversation with the chatbot progresses. You only have to tell the bot what book to book for, and the bot takes care of the rest of the process. So, let the chatbot handle web interaction or repetitive tasks to lessen the load on your hands. You might achieve a better UX, after all.

Implementation

Now, let's see the implementation part of feature you've seen in the video. The bot itself is made based on Node.js. There is an NPM package called puppeteer, which helps you using Headless Chrome. So, for starters, we will install puppeteer.

Installing

Install puppeteer using the following command.

$ npm install --save puppeteer

Coding

Now we need to code. The first thing we need to is create and run a browser instance. Browser instances take long time and consume a lot of memory when executed. Considering various use cases we may face, it will be better to have a browser instance launched when running a server.

const pptr = require("puppeteer");

(async () => {
  // Creating and running a browser instance
  let browser = await pptr.launch({
      headless: true
  });
})();

Next we create browser context, which will be used to handle user's messages and do something on the web page. This is used to separate individual user's information such as cookies.

// Creating a browser context (Separate space for each user)
let bc = await browser.createIncognitoBrowserContext();

Next we write a page. The following code is for a tab.

// Writing a page (For a tab)
let page = await bc.newPage();

Now the instance of Headless Chrome is ready get a response from a certain URL, or handle forms on a page, if it has any.

// Accessing a certain URL
await page.goto(CERTAIN_URL, {
    waitUntil: "networkidle0"
});

To enter values in an input field, if the page has any, call the `type()` method.

await page.type(CSS Selector, `foo`);

To click a button on a web page, if the page has any, call the `click()` method.

await page.click(CSS selector);

Let's imagine we have a table filled with the following data, and we want to get a list of values in the 'Title' column.

<table>
    <tr>
        <td>Number</td>
        <td>Title</td>
        <td>Author</td>
    </tr>
    <tr>
        <td>1</td>
        <td>Fire</td>
        <td>Matayoshi</td>
    </tr>
        <tr>
        <td>2</td>
        <td>Strategy for revivla</td>
        <td>Ochiai Yoichi</td>
    </tr>
</table>

We first obtain the data in the table, retrieve the values in the 'Title' column and make a list of it.

// Get the rows of the table
let rows = await page.$$(`table > tr`);
let title_list = [];
for (let row of rows){
    let cols = await row.$$(`td`);
    title_list.push(await (await cols[1].getProperty("textContent")).jsonValue());
}
// Check the values stored in the title_list array
console.log(title_list);

Now the titles are stored in the `title_list` array. What's left to do is use Flex Message to process the data to be presentable to users. The overall implementation, letting the bot select what needs to be handled on web from the conversation with a user, executing the task, and providing the result to the user, is basically an interaction between a chatbot and web.

Notes

Lastly, I'd like to share some points we should note.

  • This implementation is dependent on HTML. If the HTML code of a website is changed, we may need to change CSS selectors for buttons or input fields.
  • This implementation consumes a lot of memory. A minimum 20MB may be required per user, and thus is not suitable for the services with massive accesses.
  • Complexity of code for implementation is high because we need to synchronize the context information held by a bot (the context of the conversation so far, and questions to ask the user) with the state of the browser. If the conversation proceeds as expected, then we will be okay. However, handling unexpected uses cases, such as having to go back a few pages to modify the value entered in an input field, may be difficult
  • Because of these seemingly restriction, we cannot recommend such implementation to anyone or apply it to all of our services. But, still, I believe this integration was a nice try out and good to know for possible use cases in the future.

    How about smart speaker?

    This time, I integrated a chatbot and a web page. Integrating smart speakers and websites seems feasible too. Until now, web pages were for "viewing" but from now on, we could add "vocal conversation" as one of its UX. How about trying it yourself?

chatbot summer homework headless chrome

Kazuki Nakajima 2018.08.07

Add this entry to Hatena bookmark

Back to blog list