Github-Action on Janik von Rotz

Build a stateful serverless Telegram bot - Part 2

Tue, 21 Jan 2020 11:39:00 +0100

In part 1 of this tutorial we’ve setup the initial folder structure for our chatbot and made a first deployment. In part 2 we are going to teach the bot some new commands and implement a state management for the chat dialog chain.

When thinking about a dialog flow think of a state machine. A state machine is a set of states and transitions. Every action of sending a message, pressing a button or replying to a question in a chatbot dialog is a transition. A state is a set informations that the bot requires to complete his tasks. Here is an example of a state machine:

![state machine example](/images/state machine example.png)

In the tutorial we will implement the /newreminder command which initiates the following dialog with the user:

transition: send /newreminder command
state: ask name
transition: write name
state: ask schedule
transition: set schedule
state: confirm
transition: reset state
transition: send another command

Overview

The tutorial is separated into two parts:

Part 1:

Register a bot
Setup the now project
Add a /start reply
Deploy

Part 2:

Setup redis connector
Add /newreminder reply
Add /listreminders reply
Deploy

Setup redis connector

As mentioned we will use redis as our database. There we store state data for each user.

You can get a free redis instance from RedisLabs.

Once you’ve setup the database you are ready to create a database connector for the example bot.

Enter the db folder and create a new package.

cd db && npm init

Create the index.js file.

db/index.js

var redis = require('redis')
var bluebird = require('bluebird')

// Promisify all redis methods
bluebird.promisifyAll(redis)

// Create cached connection variable
let cachedClient = null

const createClient = (uri) => {
  // If cached client exists return it
  if (cachedClient) {
    return cachedClient
  }

  // Parse url and get pass
  const parsedUrl = new URL(uri)

  // Otherwise create a new client
  var client = redis.createClient(uri)
  client.auth(parsedUrl.username)
  cachedClient = client

  // Return new client
  return client
}

module.exports = createClient(process.env.REDIS_URI)

This module exports a redis client. Naturally you want to reuse an open connection to the database, but due to the serverless nature of now, you do not have access to the lifecycle of an object. Thus we have to create cached object for the client. Assuming the serverless instance went cold and is spun up again, the module creates a new object. If the module is executed during a hot phase, the client object is reused.

In addition to the Telegram token the redis uri must be added to the environment file.

.env

TELEGRAM_TOKEN=<your telegram bot token goes here>
REDIS_URI=redis://<password>@<host>:<port>/0

Export the variables.

export $(cat .env | xargs)

And create a now secret.

now secrets add example_bot_redis_uri $REDIS_URI

Then we add the db package to the root project.

package.json

  ...
  "dependencies": {
    ...
    "db": "./db"
  },
  ...

And install it.

npm install

For storing and retrieving data we need a new module.

api/redis.js

const db = require('db')

module.exports = {
  keys: async (key) => {
    return db.keysAsync(key)
  },
  set: async (key, value) => {
    return db.setAsync(key, value)
  },
  get: async (key) => {
    return db.getAsync(key)
  },
  del: async (key) => {
    db.delAsync(key)
  },
  hkeys: async (hash) => {
    return db.hkeysAsync(hash)
  },
  hset: (hash, field, value) => {
    return db.hset(hash, field, value)
  },
  hget: async (hash, field) => {
    return db.hgetAsync(hash, field)
  },
  hdel: async (hash, field) => {
    return db.hdelAsync(hash, field)
  },
  incr: async (key) => {
    return db.incrAsync(key)
  }
}

This module exports common redis commands.

Add /newreminder reply

It is time to teach our bot a new command. This following module is quite complex. Make sure to read the comments, especially the “Set state” comments.

api/new-reminder.js

const sendMessage = require('./send-message')
const { set, del, get, hset, incr } = require('./redis')
const Reminder = require('./reminder')
const { scheduleOptions, scheduleButtons } = require('./options')

// Processes messages matching /newreminder
module.exports = async (message, ctx) => {
  // Set some ids
  const chatId = message.chat.id
  const reminderEditKey = `edit:${chatId}:reminder`
  const requestId = `request:${chatId}`
  const reminderSetKey = `reminder:${chatId}`

  if (ctx.request === '/newreminder/name') {
    await sendMessage({
      chat_id: chatId,
      text: `Your reminder name is: ${message.text}`
    })

    // Create new reminder and store plant name
    const reminder = new Reminder(message.text)
    await set(reminderEditKey, JSON.stringify(reminder))

    // Ask for the interval
    await sendMessage({
      chat_id: chatId,
      text: 'At which interval would you like to be remindend?',
      reply_markup: {
        inline_keyboard: [scheduleButtons]
      }
    })

    // Set state
    await set(requestId, '/newreminder/schedule')
  }

  if (ctx.request === '/newreminder') {
    // Send message
    await sendMessage({
      chat_id: chatId,
      text: 'Name your plant:'
    })

    // Set state
    await set(requestId, '/newreminder/name')
  }

  if (ctx.request === '/newreminder/schedule') {
    await sendMessage({
      chat_id: chatId,
      text: `You have chosen: ${scheduleOptions[message.data].display}`
    })

    // Get reminder edit
    var reminder = Object.setPrototypeOf(JSON.parse(await get(reminderEditKey)), Reminder.prototype)

    // Set schedule
    reminder.setSchedule(message.data)

    // Generate Id
    const reminderKey = await incr('key:reminder')

    // Save reminder
    await hset(reminderSetKey, reminderKey, JSON.stringify(reminder))

    await sendMessage({
      chat_id: chatId,
      text: 'Your reminder has been saved!'
    })

    // Del reminder edit
    await del(reminderEditKey)

    // Delete request state
    await del(requestId)
  }
}

In this module we have implemented the dialog flow described in the intro. Assuming the user enters the command /newreminder, a message is forwarded to this module. A transition is matched according to the request state. Every transitions sets the expected state in the database. The final transition deletes the request state.

The inline keyboard of the Telgram API expects some heavy markup. Lets externalize the data into a separated module.

api/options.js

const scheduleOptions = {
  daily: {
    display: 'Daily',
    days: 1
  },
  weekly: {
    display: 'Weekly',
    days: 7
  },
  twoweeks: {
    display: 'Every 2 weeks',
    days: 14
  },
  monthly: {
    display: 'Monthly',
    days: 30
  }
}

const scheduleButtons = ['daily', 'weekly', 'twoweeks', 'monthly'].map(key => {
  return {
    text: scheduleOptions[key].display,
    callback_data: key
  }
})

module.exports = {
  scheduleOptions,
  scheduleButtons
}

Add /listreminders reply

We are getting close to finish the second draft of our bot. The user can create reminders, but sure he wants to have an overview.

api/list-reminders.js

const sendMessage = require('./send-message')
const { hkeys, hget } = require('./redis')
const Reminder = require('./reminder')
const { scheduleOptions } = require('./options')

// Processes messages matching /listreminders
module.exports = async (message) => {
  const chatId = message.chat.id
  const reminderSetKey = `reminder:${chatId}`

  // Get all existing reminders
  const reminderKeys = await hkeys(reminderSetKey)

  if (reminderKeys.length >= 1) {
    // Create printable list for reminder set
    const reminderPrint = await Promise.all(reminderKeys.map(async key => {
      // Get details of reminder
      const reminder = Object.setPrototypeOf(JSON.parse(await hget(reminderSetKey, key)), Reminder.prototype)

      // Return printable list entry
      return `\n${reminderKeys.indexOf(key) + 1}) ${reminder.getName()} (${scheduleOptions[reminder.getSchedule()].display})`
    }))

    await sendMessage({
      chat_id: chatId,
      text: `Here are your registered reminders:${reminderPrint}`
    })
  } else {
    await sendMessage({
      chat_id: chatId,
      text: 'You do not have any reminder yet. Use /newreminder to add one.'
    })
  }
}

This command handler simply prints all registered reminders.

For reminder object interaction we use a JavaScript class (aka just another prototype). Here is the class definition:

api/reminder.js

class Reminder {
  constructor (name) {
    this.name = name
  }

  setName (value) {
    this.name = value
  }

  getName () {
    return this.name
  }

  setSchedule (value) {
    this.schedule = value
  }

  getSchedule () {
    return this.schedule
  }
}

module.exports = Reminder

Now we connect our command modules with the message processor.

api/index.js

// Load env config
require('dotenv').config()

// Import modules
const start = require('./start')
const newreminder = require('./new-reminder')
const listreminders = require('./list-reminders')
const { get } = require('./redis')
const sendMessage = require('./send-message')

module.exports = async (req, res) => {
  console.log('BODY', req.body)

  // Check if Telegram message
  if (req.body && (req.body.message || req.body.callback_query)) {
    // Get message object
    var message = {}
    if (req.body.message) {
      message = req.body.message
    }
    // Callback query depends on request
    if (req.body.callback_query) {
      message = req.body.callback_query.message
      message.data = req.body.callback_query.data

      // Answer callback query
      sendMessage({
        callback_query_id: req.body.callback_query.id
      }, 'answerCallbackQuery')
    }

    // Build context
    const ctx = {
      request: await get(`request:${message.chat.id}`)
    }

    // Request is either current state if set or message text
    ctx.request = ctx.request || message.text

    // Match text request
    if (ctx.request.match('/start(.*)')) {
      await start(message, ctx)
    }

    if (ctx.request.match('/newreminder(.*)')) {
      await newreminder(message, ctx)
    }

    if (ctx.request.match('/listreminders(.*)')) {
      await listreminders(message, ctx)
    }
  }

  // Send default message
  res.end('This is the ExampleBot API.')
}

If a request state is already set, the bot will dismiss the message text as request path. Further the handler for our new commands have been added.

From here you should see where this is going. You can now easily add new commands and dialogs. Simply create a new command module and register the handler in the index file. Use the redis commands to manage the state of your dialog.

Deploy

Before we deploy the example ensure that the new environment variable is propagated to the deployment.

now.json

{
    "env": {
      "TELEGRAM_TOKEN": "@bunkr_bot_telegram_token",
      "REDIS_URI": "@bunkr_bot_redis_uri"
    }
  }

If done run the now command.

now --prod

If the deployment was successful open Telegram and start chatting using the new commands 😁.

Now where to go from here? If you want to build a useful bot sure we have to teach him a lot more. Here are some ideas:

Checkout my bot code: GitHub - HydroMeBot
Add additional commands (modify, delete)
Write tests for the modules
Setup a reminder notificator that runs daily
Integrate a natural language processor

Build a stateful serverless Telegram bot - Part 1

Sat, 18 Jan 2020 13:55:31 +0100

For the first time in my life I bought a plant to decor my room. The plant is a Monstera, growing up to 2 meters if watered properly. It actually needs to be watered every two weeks. Now there are various options to remind myself of watering it. Installing an app, set an alarm on my phone or place a paper calendar next to the plant. But this is too easy, lets make it more complicated.

More and more people start using Telegram. Telegram is a secure alternative to Whatsapp and most importantly it supports chatbots! For a while I’ve planed to build a bot. And by having the need to build a reminder I came up with the following idea: I am going to create a bot that manages the reminders for watering my plants. It will be hosted on the Now platform and I will use GitHub Actions to schedule the reminders. The reminders are stored in a redis database and everything is MANAGED via the Telegram bot interface.

If you are interested on how I built this bot, keep on reading 😊.

If you wanna go straight for the code checkout GitHub - HydroMeBot.

Overview

In this tutorial I will proceed as followed:

Part 1:

Register a bot
Setup the now project
Add a /start reply
Deploy

Part 2:

Setup redis connector
Add /newreminder reply
Add /listreminders reply
Deploy

Assuming you want to build a similar bot, follow these instructions.

Register a bot

On Telegram contact the @BotFather. This bot manages all bots on Telegram. Add a new bot with these details:

Name: ExampleBot
Description: Reminds you of watering your plants.
About: Reminds you of watering your plants.
Botpic: 🖼 has a botpic
Commands: 3 commands

Chose the bot username as you wish. In return you receive a Telegram bot token. We need it later on.

Add these commands to the bot:

start - Say hello
newreminder - Create a new reminder
listreminders - List all reminders

Note: In this tutorial I will not cover cases of mutating and deleting the reminders. They are implemented in the GitHub project. Once you finished part 2 it is easy to add new features to the bot.

Setup the now project

Let’s setup a new node project. We assume now is already installed and you are logged in with your ZEIT account.

Create a new project folder.

mkdir example-bot && cd example-bot

Create two subfolder.

mkdir api db

Note that now treats every file under /api as a serverless function.

Init npm and git.

npm init && git init

Create config files.

touch .env .gitignore now.json

Add packages with npm.

npm install --save bluebird dotenv node-fetch redis

Define the environment variables for the now deployment.

now.json

{
  "env": {
    "TELEGRAM_TOKEN": "@example_bot_telegram_token"
  }
}

Ignore the module folder and environment file for git.

printf ".env\nnode_modules" >> .gitignore

Update the environment file with the Telegram token.

.env

TELEGRAM_TOKEN=<your telegram bot token goes here>

Export the variable and create a now secret.

export $(cat .env | xargs)
now secrets add example_bot_telegram_token $TELEGRAM_TOKEN

That’s it for now 😉

Optional tools

Optionally add a linter and test tool.

npm install --save-dev ava standard

I personally love AVA and StandardJS.

These packages require npm scripts.

package.json

  ...
  "scripts": {
    "test": "ava",
    "lint": "standard"
  }
}

Add a /start reply

We wanna teach our bot to reply to the opt-in command /start.

Add and index js file to the api folder with the following content.

api/index.js

// Load env config
require('dotenv').config()

// Import modules
const start = require('./start')
const sendMessage = require('./send-message')

module.exports = async (req, res) => {
  console.log('BODY', req.body)

  // Check if Telegram message
  if (req.body && (req.body.message || req.body.callback_query)) {
    // Get message object
    var message = {}
    if (req.body.message) {
      message = req.body.message
    }
    // Callback query depends on request
    if (req.body.callback_query) {
      message = req.body.callback_query.message
      message.data = req.body.callback_query.data

      // Answer callback query
      sendMessage({
        callback_query_id: req.body.callback_query.id
      }, 'answerCallbackQuery')
    }

    // Build context
    const ctx = {
      request: null // await get(`request:${message.chat.id}`) -> See part 2
    }

    // Request is either current state if set or message text
    ctx.request = ctx.request || message.text

    // Match text request
    if (ctx.request.match('/start(.*)')) {
      await start(message, ctx)
    }
  }

  // Send default message
  res.end('This is the ExampleBot API.')
}

Every bot command has its own file. The API endpoint receives all bot messages and then forwards the message to appropriate command module.

Note: Callback queries are answered with an empty message. Callback queries are sent when the users presses a button on the inline keyboard in the bot chat. We will handle the dialog state differently therefore we do not need to answer callback queries.

Let’s checkout the sendMessage method.

api/send-message.js

const fetch = require('node-fetch')

// Sends messages to Telegram API
module.exports = async (body, method) => {
  const token = process.env.TELEGRAM_TOKEN

  // Get method optionally
  const useMethod = method || 'sendMessage'

  // Options for send message method
  if (useMethod === 'sendMessage') {
    // Disable notification
    body.disable_notification = !body.enable_notifications
  }

  return fetch(`https://api.telegram.org/bot${token}/${useMethod}`, {
    method: 'POST',
    headers: { 'Content-Type': 'application/json' },
    body: JSON.stringify(body)
  })
}

This module expects a telegram message body and the sends it to Telegram API. The API method can be overwritten by the optional parameter.

I assume this was easy to understand. The final file is the start module. As mentioned it simply replies to the /start command.

start.js

const sendMessage = require('./send-message')

// Processes messages matching /start
module.exports = async (message) => {
  const chatId = message.chat.id

  await sendMessage({
    chat_id: chatId,
    text: '🙋 Hi, I am ExampleBot. Let me know if I should remind you of something.'
  })
}

Deploy

Now we can deploy our bot with now. Run now --prod in the root folder.

Once the deployment and building has finished you receive url like https://example-bot.now.sh.

As final step we need to set the webhook url. Once more export the Telegram token with export $(cat .env | xargs) and run this curl command: curl https://api.telegram.org/bot$TELEGRAM_TOKEN/setWebhook?url=<your deployment url goes here>/api

If the response was successful, open Telegram and start chatting with your bot.

Hope everything worked! See you in part 2 🏃

Github-Action on Janik von Rotz

Build a stateful serverless Telegram bot - Part 2

Overview

Setup redis connector

Add /newreminder reply

Add /listreminders reply

Deploy

Next

Build a stateful serverless Telegram bot - Part 1

Overview

Register a bot

Setup the now project

Optional tools

Add a /start reply

Deploy