Voice Forth

Forth Day! - November 17, 2018

Brad Nelson / @flagxor

Introduction

  • Voice Assistants
  • Actions on Google
  • Gforth on AoG

Voice Assistants

  • Siri - Apple
  • Alexa - Amazon
  • Cortana - Microsoft
  • Bixby - Samsung
  • Google Assistant - Google

Siri

Apple

Homepod

Apple

Amazon Echo

Alexa

AmazonBasics Microwave

Alexa

Cortana

Microsoft

Harman Kardon Invoke

Cortana - Microsoft

Google Assistant

Google Home

JioPhone

Google Assistant

Google Home Mini

Voice Assistant Features

  • Always On
  • Large Domain
  • 3rd-party Extensions
  • Multiple Devices
  • Home Automation

Speech Recognition

  • Triggering
  • Speech in the cloud
  • Biasing

Actions on Google

  • 3rd Party "Actions"
  • Dialogflow
  • Actions SDK

Terminology

  • Invocation - Phrase to trigger an "app"
  • Intent - A structured encoding of user intent
  • Fulfillment - How to handle a user intent

actions.json

{
  "actions": [
    {
      "description": "Default Welcome Intent",
      "name": "MAIN",
      "fulfillment": {
        "conversationName": "Voice Forth"
      },

actions.json

      "intent": {
        "name": "actions.intent.MAIN",
        "parameters": [{
          "name": "command",
          "type": "org.schema.type.Text"
        }],
        "trigger": {
          "queryPatterns": [
            "talk to Voice Forth",
            "ask Voice Forth $org.schema.type.Text:command",
            "forth $org.schema.type.Text:command"
          ]
        }
      }

actions.json

  "conversations": {
    "Voice Forth": {
      "name": "Voice Forth",
      "url": "https://voice.flagxor.com/voiceforth"
    }
  },
  "locale": "en"
}

Voice Forth Logo!

: square dup * ;
: dist square swap square + sqrt ;
: shape
   x .5 - y .5 - dist .4
   x .3 - y .7 - atan2 9 * sin .07 * +
   swap - 30 * ;
: shadow  1 swap - shape 0.9 - 10 * 0 max 1 min * ;
: layer
   push push
   x .3 - y .5 - dist sin pop * sin
   x .7 - y .5 - dist sin pop * sin * shadow ;
23 23 layer   23 22 layer   22 23 layer   shape

Voice Forth Logo!

Request

  • JSON
  • Capabilities of the device
  • Parsed input

Response

  • JSON
  • Text
  • TTS
  • Cards
  • Suggestions

Fulfillment

  • Provide an HTTPS URL
  • Accepts Requests, yields Responses

Requirements

  • HTTPS
  • Always Available
  • Can Run Gforth

Google Firebase

  • Actions on Google Integration
  • Cloud Functions
  • Integrated database
  • Free with small traffic
  • Can't run Gforth

Google AppEngine

  • Multi-language scalable services
  • Free with small traffic
  • Can't run Gforth

Google Cloud VM

  • Run a Linux instance in the cloud
  • Free year long trial
  • Free tier with low powered system

HTTPS

  • Requires a certificate
  • Let's Encrypt

Serving

  • Apache2
  • Node.js

Login

  • Password
  • User Content
  • But can't share :-(

Slide Hookup

  • Voice Forth remembers last input / output
  • Back channel to receive commands + i/o

Long Polling

  • Hold open HTTP connection server side
  • ~30 second timeout
  • Retry

Gforth Challenges

  • Assistant strongly assumes "proper" English
  • Forth core words are odd
  • Numbers are sometimes words

Public WIFI

  • Google Home is set up over WIFI
  • dd-wrt in "Repeater" mode
  • figgy → Stanford Visitor

Demo

Possibilities & Learnings

  • Forth's DSL flexibility is brittle
  • More grammatical Forth?
  • Conversational program refinement?
  • How to make it generally shareable?

source and slides at:
github.com/flagxor

⚘

Thank you