Grammar wars

A little under a year ago, I got it into my head that I should compete in the Turing test competition held once every year. I don't remember why I thought this was a great idea, only that I had my usual stubborn fire that I was going to compete and win despite having never worked with a chatbot before. Oh how naive I was...

Most chatbots are based on A-B responses, and a lot of them are known as "Alicebots" (known for the software that is used as the backbone for the overwhelming majority of them). When I did a little research and found all this out, I was pretty disappointed since in my head, a chatbot should be a type of AI, capable of discerning your intention from your sentence structure, word choice, and its own data-banks of information. An alicebot or A-B responder will never be capable of answering a question outside of its known responses, which are all pre-scripted to the extreme. As reference for anyone who has no knowledge of these Turing competitions, using the internet to search for things is not allowed. The bot must function independent of any external references.

I foolishly decided that about 2 months before the competition was to take place, I would build my own chatbot from scratch. This decision was based on the fact that I was (and still am) confident that no Alicebot will ever be comprehensive or intelligent enough to solve logic puzzle questions, and that this sort of thing is just my modus operandi.

Step 1. Find a challenge that seems ambitious and only theoretically attainable, typically astronimcally far outside my comfort zone.

Step 2. Throw self at challenge as if life depends on overcoming it, foregoing all social life, hobbies and health considerations in lieu of striving to attain it.

Step 3. Peak on progress and have a near mental-collapse breakdown struggling to overcome whatever insurmountable obstacle I've set for myself.

Step 4. Scale that wall and grasp the glory of victory, or abandon project in a fit of rage and failure.

Needless to say, I did not defeat the Turing test, I did not even complete a chatbot I was proud enough of to even enter into the competition. This was one of those projects I gave up on after getting about 2 months worth of work done on it. Maybe one day I'll revisit it (probably after learning more about artificial intelligence) but for the time being it is collecting dust.

Onward to the meat and potatoes, what I actually built, what it did and the why/how.

The first thing I set up was a basic textbox prompter, that let me use an input field to send and receive a reseponse via JS/ajax. Initial tests for this were A-B responses, hard coded just to confirm that if I typed in a hello, I'd get a hi and a how are you would get a good back. This was beyond basic, but it gave me the UI/UX I needed to start working on the actual logic that would go behind the responses.

For the programming architecture, I decided I needed to focus on grammar first. Most things people type can fall into a few different categories, but the sentences, fragments, asides and run-ons they write all have some (albeit usually incorrect) grammar.

I set my bot up for some rules, like if the sentence ended in a questionmark, it had to be a question. If the sentence ended with a period or ellipsis, it was a sentence or fragment. If the sentence contained commas, semi colons, colons or dashes, it needed to split apart into separate chunks and each one processed separately. I also set up some cheater guildelines/rules, like if a sentence started with specific words like Who or Where, to only deliver responses back that would fit with that criteria, like a person or place.

The second thing I worked on was getting the response grammar legible. Its great and all when your bot responds like an ESL student ("Who are you?" "I are me.") but its not so great for getting your bot to pass a Turing test. I wrote in some grammar and verb tense rules for the replies the bot would spit back, so that if you asked the bot "Who went to the store?" it might respond with something more literate like "I went to the store" instead of "I going to store."

The next hurdle I had to overcome was the bots actual responses sounding quasi-intelligent and vaguely human. This is the hurlde that I never fully surmounted. At first, unaware of the no-internet rule for the competition, I hooked up a basic crawler to the bot so it would run a google search and then spit back something it deemed appropriate from the search results. This was hit or miss, as there are a lot of things that are referred to by similar terminology, colloquialisms or slang, like a "coon's age." The bot couldn't tell if you wanted a literal racoon's age or if you wanted the urban dictionary definition, or what. It was a pretty stupid bot. That said, the google-search results type answers did a great job of responding to questions that bots typically struggle with, like "How many legs does a dog have?" A standard Alicebot would have to be manually supplied with a response for this kind of question (for every animal, scenario, organ etc) and it still might not understand, but more typically would just reply with something like "I like dogs. Random bit of trivial information about dogs or dog legs." which is bizarre and doesn't come accorss as even remotely human, since a 3 year old could answer that kind of question correctly.

After learning about the no-internet rule, rather than continue to bang my head up against the challenges presented by formatting and regurgitating google search results, I turned my attention to word databases. There are quite a few, and as an added bonus to integrating with them, I was able to get the bot to handle irregular verb usages, responding in correct grammar not just with basic common verbs but also with more obscure ones like "dream" vs "dreamt". If you didn't know, most verbs follow a basic pattern of "verb" rendering out to, past tense "verbed" or "had verbed", and active present tense "verbing" or future tense "will verb", but some verbs don't follow this. There are giant lists of them on the internet, but without a word database its pretty difficult to get them all programmed in separately. Once I had a word database though (including the tense and part of speech) the bot had a much easier time responding in something that wasn't broken English, even if the replies were still not sensible at all.

I got to work next on making the bot understand slang, and shorthand. You = u = ya, that sort of thing. It would lowercase cast everything input into it too, so "How are you?" would come out to the same function as "how r u?". I also started building variable based arrays for the responses that came back, so that if you asked it the same question twice, you might get a different answer the second time instead of literally the same response over and over.

It was around this time in my chatbot progress that I got my new (and my current) job, which I love for the record, and became sufficiently distracted with learning the ropes of a new work environment to rage quit this chatbot after one particularly frustrating night trying to hook the wordbank database up with a few other databases (a plant database, an animal database, etc) so that it could relate the correct word from the user's input to the correct information between databases. I also only had about a week or two left before it was time for the competition, and the chatbot still answered most (if not all) inputs with an echo statement, like ("I am going to be so tired tonight." "Why will you be tired?") that was potentially even worse than an A-B response, or a ridiculous cop-out exception handling randomized statement to obscure the fact it didn't know what you were talking about ("Right?! Lolz" or "Ain't that the truth." or "You don't say...").

I had plans to do a lot fancier stuff with this bot once it was made semi-coherent in its replies. I was going to store the interactions it had with users, and then use a basic data-outlier model to try and find where conversations broke down between it and humans and see if I could write in smarter response handling for specific conversation scenarios (for example, what happens when a conversation veers into an extremely specific topic? Does the bot pretend to know a ton of stuff about nearly everything and reply according to whatever is in its databanks, or does it plead ignorance randomly when a topic of certain specificity arises?). I may return to this chatbot one day to make it actually functional, but I'm not convinced it will ever be smart enough to pass a Turing test. But a girl can dream.

I may be bold enough one day to add a link to the chatbot on this page, but right now its too big of an embarassment and failure.