are there better AI algorithms for offline [non spam] chatbots?

[blue21]

hello.

I would like to ask if anyone knows or can give simple links or code examples for better algorithms for chatbots in freebasic -

all my chatbots use more or less the same simple algorithm that is: if the input contains these keyword(s) replay a random replay from these replies.

that's basically it... I heard some programmers telling me "well that's the best you can do with basic languages" and offer me to switch to python...

[srvaldez]

I hope nobody helps you, we get enough spam bots on this forum, we don't need any more

[blue21]

hello...

i'm looking at old chatbots i used to collect during the mid-2000's and I found this interesting text file explaining how "ECC-ELIZA V 4.09" works

thought it would be interesting to share and maybe try to make sense or re-code it in freebasic :

Code: Select all

HOW-WRKS.TXT - Explains how ECC-Eliza V4.08 works and functions.

ECC-Eliza Homepage: http://ecceliza.cjb.net

Introduction and explanations
ÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄ
This file is useful if you are interested in more information on how the
program works and how the data file works.

The data file (ELIZA.DAT) contains all of the things Eliza may say,
as well as all the keywords Eliza could recognize, and finally, controls
the program's entire behavior.

Since version 4.08, all of what Eliza could say, starting from the request
for the user to input his/her name, and all the way through the most
sophisticated mid-conversational replies, is contained within the data file.

The program file, ECCELIZA.EXE, could be defined as the "chassis" of the
program. It collects data from the data file onto the memory, and knows
how to manuever throughout the data file, in order to do what the program
actually does: recognize patterns in the user's input, and answer in
accordance.

The data file's size is allowed to be extremly small, as long as it contains
a category named Klast and all of the "fixed categories", which will be
explained later on this document.
The size of the data file is limited to around 360KB, depending on the
memory state of the specific computer, and the contents of the data file.


Basic Data file structure
ÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄ
The data file has a special design. But in order to explain to you it's
structure I will first explain it to you in general, and how the main
executable (ECCELIZA.EXE) uses it.
Once you type in a pretty much normal reply (to Eliza), Eliza
will check the data file (which it has copied into memory).
The data file is divided into categories. Each category has keyword(s) and
replies. Keywords are text patterns to be found in the user's reply, or
other predefined patterns which the executable seeks in some cases.
A keyword can be longer than one word (several words), but not longer than
40 characters. ECC-Eliza searches the different categories (by their
order). In each category Eliza checks if ANY of the keywords are found
in the user's text, for example: MONEY is found inside I WANT MONEY, but
not found inside: I WANTMONEY. Next, Eliza will check the replies of the
specific category (the one it found a matching pattern in), and will
write one of the replies (chosen mostly randomly). There needs to be at
least one reply and at least one keyword in a category. If a category
has several replies, and is being called more than once, Eliza will make
sure to say all the possible replies in the category, before repeating a
previous reply. If all replies in that category have been used, Eliza will
still make sure to use all replies 2 times, before saying one of them
three times. In case no keyword matches the user's text, there is a category
named KLast which holds different replies for when no match has been found.
This is the general idea, although there are many exceptions to this.


Parameters
ÄÄÄÄÄÄÄÄÄÄ
Now that you've understood the basic structure of the data file, let's talk
about special parameters for the data file. A parameter is a special variable
which substitutes an expression (in the ECC-Eliza data file terms).
You might have noticed that Eliza calls you by name many times.
In the data file, a reply might be: |, HOW ARE YOU? Notice the "|" sign.
This sign will be substituted with the user's name, which Eliza asks for when
you start the program. Whenever the program find the "|" character it will
substitute it with the user's name.
Another important parameter is "\". The backslash (\) sign is used to
indicate the keyword used by Eliza. Whenever a backslash is found Eliza will
substitute it with the keyword (pattern) it has found in the user's text
and inside the data file. If there is only one keyword, it is not necessary.
But many times a category contains multiple keywords, and in that case you
might one to call up the keywords which the user has written.
An additional parameter is the exponent "^" character. The "^" character
is substituted with the time of day in words that Eliza encounters.
If it's a special day it might be the name of it (for example: BOXING DAY).
It may also be the time of day (e.g. AFTERNOON), day of week (e.g. MONDAY),
or both the time of day and weekday (e.g. MONDAY AFTERNOON).
The next parameter is "@". This parameter is substituted by the name of
a discussion subject, chose randomly. Discussion subjects are defined using
the D field. For more information, view the section on discussion subjects.
Please note that if no discussion subjects are defined, this parameter does
not return anything (appears as blank). Similarly, you could use the theta
(~) character to call up the name of a previously discussed subject, however
this character will only be substituted with a name of such subject in the
specially predefined K_SUBJ fixed category (this will also be explain later),
if you use this parameter in another category, it will simply display as is.
Another relatively new parameter is the "È" parameter (ASCII character 200),
this character is simply substituted by ECC-Eliza's version number, and is
valid for versions 4.08 and later.
There are two other more important parameters. A reply consisting of any
of them is called a variable reply. The two signs are asterisk "*", and
underscore "_". Please note that ONLY 1 of them could be used in a reply,
and it cannot appear more than once in that reply (unlike the above
parameters). The underscore (_) character is a simplification of the asterisk
character (*). The asterisk character returns the text found after the
matched keyword and before a punctuation mark or breaking word.
For instance, if you typed: I AM VERY HAPPY, and the keyword found is I AM,
and the reply in the data file is WHY ARE YOU *?, Eliza will say:
WHY ARE YOU VERY HAPPY. Eliza substituted * with VERY HAPPY, because VERY
HAPPY was found after the identified keyword I AM. Another example for the
above data file example: BECAUSE I AM SAD, RIGHT?, would cause Eliza to
reply: WHY ARE YOU SAD?, Eliza will substitute * only for SAD, because there
is a punctuation mark after it which signals another part of the sentence.
Some words (breaking words) are treated just like a punctuation mark, such
words indicate another part of speech, such as: IF, AND, IN ADDITION, etc.
The asterisk character DOES NOT return what has been found BEFORE the keyword,
only what was found after it.
The underscore character (_) returns ONLY the first word which is found
after the identified keyword. Actually it returns the first word of *,
or anything found until a punctuation or a space. It is useful at times.
Another parameter which is rather a character is %.
When % appears as the reply (with nothing in front of it, and nothing
before it) it means that Eliza will query the user if he/she wants to quit.
This should only be used mainly on the quitting category. In fact, it is
not the typical parameter, but rather a program instruction.
The underscore (_), asterisk (*), and backslash (\) are going through a
pronouns reversing routine, read below for more information.


Pronouns reversing routine
ÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄ
Variable replies (consisting of underscore or asterisk) are supposed to contain
the user's text. This is done in order to make Eliza feel more personalized.
If you say: YOU ARE A MAD COMPUTER, and it replies to you: I AM NOT A MAD
COMPUTER, it gives you the feeling as if it understands you (, hopefully).
But remember that Eliza has a different perspective than you. If it repeats
you like a parrot it is meaningless. For example, you say I AM NOT YOUR
PATIENT, and she replies WHY ARE YOU NOT YOUR PATIENT? (for WHY ARE YOU *?),
it sounds bad. And so, for the underscore (_) and asterisk (*) parameters
Eliza reverses the pronouns. That is, every time it finds "YOUR" it changes
it to "MY" and vice versa. It applies to 25 other occasions, such as YOU'VE
to I'VE, YOU ARE to I AM, etc. ECC-Eliza does it pretty good. The only
problem happens with "YOU". "YOU" can be "reversed" to either I or ME.
You have to know English much better than a computer in order to do it
correctly every time. Usually Eliza gets it right, but not always.
Other than that, ECC-Eliza should handle all other cases flawlessly.
Please note that for the backslash (\) parameter Eliza ALSO reverses the
pronouns. Although it is not helpful if you want to quote the user, you
will find it useful many times when the keyword contains a pronoun such as
I, YOU, MINE, etc.


Example category code
ÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄ
Here is an example for a general category. It will be activated if it
find the pattern "YOU ARE" in its two forms. It will then answer
you with one of two possible replies: a variable reply and a fixed reply.

# this line is used to notate this keyword-reply block:
K YOU ARE
K YOU'RE
R CAN YOU PLEASE TALK ABOUT YOURSELF?
R I SURE AM *
#

That's it. The lines starting with the "K" parameter indicate that the
text found after it (to the right of it) is one of the keywords for this
category. In this case we have two keywords: YOU ARE and YOU'RE.
Eliza doesn't recognize them as the same thing, so you have to write them
twice if you want them to active the same category.
The lines with the R parameter are possible replies for this category.
The first one will yield: "CAN YOU PLEASE...", which is a fixed reply.
The second one will reply with an asterisk. If you type "YOU ARE GREAT",
and Eliza chooses to activate the second reply, it will reply: "I SURE AM 
GREAT". The "#" sign indicates a note. Eliza totally ignores anything
found on its line. You can use it to type remarks for yourself, or seperate
different categories so that it's easier to view them.
In the data file, the categories are seperated by this parameter, to allow
easier reading of it (for people).


Quitting Methods
ÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄ
ECC-Eliza's data file may be used for telling Eliza when to quit.
Besides the % instruction, there are two other ways to do it which are
much more user definable.
The first method allows you to cause Eliza to say something and query the
user if he/she wants to quit the program, restart with a new patient or
continue on with the conversation.
Doing this is very easy, just add a colon (:) to the R (reply) field after
the space and before Eliza's output. If and when Eliza will say that reply,
the colon will be omitted, and Eliza will prompt the user to Quit, Restart
with a new patient, or Continue with the conversation.
For example, if you want Eliza to query the user if he/she wants to quit
everytime the word EXIT is encountered use:

K EXIT
R :DO YOU REALLY WANT TO QUIT?
R :ALRIGHT, YOU CAN ACKNOWLEDGE YOUR EXIT

This category will cause Eliza to answer with one of the two different replies
and to prompt the user to quit afterwards. If the user decides to quit or
restart with a new patient, Eliza will not say anything afterwards at that
conversation.

The second method is very similar, except that it does not leave the user
the option to "continue" with the conversation. Use this method if you want
Eliza to end the conversation (for any purpose).
The only difference is that you should use the semicolon (;) character instead
of the colon character. Everything will be identical but for the Continue
option, which will not be available.


Locked Keywords
ÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄ
This is a new addition to Eliza version 4.00 and later.
Locked keywords, unlike normal keywords may activate their category only
if the user's input is exactly the keyword.
For example, if a locked keyword is "BYE" then "GOODBYE" or "BYE BYE" will
not be matched to the keyword because they are not exactly the keyword.
Only "BYE" will be matched to the keyword.
However, if the user's input contains a non-alphabet character (and not a
number), such as a punctuation mark or a space it will be ignored.
So that in the above case, "   ...BYE!!!" will also be matched to the locked
keyword "BYE".
Defining a keyword as a locked keyword is very simple. Simply add ampersand
at the end of the keyword. A locked yes, will simply be defined as:
K YES&


Priority
ÄÄÄÄÄÄÄÄ
ECC-Eliza scans the categories by order. Once Eliza recognizes a keyword
pattern inside the user's text it immediately looks for the reply in the
matching category, regardless if there are other matching keywords in the
data file. This is the reason why different categories have different
priorities. The category which Eliza scans first has the highest priority,
the 2nd category has the 2nd highest priority and so on. KLast has the
lowest priority. The categories included in the data file are sorted by
the priority we found best. You may play around with it a little.
CATEGORY.TXT file lists the different categories found by their priority
(order in the data file). The highest category in the data file (uppermost)
has the highest priority.
When you configure the categories make sure it is sorted the way you want.


Priority mismatches
ÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄ
If a category is listed below another category (has less priority), and
contains a sub-keyword of another, it is considered a stuck category.
For example, one category has the keyword: YOU. The other category has
lower priority (comes after it), and has the keyword YOU ARE.
The second category (YOU ARE) is stuck. This is due to the fact that if
YOU is found in the user's text the first category will be used.
YOU ARE is a sub-keyword of YOU because every sentence that contains
YOU ARE contains YOU, but not every sentence that contains YOU also
contains YOU ARE.
Make sure not to have such mismatches, or else what you did is useless.


Subject and Missing asterisk field
ÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄ
The subject field is the name of the subject which the category handles.
It is optional. It is used when Eliza asks the user to speak about a previous
subject (using one of the fixed categories).
Eliza inserts the subject's name into a special reply for that matter.
The subject's name cannot exceed 25 characters (the next characters will be
ignored). Make sure the subject's name appears in plural (FAMILIES), or is a
general word (PEOPLE), or is in verb form (BEING HATED, YOUR LOVE TOWARDS ME).
The subject field starts with the S character on the first character of a
line and appears between the last keyword of a category (K) and the first
reply of a category (R). As said, it is only optional.
The second new field is the M field which indicates missing asterisk.
Sometimes the value of asterisk is nil (does not exist). For example,
if the keyword is I AM, and the reply is I WAS *, an input such as I AM!
would yield I WAS, this does not sound correct. In such a case you could
insert the M field anywhere between the last keyword and the first reply
of a category. The word or words coming after M are limited to 25 characters
and indicate the value of asterisk if it was supposed to be nil.
For the above example, M WHAT YOU ARE, would result in I WAS WHAT YOU ARE.
The value of missing asterisk is NOT going through a pronouns reversing
process, and appears "as is".


Discussion field
ÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄ
The recent addition to ECC-Eliza V4.00 is the inclusion of a new field which
allows Eliza to react better in certain cases, and this is the discussion
field. The discussion field allows the user to indicate different discussion
subjects, which Eliza may recall using a special predefined fixed category.
Out of all the discussion subjects, when Eliza realizes a state where the
user is being too passive, or just wants to renew the subject, it would choose
one of the predefined subjects randomally, and quote the user to speak about
it, or mensions it in another manner.
The D field indicates such discussion subject. This field may come ANYWHERE
inside the data file. During execution ECC-Eliza "collects" all the characters
found after the D field (limited to 32), and stores them in an array of
possible discussion subjects. Later when it realizes a need to "refresh" the
conversation, it will randomally choose one of these discussion subjects
(if available, i.e. there is at least one such declared subject).
Then Eliza incorporate the subject's name into one of many special replies
for that matter, found inside a special fixed category.
To declare a discussion subject type the D field followed by a space and a
subject name (up to 32 characters), such as:

D YOUR FAMILY

"YOUR FAMILY" will be memorized as a possible discussion subject.
The discussion subjects that appear after the D field, do NOT go through
a pronouns reversing routine (they will be used "as is").
In order for Eliza not to ask the user to speak about a discussion subject
too many times, it is recommended to have many discussion subjects.
Again, the D field may be used anywhere inside the data file.

Up to 128 D fields will be memorized by the program, in case of an overflow
the program will warn you at the initial loading part.

Please note that all the possible discussion subjects should be subjects that
Eliza is familiar with (i.e. they have categories), so that Eliza could really
discuss those categories and not only tell the user to talk about them without
actually being able to reply with "understanding" answers.


Probabily field
ÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄ
One of ECC-Eliza's main highlights is it's comprehensive use of probability
to generate unique replies and conversations every time, even for the exact
same inputs. Although most of this work is done internally and without the
user's interference, it is now also possible to control the probability of
Eliza choosing the different keywords. Normally, if you don't specify the
probability field, whenever Eliza locates a keyword that has been encountered
in the user's input one of it's category replies will be generated.
However, please take in mind that this will not happen 100% of the time.
Eliza has many internal creterias for this, if for some reason Eliza thinks
this category should not be activated now, it will be skipped.
You can do nothing about it, but of course this is only done when Eliza is
sure her choice of skipping will produce a better answer.
However, the probability field allows you to decide on a certain probability
for the keywords of a category to be activated if one of them have been
encountered. The field starts with the letter C (indicates chance), and is
followed by a whole number between 0 and 100. For example: C 0, C 21, C 100.
The number found after C indicates the chance in precentage for Eliza to
execute one of the replies if the keywords AFTER the C field have been found.
C 100 means that 100% of the times Eliza will choose this category if the
keyword has been found unless it internally chooses not to for it's own
reasons (as explained above). Contrary, C 0 means the category will not be
analyzed if any of the keywords found after the C field have been found,
and of course if they were not found.
However, you must note one thing, if Eliza has skipped a category and as a
result found no other matching category for the input (i.e. KLast is used),
then it will reprocess the input until it has settled on a category other
than KLast.

To illustrate this, look at this example:

K YES
C 90
K YEAH
K YEP
R YOU SAID \

In this case, all of the keywords found after the C field have a 90% chance
of being executed if they were encountered in the user's text (and no other
category has higher priority). This means that if YEAH or YEP were found
then 90% of the time Eliza would reply YOU SAID YEAH or YOU SAID YEP.
However, the C field does not refer to the keywords above it, so the keyword
YES will always execute this category if it is found (it's chance is 100%).

Any number between 0 and 100 allows you to tweak the chance for Eliza to
activate a category supposing the keywords found after C have been found.

Please note that more than one C field may be used in a category, actually
they are unlimited, look at this example:

C 90
K A
C 40
K B
K C
C 0
K D
C 100
K E
K F

In this example, if A is found there is a 90% chance this category will be
activated. If B or C are found there is a 40% chance this category will be
executed. If D is found, it will never execute this category (0%), unless it
runs into KLast. If E or F are found this category will be executed (100%).

This field is useful, for example, if the category consists of uncertain
keywords such as "yes". If you think there may be a better keyword later on
in the program, a more specific keyword, you may activate the chance on yes,
to C 30, for example. This way, the program will skip "yes" 70% of the times
to find a better keyword, but 30% of the times it will not skip it, this
30% may give the user the feeling that Eliza sometimes only listens to his
final verdict (i.e. yes) without the reasoning.

There is no need to add this field to all categories because even if
you do not add it there is still a chance that Eliza will skip this category,
so it is certainly okay if you do not add this field to all categories.
Feel free to play with this setting as much as you like.
Please note that not specifying this field is equivalent to specifying C 100.

Please remember that if a chance parameter has caused Eliza to skip one or
more categories, and thus refer to KLast, Eliza will reprocess the input
90% of the times (i.e. 10% of the times - it will still refer to Klast).

In Eliza's data file (ELIZA.DAT) this field, if used, is placed on top of the
category to affect all the keywords of the specifyed category.


Fixed Categories
ÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄ
This option was first added to version 4.08.
Fixed categories are categories that have only one keyword, that is
specifically predefined within the main program file.
For example, when you begin a session with Eliza, it always asks for your
name using one of plenty possible messages. These messages are found inside
the data file under a fixed category with the keyword "K_NAME".
At the beginning of every session, the program file automatically seeks
for the category with the K_NAME keyword, and chooses one of the replies
for that category. That reply will serve as a message for entering the
user's name. This keyword, as well as the keywords for all other fixed
categories, contain the underscore (_) character. The reason for this is
that the user cannot input that character throughout the conversation,
and thus the program cannot accidently detect this keyword when it scans
the users text. Imagine the user replying: WHAT IS K_NAME, and Eliza replying
with: "ENTER YOUR NAME PLEASE:". The underscore character promises this
possibility does not exist.

There are currently 13 predefined fixed categories, as follows:

K_NAME - as explained about. Contains queries which Eliza uses to ask
         the user for his/her name.
K_HELLO - Contains the welcome messages which Eliza uses to greet all of
          her patients.
K_BLANK - This category is called if the program detects a blank reply
          from the user (not writing anything).
K_REPEAT - This category may be called by the program if it detects a
           repetition in the user's input, twice in a row.
K_IMPRP - This category is called if the program detects an improper input.
          This could be any reply which Eliza finds to be imporper, and
          usually replies such as: "!?!?!", "BLAHHHHHH", "ASFSDFFGD", etc.
K_DISC - This category is called when Eliza decides to ask the user to
         raise one of the predefined discussion subjects. For this, the
         program uses the @ parameter, which is converted with a name of
         a discussion subject (declared using the D parameter).
K_SUBJ - This category is called when Eliza decides to ask the user to
         refer or to return to a previous subject (detected by the S
         parameter). The program uses the ~ parameter for this, converting
         it to the name of a previous subject.
K_SHORT - This cateogry may be activated if the user's reply is too short.
K_LONG - This category may be activated if the user's reply is too long.
K_NORMQ - This category contains messages which Eliza uses to say goodbye
          to a patient at the end of a conversation (when the user decides
          or asks to quit).
K_SOONQ - This category is same as K_NORMQ, but is activated if the user
          decided to quit after a very short conversation.
K_ASKQ - This category contains the queries which Eliza uses to ask the
         user if he/she is sure about quitting (called by the % parameter).
K_LC - This category may occasionally be chosen by Eliza, if the conversation
       is becoming long.

ECC-Eliza treats the fixed categories just like any other category,
so advanced flags, and other options could be incorporated in them.

Because all of the fixed categories may be called by the main program file,
they should all appear within the data file, otherwise KLast may be used,
and that may not correspond well (especially instead of categories such as
K_HELLO, K_ASKQ, etc).


Advanced flags
ÄÄÄÄÄÄÄÄÄÄÄÄÄÄ
This is one of the more sophisticated features for versions 4.00 and above.
Advanced flags allow Eliza to manipulate different flags (or variables)
throughout the conversation. Using the data file, the user may use up to
128 flags, which can carry a numeric value of between -127 to 128.
The values of each flag may be changed according to the category that
Eliza chooses and according to the desire of the data file programmer.
The value of these flags can be checked at any time, and according to that
ECC-Eliza may choose to activate special flag-based categories, or to
redefine the value of some flags.
For example, imagine we want to keep track of the user's politeness.
We will start out with a flag set to 0. Each time the user says "PLEASE"
or "THANK YOU" we shall increment the value of the flag by 10.
Each time the user says "DAMN" we will decrement the value of the flag by 10.
Then, let's say the whenever the value of this flag reaches 50, we will
tell the user how lovely he has behaved. However, if the flag reaches
-50, we will tell the user he has been a "bad boy" and end the conversation
with him.
To do this we need to do three things: reset the flag initially, update
the value of the flag in all categories that should effect it, and finally
build two special categories that will only be activated if the value
of the flag is 50 or -50. The following sections will explain this.


The Grant (G) parameter
ÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄ
Whenever we want to use a flag, we will want it to be set to a specific value
(between -127 and 128) in the beginning of the conversation, this will be
a default value for it. Up to 128 flags may be used, and they will be
referenced by their numbers, i.e. flag 2, flag 18, etc.
There is no need to define or declare a flag. All 128 flags can be used in
the data file whenever we wish to. However, we might want to declare an
initial value for them before we begin. To do that, we will use the G (grant)
paramter. This parameter can be used ANYWHERE inside the data file, and at
the loading process Eliza will go over all of them and set the values of
the flags according to the grant fields.
However, it is best to have all Grant fields together in the data file,
so that it is more readable. The format of the grant parameter is as follows:

G #flag,#value

#flag is the number of the flag, and #value is the initial value the flag
should be set to.

For example:
G 3,6

This instruction tells Eliza to set the value of flag number 3 to 6 before
the beginning of the conversation.

Please note that if you do not set an initial value for a flag, it will
default to 0. So, if you want your flags (or some of them) to be set to 0 at
the beginning, there is no need to activate this command, it will do so
automatically.


The Flags (F) parameter
ÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄ
In order to take advantage of the Advanced Flags, we will need a way to
determine the values of the flags within the conversation.
The F (flags) parameter is used for this. Once inside a keyword-reply
category, the F parameter can be used to manipulate the value of up to 5
flags at the time.
In our above example, we would want to use this in the "PLEASE", "THANK YOU",
and "DAMN" categories. In the first two categories we would want to increase
the value of the flag by 10, and on the last category we would want to
decrement its value by 10. The format of the F parameter is as follows:

F #flag,#change,#value,#prob

#flag is the number of the flag being manipulated.
#change is the type of change to be done and is either "+","-", or "=".
#value is the value to be processed according to #change.
#prob is the probability in percentage (0-100) for this parameter to be
executed.

If #change is "=", #flag's value will be set to #value.
If #change is "+", #flag's value will be incremented by #value.
If #change is "-", #flag's value will be decremented by #value.

Take a look at this example:

F 3,+,1,50

In this case, 50% of the times, the value of flag number 3 will be incremented
by 1. The other 50% of the times, nothing will happen.

The F field must be used between the keywords and replies of a category.
Up to 5 F field's may be used in each category and can used to manipulate
up to 5 different flags at once.

If you wish to manipulate the flag each time the category is executed,
set the fourth value (probability value) to 100, 0 can be used to bypass
any possible changes.

Finally, the value of a flag must always stay between -127 and 128.
And so, if you increment a flag evaluating to 125 by 10, it will be set
to 128, which is the possible maximum. You need not worry about an overflow
error, but on the other hand, please remember that the flag is limited
to 128 on the upper bound, and -127 on the lower bound.


The If (I) parameter
ÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄ
So far we have been able to change the values of flags before the conversation
and throughout it, but it is all useless if we cannot take advantage of them.
The I (if) parameter is condition used to check if a flag has reached a
desired value, and if so allows Eliza to activate a specific category.
The format of the I parameter is as follows:

I #flag,#compare,#value

#flag is the flag number to be checked.
#compare is either "=","/","<", or ">" and is the type of comparison to be
 done ("/" means not equal to).
#value is the value to be compared with #flag's value according to #compare.

For example:

I 2,<,30

In this case the value of flag #2 will be compared to 30. If the value of
flag #2 is smaller than 30, the condition becomes true the category following
the I parameter can be executed if any of its keywords match the user's reply.
However, if flag #2's value is 30 or greater, the category following this
IF parameter will be totally ignored regardless of anything.

Another example:

I 4,/,0

In the above example, the condition is: activate the category unless the
value of flag #4 is 0, i.e. for any other value (other than 0), activate it.
This is in fact the opposite condition of I 4,=,0.

The I parameter must come on top of a category, right before it.
Only one I parameter can be used for each category, and thus one condition
may be set for a category. If #compare is "=", the condition becomes true
only if #flag's value is equal to #value.

The I parameter may be used in conjunction to the C parameter so that even
if the condition becomes true, and one of the keywords is found, there will
be a chance the category will be skipped. The C parameter may come right
before or right after the I field.


The _EXE keyword
ÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄ
"_EXE" is a very special keyword. If this keyword is found in after the K
parameter, it tells Eliza that this category is a "must-activate" category.
If Eliza doesn't encounter any special event (i.e. repeating, blank reply,
etc), Eliza will allow to execute the category containing the _EXE keyword,
regardless of the user's input.
You may ask yourself what is the use of this thing, the answer is simple:
Advanced Flags. Many a time we will want Eliza to check for a certain
flag condition, and if true, we will want Eliza to activate a specific
category regardless of the user's input.
For example, if we want Eliza to say "YOU ARE VERY KIND" every time flag
#1 evaluates to 50, then we shall write the following code:

#
I 1,=,50
K _EXE
R YOU ARE VERY KIND
#

In this case the category will always be executed IF the condition for the
I field is true, i.e. if flag #1 = 50.
However, we must not forget something, if this does happen, then we will
be stuck ever after this occurs, i.e. this category will continue to active
itself all the time, because the value of flag number 1 will continue to be
50. And so we shall actually use the following:

#
I 1,=,50
K _EXE
F 1,=,0,100
R YOU ARE VERY KIND
#

Now, if this category is activated, Eliza will say "YOU ARE VERY KIND",
however, it will also set the value of flag number 1 to 0, right away and
100% of the time (always). Thus, this category will not be activated again
unless the value of flag #1 becomes 50 again.
It is very important to remember this issue, otherwise Eliza might get
trapped in a "black hole" and never be able to NOT active this category.


Reserved Flag Changes
ÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄ
Of the 128 flags available, numbered 0 to 127, there are 37 special flags,
numbered 90 to 127. These flags act the same, but their value is changed
automatically, regardless of the user's input. These special 37 flags are
divided into four different groups, varying in the type of automatic
change that occurs to them. The change occurs after each reply by Eliza,
and does not require any special defining or declaration within the data
file. For example, the value of flag number 90 is incremented by 1 after
every reply by Eliza, automatically. It could (and should) also be effected
by the flag changing instruction (F), but it is also independant, as it
could also change automatically.

The four groups of flags and their special changes are:

Flags 90-99   : Their value is incremented by 1 after every reply. 
Flags 100-109 : Their value is decremented by 1 after every reply. 
Flags 110-119 : Their value is incremented by 2 after every reply. 
Flags 120-127 : Their value is decremented by 2 after every reply. 

Take a look at this simple example code segment:

I 104,=,-9
K _EXE
R I HAVE REPLIED TO YOU TEN TIMES ALREADY
#
K KLast
R REALLY?
R GREAT!

In this example the program always accesses KLast, unless flag #104 evaluates
to -9... Let's check out the meaning of this. Because we have not defined
an initial value for flag #104 (no G field), we start the conversation with
flag #104 evaluating to 0. However.. flag #104 is between the range of
100 and 109, and thus, as explained about, the value of it is incremented
by 1 after every reply by Eliza. This is done automatically and without
the need for use of a special field (the F field).
In this example, while we continue to speak to Eliza (and it replies with
one of two stupid replies), it keeps on decrementing the value of flag #104.
After exactly ten times, the value of it will evaluate to -9 (because the
program first replies, and only then updates the value of the flags).
Once the value will become -9, the condition of flag #104=-9 becomes true,
and so the program would reply: "I HAVE RE...TEN TIMES ALREADY", which is
indeed a true saying.

If you are not interested in any automatic changes occuring to your flags,
just make sure to use the flags within the range of 0 to 89.


Advanced Flags Example Code
ÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄ
Here is the full code for the example we showed on the introduction of the
Advanced Flags section:

# reset the value of flag #1 to 0
G 1,0
#
I 1,>,49
K _EXE
F 1,=,0,100
R I APPRECIATE YOUR POLITENESS
#
I 1,<,-49
K _EXE
F 1,=,0,100
R I DON'T APPRECIATE YOUR LANGUAGE
#
K PLEASE
F 1,+,10,100
R DON'T BE SO POLITE
R YOU ARE VERY POLITE
#
K THANK YOU
F 1,+,10,100
R THERE IS NOTHING FOR
R IT'S MY PLEASURE
#
K DAMN
F 1,-,10,100
R ARE YOU MAD AT SOMETHING, |?
#
K KLast
R HOW ARE YOU |?
R ITS A BEAUTIFUL ^
R I SEE, NICE

The code first assigns the value 0 to flag number 1. This is unnecessary
because all flags default to 0 at the beginning, but it's just an example.
The first category will always be execute (_EXE) ONLY IF the value of
flag #1 is greater than 49 (i.e. 50 or more).
In this case Eliza will say show her appreciation for your politeness and
reset the value of the flag to 0.
The next category is the complete opposite: it will execute always ONLY IF
the value of flag number 1 is less than -49 (i.e. -50 or less), and will
cause Eliza to say she doesn't appreciate the user's rudeness.
Then, the value of flag #1 will be set to the initial value of 0.
The next three categories, are the keyword-reply blocks of this example
and change the value of flag #1 according to the user's input (good or bad).
These categories also yield out between 1 and 2 replies according to the
keyword encountered.
Lastly, the KLast category is used, and can yield one of 3 possible replies,
if none of the other categories matched the user's input and the value of
flag #1 is between -49 and +49.


Checking the values of Advanced Flags
ÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄ
When experiencing with the Advanced Flags, and especially when doing some
modifications to them, or adding new ones, you may want to check their
values in the middle of a conversation, to make sure everything works
fine. You may check the value of any one of the flags from within the
conversation. Just type "dFv ###". dFv stands for "display flag value",
it must be written in the EXACT case as above ("F" written in uppercase).
The second part, ###, stands for the 3 digits of the flag. For flag number
104 it would be 104, for flag number 6 it would be 006 (not only 6!), etc.

So, if you want to check the value of flag #18 in the middle of a conversation
with Eliza, just type (in the normal input prompt): dFv 018

Eliza will recognize the special command and will display the flag's value.
If the flag number is illegal or the second part isn't written correctly
(3 digits numbered 000 to 127), then an error message will be displayed.

As for the conversation, Eliza will totally ignore this command. It will
not try to process it in any way, and will not update any of its properties
if this command is detected. Of course you may check the value of several
flags by typing the command several times (reserved flag changes do not take
place if when you use this command, and it will also not be logged).


Tips on modifying the data file
ÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄ
First of all, you need to be creative. The tools (parameters and fields)
available to you, open you a wide variety of choices. This does not mean that
the time-of-day parameter (^) needs to be written every two replies, but you
may integrate it in many replies. We'd recommend you to check the data file
for yourself and see how such parameters are being used.
You must remember that editing the data file is not a simple task at all.
You need to think about all of the input possibilities when modifying or
adding new categories, keywords or even just replies.
Your replies should make sense whenever one of the category's keywords is
encountered. Think of many sentences that could include the keyword and act
accordingly. If you are not sure, either use more specific keywords, or add
another word to them. However, don't change current keywords to more
specific ones, as those can cancel the affect of the previous keywords.
Also, if you are interested in modifying a category by adding keywords,
it is important that you check for any asterisks (*) or backslashes (\) in
the replies. If a reply contains a backslash make sure the new keyword would
fit that reply logically. Also make sure that the words that could come after
that keyword in a sentence would fit the asterisk (*) parameter.
You must think of generally every possible option. The more time you spend
on that, the more logical Eliza may sound in different events.
You should also check for priority mismatches, and make sure that your
keywords are in the right priority. Also, if you have two categories, one
with the 'EVERY' keyword, and the other with the 'EVERY DAY' keyword,
make sure the 'EVERY DAY' keyword comes first, and acts as a logical filter.
You will uncover additional tips once you get to know the usage better.
Please note that the start of the data file contains useful and summarized
information on how to edit it. You may find that helpful.
When adding and working with Advanced Flags, you should take extra care
so that everything works logically, and you don't get Eliza to enter
'black holes', where one flag continually calls an _EXE-based category,
but its value doesn't change in order to prevent it from being called to
that category again. In any case, we recommend you to experience a bit,
but make sure to back-up anything before you touch it.


Behind the scenes work
ÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄ
The way the data file works is listed in this information file.
However, ECC-Eliza does NOT always use the data file as listed here.
ECC-Eliza processes the user's text many times before it contacts the
data file contents. In addition, there is a chance that keywords will be
skipped even if identified due the program's decision.
There are many other "behind the scenes" work that are used to promise
better manipulation of the user's text. For example, these decide the
program's attitude, and helps it decide upon a reply from a variety of
replies if it encounters a different attitude in it.
So the bottom line is that although Eliza follows many of the guidelines
as written here, it may also act differently many times. But as said, this
is done to make Eliza better, and in our opinion, it achieves that.


Limitations and errors
ÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄ
The data file and the program have some limitations:

- The maximum reply length is 255 characters (beyond that truncated).
- The maximum keyword length is 40 characters (beyond that truncated).
- Up to 255 replies per category.
- Missing asterisk length is limited to 25 characters.
- Subject name length is limited to 25 characters.
- Amount of different categories is currently limited to 364.
  If you exceed this number the program might act unexpectedly.
- Amount of different keywords in the data file is currently limited to 3750.
  If you exceed this number the program may fail to execute.
- Amount of different discussion subjects (D field) is limited to 128.
- Amount of characters per discussion subject is limited to 32 characters.
- Size of data file is limited to around 350KB, depending on memory state.
- Up to 1 asterisk (*) parameter may be used in a reply.
- Up to 1 underscore (_) parameter may be used in a reply.
- Up to 1 backslash (\) parameter may be used in a reply.
- A reply cannot contain both an asterisk and an underscore parameter.
- Up to 128 flags, numbered 0 to 127, can be used.
- Flags numbered 90 to 127 are automatically updated according to the
  reserved flag changes.
- Up to 5 flag change commands (F) can be used within each category.
- The value of each flag can be any whole number between -127 and 128.
- Each flag defaults to 0, unless set to another value by the G parameter.
                         
The only valid fields of the data files are #, K, R, C, F, G, D, M and S.
If a field begins with another character it will be ignored and you will
be notified at the execution of the program (ECCELIZA.EXE), with the line
number and line context.

[blue21]

@srvaldez I meant offline nonspam chatbots... just cause I'm asking a question about chatbots intelligence doesn't mean I wanna create a spam bot just like I don't assume that the fact you're programing in freebasic and other languages means you hack sits or code computer viruses...

here is one of my chatbots for you to see for yourself - you tell me if it's a spam bot or not...

viewtopic.php?f=8&t=31369

[Plasma]

There is an old book from 1988 titled "Exploring Natural Language Processing" by David Leithauser, which has BASIC code but it's old school spaghetti (line numbers, GOTO, etc). The algorithms also aren't very advanced by today's standards.

There are a lot of current books on Natural Language Processing, I would look for one that doesn't rely on Python libraries.

[Edgardo]

Hey, thanks for the post i am beginner and i was facing some problems and i got the one of them solved keep posting i really have to know other answers i will share shortly in a post. Lite Blue