New Domain

Personage Background

Personage is meant to be used as the generation component of a dialogue system. After a dialogue system gets and processes the user's input and decides roughly what it wants to say, it passes that rough characterization on to the generation component to generate a fully fleshed out utterance. The generation component is responsible for turning an abstract action like "Say Hello" into something more concrete like "Hey, how are you doing?".

What makes Personage different from most generators is that it generates language with recognizable style. Personage can alter text in many ways such as controlling the ordering of content (should the positive information come first?) and inserting stuttering. You specify probabilities for these parameters. For example, setting the stuttering insertion parameter to 0.1 will result in very little stuttering whereas 0.9 will result in a lot.

So in addition to providing Personage with that rough characterization of what to say, you also specify probabilities for each of these parameters. These values can be specified one by one, or you can define personality types in terms of these individual parameters and then use those higher-level personality types. For example, Personage has the Big Five personality traits pre-defined. You can see this in action by trying out the demo. You can set the parameters individually or click one of the personality types to set all of the parameters. 

Tutorial Structure and Goals

This tutorial will walk you through setting up Personage to be used as the generation component in a personal trainer dialogue system. Following the example will hopefully give you a sense of where to look and what to change when you want to add functionality. However, you should keep in mind that the example is only meant to give you a sense of where to start.

The tutorial is written as instructions for you to follow to if you were setting up the personal trainer system. However, all of the code referred to has already been written. If you download Personage from the Download & Setup page then you should have all of it in your Personage distribution.

Personage has the ability to build statistical models of personality in a new domain, but it depends on first programming a rule-based generator to create training data. This tutorial only covers the creation of a rule-based generator. Reading the first chapter (introduction) of this paper will give you a sense of the overall purpose of Personage, and reading the second chapter (on the Base Generator) will give you a sense of how the architecture of the rule-based generator works.

The tutorial assumes that you have followed the setup instructions and have Personage set up. Reading the paper linked to above would also be helpful before starting.

Summary of the tutorial's goals with pointers to relevant code

The overall purpose of this tutorial is to walk you through the extension of Personage to a Personal Trainer domain. The major things we will be doing are:

  1. Construct a generation dictionary, which is a mapping from dialogue acts (a single abstract idea, command, or question) to a set of syntactic representations of ways of expressing that act. For example, one dialogue act that we will create is a command to run X miles, where X is a variable to be filled in later.

  2. Create content plans (interchangeable with "text plans"), which are basically high-level specifications of what we want the system to say. Each content plan has two parts. The first is a set of references to dialogue acts in the generation dictionary and the second is a set of relations that hold between the dialogue acts. For example, a text plan could list the dialogue act from above, the command to run X miles, and a second dialogue act used to inform the user that he or she has not exercised enough. In addition to these two dialogue acts, we could say that a justification relation holds between them. Specifically, we could say that the second dialogue act is used to justify the first-- that the user hasn't exercised enough justifies commanding him or her to run X miles.

  3. Alter the built-in generation code so that it works with the generation dictionary and content plans.
  4. Control which pragmatic markers are allowed to be inserted. Pragmatic markers are words or phrases that don't carry actual meaning, but instead signify something about the discourse. One example is "you know", as in "I really liked it, you know, it's great". Another example is "well". Personage has a number of pragmatic markers available, but not all of them will make sense in all applications. So this section has pointers to the code that allows you to disable them. Note that you can see the discourse markers in the demo under the heading "Hedge Insertion".
  5. Make use of the lexical choice functionality, which allows more or less intense and longer or shorter words to be selected.
  6. Create new personality model that affects generation.

Parts (1) and (2) will be fairly contained and there will be tangible results when they are done; the text plans will successfully be turned into utterances. However, at this point Personage will not be doing much processing.

The remaining sections will cover how to change that-- how to get stylistic variation in the output. We will be using very simple styles/personalities through most of this. Until we take on personality modeling as our actual goal, we will be using the built-in Big Five model of personality. The trainer will appear to be either highly extraverted and agreeable or appear to be introverted and disagreeable. 

The first thing to do is set up a main program to run our example code. This code is in generator.ExampleTrainer (the file is at Personage/src/generator/ExampleTrainer.java). Some things will be changed as we move forward, but this should give you a basic idea of what's going on at the highest level.

ExampleTrainer.java

File propertyFile = new File("PersonageTrainer.properties");
Personage generator = new PersonageRuleBased(propertyFile, 0.10);

A lot of Personage's behavior is controlled by a property file. Here we load the property file, which is at the root of the Personage directory. We then create an instance of the rule-based version of Personage. In contrast with the (statistical) parameter estimation version of Personage, the rule-based version uses hand-crafted rules to make its generation decisions. This paper discusses some of those rules and the Model section on this page briefly does. The second argument to Personage's constructor specifies the amount of noise to use so as to get more variety in the output.

Map<StylisticDimension, Double> extravert = new LinkedHashMap<StylisticDimension, Double>(1);
extravert.put(BigFiveDimension.EXTRAVERSION, 7.0);

Next we create a persona, which is implemented as a mapping from the stylistic dimensions to values between 0.0 and 7.0. However, the rule-based generator only uses binary choices--values above 4 are mapped to high and values at 4 or below are mapped to low. So we can use 0.0 to represent "low" and 7.0 for "high". In this case the stylistic dimension is EXTRAVERSION, one of the Big Five personality traits, and map to the high value. We could also use any of the other four traits. Or, as we will see later, we can create our own.

Document commandTP = XMLUtils.parseDocument("resources/tplans/trainer/command.xml");
String extravertCommand = generator.generate(commandTP, extravert);
System.out.println(extravertCommand);

We then load a text plan and store it in commandTP. The generate method of PersonageRuleBased is called with the text plan and persona as arguments. It returns the output as a string, which at one run turns out to be "Let's see... You should run around 5 miles!".

Generation Dictionary

Our first step is creating the generation dictionary. In concrete terms, the generation dictionary is where you put the basic units of what you want the system to say. Our basic units are dialogue acts. These are usually single sentences and they usually express individual propositions, but are sometimes commands or questions.

We will put the generation dictionary for the Personal Trainer in Personage/resources/dsynts/trainer. "DsyntS" and "DSS" both stand for "Deep Syntactic Structure", which you can read more about in the RealPro Manual. Each of the filenames in the trainer directory is prefixed with the dialogue act type. The actual act comes next, followed by the .xml extension.

There will only be two categories that dialogue acts in the Personal Trainer will fall into: inform and command. One dialogue act of the inform type is inform_not_enough, which is used to tell the user that he or she hasn't exercised enough. An example of the command type is command_run_X_miles. Unlike the previous act, this one requires an argument to be whole (the number of miles to be run, X).

Each file holds one dialogue act expressed in a number of different ways. The top element in each xml file is <dsynts-list>, which contains a number of <dsynts> elements each representing one of the "ways". Here is a DSyntS from inform_not_enough.xml:

  <dsynts id="nephew" polarity=".1" complexity=".5">

    <dsyntnode class="verb" lexeme="exercise">

      <dsyntnode lexeme="newphew" class="common_noun" gender="masc" rel="I">

        <dsyntnode lexeme="&lt;POSSESSIVE_PRONOUN>" person="1st" rel="ATTR"/>

        <dsyntnode lexeme="6_year_old" class="adjective" rel="ATTR"/>

      </dsyntnode>

      <dsyntnode class="adverb" lexeme="more_than_that" rel="ATTR" position="post-verbal"/>

    </dsyntnode>

  </dsynts>

The polarity value represents how positive the sentence corresponding to the DSS is, and the complexity value represents its syntactic complexity. These two attributes are used during syntactic template selection (when a <dsynts> is chosen from the <dsynts-list>). For example, extraverts tend to be more positive and use less complex language, so Personage is less likely to select this DSS than one that has high polarity and lower complexity.

The child element of the <dsynts> element contains the actual deep syntactic structure. To really get a sense of how these work you'll probably want to read the RealPro manual, read a bit about English syntax and dependency grammars, and most importantly play with the representations. You can run RealPro with a DSS file as the argument and it will print out the realized contents. There are windows and shell scripts that you can copy or use in Personage/lib/RealPro-2.3/sample-applications/Realizing-DSyntS-Loaded-From-XML-File. Running RealPro on the DSS above results in "My 6 year old newphew exercises more than that.". 

As mentioned above, each DSS file usually contains more than one DSS. Here is another DSS from the same file as the one above, inform_not_enough.xml: 

<dsynts id="notrunenough" polarity = ".9" complexity=".5">

    <dsyntnode class="verb" lexeme="exercise" polarity="neg" tense="past">

      <dsyntnode lexeme="&lt;PRONOUN>" rel="I" person="2nd" number="sg"/>

      <dsyntnode class="adverb" lexeme="enough" rel="ATTR" position="post-verbal"/>

    </dsyntnode>

  </dsynts>

When realized by RealPro, this DSS becomes "You did not exercise enough". The idea here is that both of these DSS say roughly the same thing--that the user didn't exercise enough. The first one uses a more indirect approach while the second one is completely literal. The important thing is that they are more or less interchangeable in terms of information presented, at least with respect to our domain. For example, one of these DSSs has information about a nephew while the other doesn't, but that isn't domain-relevant information. It would be a problem if one of the DSSs said that the user exercised a lot since our DSSs would contradict each other.

Description of the Deep Syntactic Structure

Here's a brief description of the DSS above which may or may not be helpful. Each node in the dependency structure begins with the tag <dsyntnode>. The value of the class attribute is the part of speech of the word and the value of lexeme is the uninflected (unconjugated) word. So our root node is the verb "exercise". Note that when the DSS is realized, the "s" is added to "exercise" so that it agrees with the subject "my 6 year old nephew".

There are a few relationships that can hold between nodes. The relationship between a node and its dependent nodes is marked in the dependent nodes as the value of the rel attribute. The node headed by "nephew" is the subject of "exercise", so its value is "I". Direct and indirect objects have the relations "II" and "III" respectively. Adjuncts are marked "ATTR".

The "nephew" node is modified in two ways. It has as an ATTR the adjective  "6_year_old". Using underscores in a lexeme is a way of treating multiple words as one and ignoring their internal structure.  For our purposes the internal structure doesn't matter. It also has the first-person possessive pronoun as a dependent, which changes the default "the 6 year old nephew" to "my 6 year old nephew".

The verb "exercise" also has the adverb "more_than_that" attaching as an ATTR. Once again, there may be some internal structure here but that probably won't matter.

Property File

In order to get Personage to use our new DSSs, we have make a change to our property file. As mentioned above, the property file is in the root of the Personage directory. There is one property file per project (unless you want to keep all the same settings), and we call ours "PersonageTrainer.properties". So, we tell Personage where to look for our generaction dictionary (our DSSs) by setting "DSSDirectory" in the property file to "resources/dsynts/trainer". 

Text Plans

The DSSs described above are representations of the individual dialogue acts that are available to the Personal Trainer system. At times we may want the system to only express one act at a time. For example, the system could express the command to run X miles  (command_run_X_miles.xml) with the argument 47, which would turn out as something like "Please run 47 miles!".

While this may be desirable at times, there are also many cases where we want the system to express multiple dialogue acts in the same utterance. And when there are multiple dialouge acts, there are often relations that hold between the acts. The types of relations in Personage are based on Rhetorical Structure Theory (RST). The main idea behind RST is that spans of text are related to each other. For example, the two sentences in "John was late for work. He got fired" are related to each other in that first results in the second. Sometimes the relations are marked explicitly. If two sentences are related by result, they are often joined by "so". Or if the result comes first, by "because". Even when there is no explicit sign that there's a relation, it's usually assumed by the hearer. For example, if you heard the two sentences above you would probably conclude that John got fired because he was late.

Most relations contain a nucleus (the central information) and a satellite (the tangential information). For the justify relation, which is one of the relations in Personage, the nucleus is the main point and the satellite justifies the speaker's saying the main point. For example, if we have one dialogue act that commands the user to run X miles and another that informs the user that he or she hasn't exercised enough and we have a justify relation holding between the two, then we are telling the user to exercise more since he or she hasn't done ehough.

The current set of content (aka text) plans for the Personal Trainer are in Personage/input/tplans/trainer. 

Example

<speechplan>
    <rstplan>
        <relation name="justify">
            <proposition id="1" ns="nucleus"/>
            <proposition id="2" ns="satellite"/>
        </relation>
    </rstplan>
    <proposition dialogue_act="command_run_X_miles" id="1">
        <attribute arg="X1" name="SELECTION" value="3"/>
    </proposition>
    <proposition dialogue_act="inform_not_enough" id="2">
    </proposition>
</speechplan>
The text plan is composed of two main parts: the rhetorical structure (rstplan) and the propositions. Each <proposition> element references a dialogue act from the generation dictionary (one of our xml files) and includes any additional information that the DSS will need, such as what X is in the command to run X miles.. The <rstplan> element can have any number of child relations, but in this case only has one. The child is a justify relation thats nucleus is the first proposition and thats satellite is the second proposition.

Property File

There's a property called "TextPlansDirectory" which should be set to "input/tplans/trainer". All of the text plans in the personal trainer example are written by hand and placed into that directory. Another approach which you may come across at some point is using a text plan builder. For the restaurant domain (see generator.ExampleMATCH), the text plans are built automatically by using a text plan builder (see "Personage/src/generator/textplan/TextPlanBuilder.java"). If you want to use a text plan builder for a new domain, you would need to write a new Java class for your specific application.

Aggregation

Aggregation is the merging of syntactic constituents, often sentences. For example, the sentences "John eats oranges" and "John likes cats" could be aggregated by mering them into "John eats oranges and likes cats" or into "John, who eats oranges, likes cats".

Nothing more should be required for this simple example to get aggregation working. But for more complicated examples, you'll need to create new dialogue strategies. You'll want to look at the transform method in RSTTransformer.java.

public RSTPlan transform(SpeechPlan speechPlan, DSSTransformer generator)

Pragmatic Marker Insertion

Personage can insert pragmatic markers such as hedges (adding "sort of" to lessen effect), stutterings, etc. into the aggregated text structure according to the personality parameters passed in. Sometimes these will be inappropriate for a domain though. You can turn these off by using the "disableParameters" method in the Personage class.

Model

At this point we'll talk about the structure of personality models within Personage, and briefly cover how to create a new one. Personage can insert restatements of a message into an utterance, hedge claims to weaken them, select weaker or stronger words, and do many other operations that influence how the personality of the "speaker" is perceived. A model consists of a mapping from each of these operations to a number between 0.0 and 1.0, which represents the likelihood that that operation is used.

The Personage Rule Based Generator has five personality dimensions built in-- the Big Five personality traits. You can see the parameter files in Personage/resources/parameters/handcrafted/bigfive. There are ten files in that directory, two for each trait. Each trait has a file that defines the low end of that personality (e.g. params-extra-low.xml for introversion) and another file for the high end (e.g. params-extra-high.xml for extraversion).

New Model

The first step is to create a new directory in Personage/resources/parameters/handcrafted. Our new directory will be called "trainer". It will only contain two files: params-com-low.xml and params-com-high.xml. "com" stands for "commitment", which is the only dimension we'll be using. A trainer with low commitment would be apathetic about your progress and one with high commitment would really care about you. However, the two files are actually just copies of the extraversion parameter files. The point here is to show what files you need to create. What values to give to the settings in order to create the personality you want is up to you. The next section will summarize the parameters you can control.

Now we have to add a new class to the package generator.parameters.dimensions. In this directory (Personage/src/generator/parameters/dimensions), you'll see the file StylisticDimension.java contains the base class for all dimensions. BigFiveDimension.java specifies the five dimensions available in that model. Our new file TrainerDimension.java only contains one dimension: commitment. It's important that its short name ("com") matches the name of the xml files.

Parameters

This section will quickly summarize the types of parameters you have control of. You can find most of the information in the paper linked to at the top of this page. Also, you can see all of these parameters in the online demo.

Property File

Two properties must be changed. The value for "StylisticDimensions" should be set to "Trainer" and the value for "StylisticParametersDir" should be set to "resources/parameters/handcrafted/trainer".

Results

In this section we'll look at some output. We'll use the text plan justify_command.xml and the three personalities neutral (nothing specified), committed (commitment=7.0) , and apathetic (commitment=0.0). 

Neutral

You should run 3 miles because my 6 year old newphew exercises more than that.

The neutral utterance is the closest to what we find in the text plan. The text plan used for all three examples has a justify relation with the dialogue act command_run_X_miles as the nucleus and inform_not_enough as the satellite. Both propositions are included in the utterance, joined by the connective "since".

Apathetic

Well, I mean, run 3 miles.  
The apathetic personality has all of the parameters' values set to .25. Since verbosity, repetitions, and restatements are set to low, the satellite isn't even included. Also, the pragmatic markers "I mean" and "well" both have values 1.0, so they are inserted.

Committed

Let's see... Oh gosh my really 6 year old newphew exercises more than that! You should run 3 miles, okay? Run their miles.
 
Since "verbosity" and "restatements" are both high, both propositions are said twice. Also, a number of hedges are inserted. Near Expletives has the value 1.0, so "Oh gosh" is inserted. Note that the last sentence "run their miles" doesn't really make sense. This happened because the rules used to create restatements didn't work in this domain and need to be altered.