Table of Contents

MyRubberBands.com - A Case Study

Creating a Markup Language

Summary

VoiceXML with XSLT

Copyright and Authorship Notice

This is an abridged chapter extract and was written by Stephen Breitenbach. It is taken from "Early Adopter VoiceXML" by Eve Astrid Andersson, Stephen Breitenbach, Tyler Burd, Nirmal Chidambaram, Paul Houle, Daniel Newsome, Xiaofei Tang, and Xiaolan Zhu, published by Wrox Press Limited in August 2001; ISBN 1861005628; copyright © Wrox Press Limited 2001; all rights reserved.

No part of this chapter may be reproduced, stored in a retrieval system or transmitted in any form or by any means -- electronic, electrostatic, mechanical, photocopying, recording or otherwise -- without the prior written permission of the publisher, except in the case of brief quotations embodied in critical articles or reviews.

This chapter examines the use of the Extensible Stylesheet Language for Transformations (XSLT) as a tool for the generation of VoiceXML. I intend to illustrate a complete, end-to-end example of implementing a voice interface for a client-server database via XML and XSL. The case study will demonstrate the power of XSL for simultaneously delivering multiple interfaces to the same data by developing HTML and WML front-ends also.

Our case study takes us inside the rarified atmosphere of a fictional cash-strapped dot-com called MyRubberBands.com, purporting to be the "premier rubber bands site on the Internet". In the aftermath of the stock market meltdown, where our heroes saw their market valuation drop by over 95%, senior management, led by CEO Dr. Todd, has decreed that adding WML and voice functionality to the existing order status web site is do-or-die. MyRubberband's competitors have just rolled out their own WAP/voice access solution, and an all-out effort is necessary to catch up. Follow the programmers as they embark on their project to quickly roll out an equivalent capability.

Our development team have decided to implement an XSLT-based solution to the problem. XSLT is an XML-based language for transforming input structured according to one XML vocabulary to structured output in another XML vocabulary, or some general text form.

XSLT treats the document to be transformed as a set of nodes. An XSLT stylesheet defines a set of rules, or templates. When a template matches one of the nodes in the source document, the output structure given in the template is created in the transformed document. XSLT uses the W3C XPath specification language to query XML data. XPath is strongly analogous to SQL, and lets us specify complex rules to match nodes in a document.

For a lot of applications, and for getting off to a quick start when processing XML, it is just the ticket - especially when you consider that XSLT is a relatively new technology so that processors should still have plenty of performance improvements possible. Use of an XSLT processor avoids the startup overhead of using a full parser API from a compiled language, making it more suited to dynamic web applications.

MyRubberBands.com - A Case Study

Our legacy database is implemented with MySQL, an open source client-server relational database management system. SQL code for the schema and a set of sample data is included in the code download for this book.

An XML schema is used to represent an export of the legacy database instead of a Document Type Definition (DTD) because the schema standard is now complete, and increasing numbers of developers will be looking for the extra power schemas offer, especially as new development tools become available. An excellent primer on schemas is available from the W3C at http://www.w3.org/TR/xmlschema-0/.

Scripts to export the database to XML format have been written using the Perl language, and the Data Base Interface (DBI) library. The Perl script shown was developed on Windows using Active State Perl, but should run on any platform, be it Windows or Unix. Many commercial databases, such as SQLServer 2000, are capable of exporting directly to XML, and so this step could be avoided entirely.

Business Requirements

With their competitors rolling out both voice and WAP access to services, MyRubberBands.com has no choice but to follow suit or lose market share in the cutthroat world of elastic band marketing. Due to market pressures, the new system must be up and running as soon as possible, and given this short development cycle, the requirements have been scaled back to providing simply voice and WAP access to a customer's order status data.

However, some thought can still be given to the future. Rather than develop a quick and dirty "throwaway" voice interface, by putting in a little extra work now, the engineers can build a reusable infrastructure. By exporting their database to an XML format, they access the power of XSL to create VoiceXML and WML interfaces, and are able to transparently replace parts of the existing HTML site with dynamically created pages.

System Architecture

The figure below is a block diagram showing the existing components of the system, and the relationships to the new XML/XSLT system required to implement the voice interface.

This drawing is not complete in all areas. For example, no method for user login and authentication is given, because such a system would already exist for an e-commerce site, and because although XML/XSLT would be helpful for creating device-specific login code, we are not going to examine on-the-fly transformation (inside a web server, for example) in this particular study.

Designing a Voice Interface

With these rather vague requirements in mind, we can make some design decisions, and sketch out a rough model for the voice interactivity envisaged. The goal is to make the experience simple and intuitive.

There will be a main menu of options. This is the entry point to the application, and the user can always return to it with a single voice command.
Online help will always be available. This will use the VoiceXML <help> tag to simplify implementation, and also to overload any built-in help that may be offered by the voice platform.
The number of available options from the main menu should be kept to a minimum. The total number of states should also be minimized. This means that the behavior of the current command should not depend on what the previous command issued was. For example, the word "menu" should always refer to the main menu in every context.
The top-level commands from the main menu should always be active. If the main menu offers the command "foo", the user should be able to say "foo" at any point in later dialogs with the same result.

The following state diagram illustrates these design goals. The main options are "order status", "product list" (with a link to voice ordering via the existing phone service bureau), and "more information" to access a frequently asked questions list. For a more detailed examination of the issues to consider when designing voice applications, refer to Chapter 6. The order status menu leads to a variable number of additional choices, depending on the number of records in the user's order history.

Creating a Markup Language

Naturally, our fictional rubber band team already has a database-driven e-commerce web site. Like all legacy databases, it has evolved over time into a hodgepodge of tables, some of which were hastily knocked together to implement poorly-defined requirements. We will assume that the company is operating a traditional Java Server Pages (JSP) site.

Since most of the tables in the database are relevant to the requirements of the various interfaces, the developers plump for a "verbose" approach to their XML. They will dump all of the data from all of the tables into XML form, even though some of it may be unnecessary in the VoiceXML, WML, or HTML contexts.

MyRubberbandsML by Trial and Error

The first thing any XML dialect needs is a top-level element. Since we might want to export all the customers in the database, or only one at a time, let's add an attribute on the top level element to describe what kind of data feed this XML document constitutes.

The thing we are most interested is a customer record, because that will be the set of data needed to generate the voice interface for querying order status. Since we might have more than one customer in a file, each individual <customer> and their associated order history will be contained by a <customer_record> element. Note that the time stamps are in XSL's standard format, and won't translate easily for rendering by a TTS engine. The <customer_record> element that starts here is very lengthy, and is not closed until the associated addresses and order history that follows have been given.

<customer_record>

<middle>Quincy</middle>

<lastname>Public</lastname>

<username_or_email>jqp@foo.foo.com</username_or_email>

<password_or_pin>bar</password_or_pin>

<date_joined>2001-05-18T16:17:15</date_joined>

<date_lastchg>2001-05-18T16:17:15</date_lastchg>

</customer>

As shown in the database schema, a customer can have one or more addresses. The XML representation should preserve the foreign key relationship with the customer table, and this relationship should not be dependent on the position of the elements. In this case, for example, both the customer profile and all associated addresses are nested within the <customer_record> tag. This is why all of the <customer_address> elements carry the customer_id attribute inside: it mirrors the relationship between the customer and customer_address tables in the database schema.

In this case, the database schema allows the customer_address table to store real physical addresses like billing and shipping addresses, or e-mail address for alternate methods of customer contact. Hence, the <customer_address> element can contain the optional <email> element.

<customer_address address_type="Ship To Address"

customer_id="1">

<address1>4321 La Place Ct</address1>

<city>Carlsbad</city>

<state_or_prov>CA</state_or_prov>

</customer_address>

Since the main objective of the voice application is to allow the user access to their order history and status information, it makes sense to include the history inside the <customer_record> scope. In this case, because we will need to enunciate the order time, and because we'd rather not use XSL's limited text processing capabilities, we'll add the attribute sayas to the <order_date> element. This provides a pronunciation that can be used with the VoiceXML <sayas> tag for TTS. However, the desired pronunciation of the date and time cannot be derived from the database alone, as addressed in the section Generating MyRubberbandsML.

<order_history customer_id="1">

<customer_address address_type="Ship To Address"

customer_id="1"/>

<order_date sayas="May 18, 2001 at 16 17 hours">

2001-05-18T16:17:16

</order_date>

<order_status>Processing</order_status>

<shipping_charge>0.4</shipping_charge>

<total_charge>2.48</total_charge>

</order>

</order_history>

Finally, to provide the user with detailed order information, the full product name and description must be available. This will also allow the user to ask for a product list, and eventually we can perhaps extend the interface to enable products to be ordered by voice. Note that the <product_list> is not associated with any particular <customer_record>.

Also note that, ideally, the <product_list> wouldn't actually be in the same document as the customer data. However, we'll keep everything in one file here to avoid the issue of linking between documents. It might amuse us to picture the harried developers reaching the same conclusion to save time and give themselves some chance of meeting their beloved boss's deadline. Later on, they will no doubt want to refine the process and generate smaller XML documents that can be processed more quickly.

</customer_record>

<product_list>

Mixed Bag of 1000 Rubber Bands</product>

Mixed Bag of 5000 Rubber Bands</product>

Bag of 1000 Red Rubber Bands</product>

Bag of 10000 Red Rubber Bands</product>

Bag of 1000 Blue Rubber Bands</product>

Bag of 10000 Blue Rubber Bands</product>

</product_list>

</myrubberbands>

This example is formatted to fit the space above, and for readability, adds quite a bit of whitespace between <product></product> tags that would probably not occur in a real document.

Generating MyRubberbandsML

Now we have examined the existing database, outlined a suitable voice interface for it, and defined our source markup language and the method for generating it. Now, we are ready to create a stylesheet to convert it to a VoiceXML form implementing the design we decided on in the previous section Designing A Voice Interface. This stylesheet, myrubberbands2vxml.xsl, is quite lengthy, and can be found in its entirety in the code download. Here, I shall pick out just the important points in the code for discussion; including the dynamic generation of grammars, some VoiceXML features worthy of particular attention, and fundamental XSL concepts used.

VoiceXML Stylesheet

Note that the stylesheet is designed to produce a single VoiceXML document containing just one user's data. So, its top-level template only matches documents where the top-level attribute export_type is set to single. The indent attribute on the <xsl:output> tag will produce a well-formatted result document that will be easier for a human brain to examine.

<?xml version = "1.0"?>

<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" version="1.0">

<xsl:output method="xml" encoding="ISO-8859-1" indent="yes"/>

<xsl:template match="myrubberbands[@export_type='single']/customer_record">

The next block illustrates one way XSL can generate elements with an attribute having dynamic content: the <xsl:element> construct.

<xsl:element name="meta">

<xsl:attribute name="name">description</xsl:attribute>

<xsl:attribute name="content">Voice Interface for #<xsl:value-of select="customer/@id"/>

</xsl:attribute>

</xsl:element>

We will need to set up some variables for use in the VoiceXML document. First off, we grab the user's Automatic Number Identification (ANI) and Dialed Number Identification Service (DNIS) for later use. These correspond to the phone number that originated the call (analogous, but not identical to, the consumer caller ID service) and the number that the user dialed. The implementation of these is system dependent, and the data may not be available for all calls in any case. They are included here mainly for illustration. In a real application, the ANI can be used for auto-identification of the user.

The form_pointer variable will be used for navigation later.

Next come the form level help dialogs, which here attempt to mimic typical responses likely from a real life call center, contrary to the advice of Chapter 6:

What seems to be the trouble?

</help>

Come on - isn't this easy enough to understand?

</help>

Hey <xsl:value-of select="customer/firstname"/>, are you stupid or something?

</help>

The design specifies that the main menu command is always available. We can implement that with a global VoiceXML <link> element:

</link>

Again, we use the information from the XML file to customize the prompts for the user. This form welcomeMessage corresponds to the "welcome message" box in the interface design diagram earlier.

<block>

Hello, <xsl:value-of select="customer/firstname"/>.

Welcome to the my rubber bands dot com voice order status system.

</prompt>

</block>

</form>

Next, we come to the mainMenu form, the primary form of the voice application. There is a form level <nomatch> element here to transfer control flow to the errorHandler form when an utterance doesn't match the grammar. This form is used for most no-match events throughout the application, to keep track of the total number of such errors that have occurred this session. The <noinput> handler here ensures the main menu is repeated when the user doesn't respond to the prompt. In a future version of the product, the designers may implement some kind of timeout to restrict the number of loops, and disconnect the user if there is no response for a long time, but this issue need not concern us now.

</nomatch>

</noinput>

<block>

</block>

This is the main menu.

You can say product list to hear a list of products.

You can say order status to check your order status.

You can say frequently asked questions to get more

information.

You can always say main menu to return to this

menu, or help for additional help.

</prompt>

list

product list |

more information |

frequently asked questions |

questions |

order status

</grammar>

In order to keep track of the currently active form, the global variable form_pointer is set. This could be used to implement specific navigation logic, for example, by changing the behavior of a command slightly depending on where the command originated. Once the prompt has been played, and the user has responded with an utterance matching the inline grammar, the <filled> handler for this <field> is entered. This copies the value returned by the grammar to the global variable user_command, which is used by the navigator form to direct control flow to the required form. We could also use <subdialog> and pass a parameter, but this is simpler, and sufficient for this application.

</filled>

</field>

</form>

Now our main menu form is finished, we can start to implement the dialogs that provide the application's basic functionality. First up is the orderStatus form that makes the most extensive use of dynamic content generation:

</noinput>

</nomatch>

We need a test here to check that the customer does indeed have outstanding orders, and play a message to that effect:

<xsl:if test="count(/myrubberbands/customer_record/order_history/order) != 0">

This is a list of all your orders.

</xsl:if>

<xsl:if test="count(/myrubberbands/customer_record/order_history/order) = 0">

You have not placed any orders within the last thirty days.

</xsl:if>

The next block creates a prompt for each order that was in the source XML document by using the XSLT element <xsl:for-each> to select XML elements that match the XPath in its attribute. The sayas attribute from the <order_date> element in our XML file is used here to provide an audio cue to identify the order to the user.

<xsl:for-each select="order_history/order">

<xsl:value-of select="order_date/@sayas"/>,

</sayas>

you placed an order. Say order number

<xsl:value-of select="position()"/>

to hear more about it.

</xsl:for-each>

</prompt>

list |

product list |

more information |

frequently asked questions |

questions |

order status |

The grammar also includes an option dynamically generated by the XSLT code. The user can say, "order number one" to access data on the first order in their list. This JSGF could be improved to accept shorter instructions, such as "order one", or even "one", but be aware that using "order number" followed by the number will help the ASR system correctly identify the user utterance, and will probably improve application performance in this situation.

In this chapter, all the grammars are inline, but XSLT could just as easily be used to create standalone external grammars in separate files, or to generate grammars in multiple formats (GSL, JSGF, XML) from a common data source, by the application of different stylesheets.

<xsl:for-each select="order_history/order">

order number <xsl:value-of select="position()"/> |

</xsl:for-each>

buy me

</grammar>

</filled>

</field>

</form>

Just as we iterated over the order_history/order node-set to create grammars with options for all the orders of a customer, we must now iterate over these nodes again to create individual forms containing the details for each order. If there were consistently a very large number of orders per user, we might look at generating the order details on the fly using JSP or PHP to create a separate VoiceXML document.

<xsl:for-each select="order_history/order">

<xsl:variable name="order_detail_counter" select="position()"/>

The block above illustrates the other way XSLT offers for including dynamic content in an attribute value, this time through the <xsl:variable> element, the value of which can be output by preceding the identifier with the dollar sign, $. This technique is used several times when creating the VoiceXML forms for each order.

</noinput>

</nomatch>

<block>

</block>

The following code generates the prompt to announce the order detail:

This order was placed on <sayas class="date">

<xsl:value-of select="order_date/@sayas"/></sayas>.

The order consisted of

<xsl:for-each select="product">

quantity <xsl:value-of select="./@quantity"/> of product

The <xsl:value-of> element below retrieves the name of the product with an XPath expression that selects products from the XML product_list section where the id attribute matches the id attribute of the current order. This is analogous to an SQL join between the customer_order and customer_order_product tables from relational schema. Note that XPath denotes an attribute with use of the at symbol, @.

<xsl:value-of select="/myrubberbands/product_list/product[@id=current()/@id]"/>

</xsl:for-each>

The total of the order was

Next, we see the VoiceXML <sayas> element put to use. Whether or not this causes the contents to be rendered correctly as currency depends on the TTS engine used, and its support for pronunciation markup. The rest of the block repeats the previous technique to add order options to the inline <grammar> element.

$<xsl:value-of select="total_charge"/></sayas>.

The status of this order is

<xsl:value-of select="order_status"/>.

</prompt>

list |

product list |

more information |

frequently asked questions |

questions |

order status |

<xsl:for-each select="/myrubberbands/customer_record/order_history/order">

order number <xsl:value-of select="position()"/> |

</xsl:for-each>

buy me

</grammar>

</filled>

</field>

</form>

</xsl:for-each>

At this point we'll skip over the product list option, because it doesn't illustrate anything we haven't already seen. However, the product listing prompt offers the user the option to say, "Buy me", in which case their call is transferred to the MyRubberbands.com's telephone service center by the placeOrder form below. This is the quickest way for the company to add some commerce capability to the voice system, but note that the VoiceXML <transfer> element is not implemented by all platforms. When the user returns from the call, they are unconditionally sent to the main menu by the subsequent <goto> element.

<block>

Transferring your call to customer service.

</prompt>

</block>

<transfer name="callSales" dest="MYRUBBERBND" connecttimeout="30s"

bridge="true"/>

<block>

</block>

</form>

We can also skip over the frequently asked questions option, since it contains only static VoiceXML code. We are now almost at the end of our stylesheet, where the navigator form is located, holding the navigation logic for many of the state transitions in our interface design.

It consists simply of an if-else construct that expresses the state transitions on the diagram of the user interface earlier. If more commands, states, or transitions were to be added to the design, the complexity of the interface might exceed the limitations of this method of implementation.

<block>

<elseif cond="user_command == 'questions' ||

user_command == 'frequently asked questions' ||

user_command == 'more information'"/>

The stylesheet drives the rest of the navigation engine by creating forms for each individual order in this customer's order history. It does this by means of the following <xsl:for-each> construct:

<xsl:for-each select="order_history/order">

<xsl:variable name="order_counter" select="position()"/>

</xsl:for-each>

<else/>

</if>

</block>

</form>

Finally, we come to the errorHandler form that is used by most of the <nomatch> handlers in the dialog. This error handler keeps a running count of the number of errors, and when four errors have occurred, it will apologize and disconnect the user. This isn't the friendliest way of handling errors, and is not suitable for a long-range solution that would more likely transfer the user to a human operator at this point. Not only that, but we'd probably want more sophisticated logic for processing errors, maybe using ECMAScript to vary the response according to the time elapsed since the last error.

<block>

<assign name="session_error_count"

expr="session_error_count + 1"/>

I'm sorry, but I'm unable to understand you.

</prompt>

It seems I am having trouble.

</prompt>

</if>

<else/>

I'm sorry, but I'm having a lot of difficulty understanding you. If you

are currently in a noisy environment, please call back later.

</prompt>

<exit/>

</if>

</block>

</form>

All that needs to be done now is to close the document, after including an empty template matching standalone <product_list> elements, to suppress any output from them. Without this, default XSLT templates would be applied that output text children of any elements that aren't explicitly matched by a template already.

</vxml>

</xsl:template>

<xsl:template match="product_list"/>

</xsl:stylesheet>

We've now reached the end of our VoiceXML stylesheet.

Running the Stylesheet

Now it is time to run the stylesheet transform, and produce a complete VoiceXML document for one of our users. If you are using Saxon, enter the following command at the command prompt:

C:\> saxon customer_1.xml myrubberbands2vxml.xsl > customer_1.vxml

We are now ready for our VoiceXML interface to go live. This simply requires us to upload the result of our XSLT transformation to our chosen Voice gateway.

Summary

Perhaps the greatest advantage to implementing an XSLT-based system such as this is that documents in other markup languages can be readily generated from the same XML source data. The complete version of this chapter goes on to demonstrate how easy it would be to use MyRubberBandsML with an appropriate XSLT stylesheet to produce WML and HTML interfaces in addition to the above VoiceXML code.

The full chapter explores the process of using XSLT to provide user multiple interfaces to a legacy database that does not provide native XML access to data. Starting with a set of requirements for a voice interface taken from our business needs, an XML format to mark up the legacy data is created, along with an associated XML Schema to document it. We then create XSL templates to automatically generate our VoiceXML interface, and run this VoiceXML inside a simulator, before moving on to look at stylesheets that automatically produce parallel WML and HTML interfaces.

Recent Jobs

Sr. Software Engineer - Analytics
Immediate Mainframe openings for Ch
Immediate TANDEM-TAL openings for C
Immediate ASP.NET/C# Openings for C
Sr. Software Engineer
View all Jobs (Add yours)
View all CV (Add yours)

teleconferencing services
water softener
Teleconference
Host Department NOLIMIT Web Hosting
MSN
sunglasses
UK Domain Names
online fax service

Partners

Dream.In.Code

dotNet Slackers

dotNet Spider

Your HTML Source

VisualBuilder.com

DevGuru

Planet Source Code

ZVON.ORG

Web Design

ASPAlliance

XML Pitstop

Scripts

The Spot 4 SAP

Bitshop Web Hosting

Email TopXML

Front Page Daily Stuff TopXML Forum XML blogs XML Newsgroups BizTalk Biztalk Utilities Biztalk Utilities Tutorial B2B SAP XML Microsoft .NET Dotnet System XML Soapformatter SQLXML XMLserializer XQuery PHP PHP SimpleXML PHP XML Dom PHP XML RPC PHP XSLT Java Java Java XML Xalan Microsoft ASP ASP Schemas XML SQL Server XML XMLDom XSL XSL Tutorial XSLT Stylesheets General Javascript CSS XHTML WAP