Custom Tailored: Personalizing the Web experience
Beth Ann Mardekian
Fall, 2000
Table of Contents
Abstract
Gone
Surfin
Virtual
Markets
Personalizing
Personalization
Discussion
Goals
Defining
Personalization
Types
of Personalization
Personalization
Packages
Personalized
Technology
In
the Beginning: Defining the
users
Mentors
Filtering
Techniques
Serving
Personalized Content
Business
View of Personalization
Considerations
Using
Metrics
Misconceptions
Users
View of Personalization
Custom
Tools
Seeking
Something?
Privacy
Please!
My
Yahoo!: A case study
Personalizing
the Future
Appendix
I: Sample User Survey
References
The World Wide Web is fast becoming the central location for goods,
service, and information. The
population of the Web is also increasing in diversity as well.
No longer is this technology for military personal, college professors,
and college students. Content
personalization individualizes Web experiences; just as the online population is
becoming more diverse, so too is the type of content that needs to be displayed.
Responses to surveys and data collected while navigating through a
website are compiled and analyzed to present content which exhibits the same (or
similar) characteristics. The same
content does not and cannot apply to everyone.
Content personalization is also useful for helping businesses evaluate
their websites design. User
preferences drive the content. Personalization
is the key to future Web success.
Need to do research for a term paper?
Try an online information archive. Looking
for a new outfit? Try one of the
many online clothing merchants. Looking
to finance or lease a new car? Visit
the automakers at their official sites to customize a model, locate a dealer,
and/or negotiate a price. Want to
trade some stock? Want to buy a
music CD? How about rent an
apartment or get a second mortgage? Or
how to get an Elvis impersonator to appear at the next company picnic?
Find it all online.
World Wide Web
technology is at the forefront of the burgeoning communication revolution.
What once seemed reality only in science fiction has now become the focal
point of todays society. No
longer is there an exclusive dependence upon POTSplain old telephone
serviceto be the prevalent link to products, services, and the rest of the
outside world. Many have migrated
towards accessing such resources through the use of various electronic devices;
among the most widely used are personal computers, cellular phones, and handheld
personal data assistants. What
needed to be accessed a few years ago via telephone from ones residence can
now be done from almost anywhere, anytime.
According to the
1999 State of the Internet Survey by USIC (United States Internet
Council), From 1992 until now, a year in the development of the Internet is
likened to five to ten years of evolution of other media.
The backbone of the Internet now doubles in capacity every 100 days
(1). According to a Network
Wizards survey discussed in the same commentary, as of January 1999 the Internet
has grown to over 43 million hosts worldwide; it is projected that the number of
hosts can grow to 100 million by 2001 (Figure 1).
A host is considered any computer that is reachable through the Internet;
it cannot be behind a firewall. The
host must have a unique address and have the capacity of providing information
in addition to access (USIC 3).
As
information becomes more and more abundantly available with the click of a
mouse, it is becoming vital that site masters design content to reflect the
users individual needs. No
longer is the Web exclusively surfed by research scientists, military personnel,
and college students.
The Webs
constituents have expanded to include senior citizens, at-home mothers, and many
other age, ethnic, race, and economic classes.
Figure 2 represents the breakdown of ethnic group use for the year 1999
with a projection for the year 2000. Such
diversity in Web user demographics is case and point for the vast disparity in
the range of tastes, comprehension, and/or level of computer proficiency.
In many instances offering the same content to all visitors does not
suffice. There is little guarantee
that the information presented will meet everyones needs; although, this may
have sufficed during the earliest days of the Internet when the users shared
more similarities. Users, even the
most casual, need to be assured that the site has all the information they
need, and that they are likely to retrieve that information.
Users do not want to waste time hunting or wading through an ocean of
material (Hysell 167).
One cannot assume
that the guests who take advantage of the Web's resources today and tomorrow are
able to properly navigate the site's interface to find the information,
products, or services they are seeking. Chances strongly favor that a sampling
of Web surfers are NOT well versed in any of these methods. A perceived loss of
control may cause the user to resist the technology. Instead of Internet usage
growth, there may be a regression in the use of available resources.
As
US and worldwide use of the Internet continues to boom, it is also expected that
electronic commerce activity will flourish.
International Data Corp (IDC) estimates that the worldwide volume (in US
dollars) of business-to-business (b-to-b) electronic commerce in 1998 registered
$27.4 billion with volume growth projected at $978.4 billion for 2003.
IDC also estimates that the worldwide volume (in US dollars) of
business-to-consumer (b-to-c) sales will rocket to $177.7 billion by 2003. In 1998, $31 billion in b-to-c sales transactions occurred
with $50.7 billion projected for the year 2000 (USIC 12-13).
In order to sustain or exceed such projected levels of growth, there
needs to be a method of insuring that consumers, whether they be other
businesses or private patrons, are satisfied in their efforts to find such goods
and services.
Content
personalization may be the key to meeting the individual needs of those who
currently surf the Web and attract those who have yet to catch its wave.
Conceptually, accommodating the individuality of a sites visitors
seems trite. Give the visitors what they want and they are happy.
Realistically, personalization is not childs play.
It is quite difficult to determine what it is that the individuals are
seeking. Since there is a wide
range of personal tastes that need to be considered, how can the content of a
site address all of those differences? The
ensuing discussion will discuss the following topics in effort to shed light on
the many facets of content personalization and what it all means for the future
of websites and their design:
Definition of personalization:
What content personalization is and what it all means
Types of personalization: There
are four types of personalization ranging in the most simple (name recognition)
to the most complex (preference based) to implement.
Technology behind personalization:
How does personalization do its thing?
Business aspects of personalization:
What businesses need to consider.
User aspects of personalization:
What users need to consider. Includes
a discussion of privacy issues.
Real world examples of personalization:
Some efforts to date.
Personalizations future: Is
there one? If so, what does it look like?
Determining an
exact definition for personalization is a bit of a conundrum.
Personalization tends to be defined according to how it is implemented by
various organizations and individuals. Kramer
et al write:
Personalization is a toolbox of technologies and
application features used in the design of an end-user experience.
Features classified as personalization are wide-ranging, from
simple display of the end-users name on a Web page, to complex catalog
navigation and product customization based on deep models of users needs and
behaviors. (45).
Personalization.com, a website
dedicated to creating a clearinghouse of objective information about [Web]
personalization, offers the following definition:
As used on the Internet, the term
Personalization [sic] has come to mean person-specific content.
Personalized content may be advertising, items for sale, screen layout,
menus, news articles, or anything else we see via the Internet.
Personalization is the result of
technology integrated into a website that allows the server to modify what is
presented to each viewer. With
personalization technology working, two individuals accessing the same website
simultaneously may see two completely different sets of information. (Personalization.com
FAQs)
Many ponder
whether or not there is indeed opportunity behind personalizations hype.
Riecken writes in Personalized Views of Personalization:
I suggest that
personalization is not a silver bullet, but instead is part of the following
prime directive for business: Give
the customer a high-quality product or service they really need and can use at
the best (lowest) price, and give the customer high-quality service with
integrity. Do this and the result
will be successful corporate branding and customer loyalty.
Simply stated from
one point of view, personalization is about building customer loyalty by
building a meaningful one-to-one relationship; by understanding the needs of
each individual and helping satisfy a goal that efficiently and knowledgeably
addresses each individuals need in a given context. To extend this point, it is about the mapping and satisfying
of a users/customers goal in a specific context with a
services/businesss goal in its respective context.
Clearly, this is a difficult problem. (27)
Belkin adds that personalization is an important method
needed to help individuals find what they do not know.
When people engage in information-seeking behavior, its usually
because they are hoping to resolve some problem, or achieve some goal, for which
their current state of knowledge is inadequate (Belkin 60).
There are four major forms in which content
personalization can be found (as outlined by Personalization.com).
These methods are not exclusive and can, and often do, coexist.
Some are simpler to implement than others.
In many cases the concepts of one are built upon and made more robust in
another.
Name Recognition: When
a user starts a session either by logging in to the site or by simply returning
to a page that has been previously visited (through the use of session tracking
technology, or cookies), they are addressed by the name that the system knows
them as (e.g., login name, first name, etc.).
Most people like to be acknowledged by nameit tends to give the notion
that that individual is not just another number and that they are important.
This is the simplest of all to implement.
Check-box: In this
case, information is provided by the user.
Questionnaires, surveys, registration forms, and other solicitation
methods are used to gather information about the users likes, dislikes, and
any other factors that can help paint a picture of that individual.
For example, a registration form may ask a user from which vendor the
item was purchased, where the item will be used (home, office, gift, etc.), and
if the user has purchased this item as an upgrade or replacement.
This information is used to custom tailor content based on the users
responses.
Segmentation and Rules: Demographic,
geographic, psychographic profiling, or other methods of information collection
are used to divide or segment the entire user population into smaller groups, or
pools. Data such as income level,
geographic location, and buying history is aggregated and processed and the
results divvy the users into appropriate groups. Content is then dished out according to if this, then
that rules processing.
Preference-based (Affinity):
This is perhaps the most complicated of the four forms to implement. The code behind the scenes of the site needs to be smart and
adapt quickly and smoothly to changes in the population; thus these systems are
usually updated in real time. Preference-based
personalization attempts to comprehend a users affinity for certain items,
goods, or services based on previous behavior of not only that user but also
similar users. Complex statistical
algorithms are needed in order to make the most accurate predictions as
possible. Resulting is a profile of
the user and a set of predictions matching what the user would (possibly) want
to view or buy next.
Due to the complex nature of content personalization, in-house
development of such software is not a normal occurrence.
There are several personalization packages on the market; many are
frameworks that require further customization for each environment in which they
are employed. Such products include
Allaires Cold Fusion (allaire.com), Black Pearls Knowledge
Broker (blackpearl.com), Macromedias LikeMinds Personalization Server
(macromedia.com), and NetPerceptions Recommendation Engine Suite (netperceptions.com).
These products, as well as many others available on the market, encompass
a wide range of functionality, expandability, and ease of use.
Some are easier to implement than others and some come at a much cheaper
price than others. The following
discussion will outline concepts general to personalizations technology.
There is a wide range of capability that each of these products has to
offer; many of these products are so complex that they are worthy of their own
discussion. For many, the
documentation consists of several volumes.
Visit http://www.personalization.com/resources/vendors/ for descriptions
of the various personalization products available and links to vendors sites.
Websites have
become central repositories of information for many products and services around
the world (Fuccella 69). An
issue that plagues site architects is how to serve the right content to users
who have diverse sets of tastes, values, wants, and needs.
Also to be considered is how to turn users into loyal followers of a
site. Coaxing them to hit the site
is not nearly as difficult as retaining them.
It is difficult for site architects to predict the type of content the
target audience(s) is seeking when there is little known about them (other than
who they are). A designer is incapable of perceiving exactly what each
individual is seeking when he or she visits a site.
As with any
disorganized assortment of toolsespecially fascinating new toolsdesigners
are often drawn into the trap of trying to find uses for the tools, and
deploying the coolest new features, forgetting the primary focus should be on
providing value to the end user (Kramer 45).
In order to implement the technology behind personalization the designers
must develop a crisp audience definition (Fuccella 69). Marketing departments determine who the user group is, but it
is the designers collective responsibility to paint the accurate picture of
just who the real user group is and the characteristics they embody.
This is difficult as the designers must figure out how to deliver the
sites content to the user and make sure that the organization achieves its
goals. The way to do so is to
compile profiles of the actual users who hit the site and learn about them from
them. Thus it is necessary to
constantly probe the user pool.
Typically
individual user profiles contain both demographic and transaction data.
Demographic data describes who the user isgender, birth date,
education, salary, type of music listened to, favorite stores, etc. (Adomavicius
377). Generally speaking, this
includes anything that can outline an individuals likes, dislikes, and
values. Usually this data is
collected using surveys, e.g., check boxes, fill-in.
Often this data is collected with the initial visit/login to a particular
site and becomes a permanent part of the users profile.
Other demographic data may be collected as needed or inferred from the
types of goods, services, or information that a user selects or purchases.
Transaction data describes what the user has done while navigating
through the site (Adomavicius 377). For
example, clickstream data reflects a particular users travels around the site
(i.e., what a user has selected or clicked on with his or her mouse).
It gives an indication of what types of goods, services, or information a
user decided to explore.
One of the key
technical issues in developing personalization applications is the problem of
how to construct accurate and comprehensive profiles of individual[s] that
provide the most important information describing who the customers are and how
they behave (Adomavicius 377). Task
analysis methods are employed on the profiles to learn the impetus for
users actions . . ., methods of completing the tasks . . ., and the ultimate
intention of the user . . . (Kramer 46).
Complex statistical algorithms are used to sift through the data
compilations. Not only are these
algorithms intended to classify the individual users, they are intended to
compare and contrast behavior patterns of different users.
The results depict what a typical, generic user prefers.
These results serve a dual purpose.
The results of the profiling effort are reused to determine what content
the personalization engine will serve tot eh users.
They are also reused by those who evaluate the sites design and
content to determine if and where changes need to be made.
The statistical
algorithms that are used consist of formulas made up of combinatorics, weighted
properties, and rates of decay/half-life. Combinatorics
principles make comparisons among the items (products, services, pieces of
information, etc.) that the site offers. The
results give an understanding of any possible correlations between one item and
another. For example, those who
purchased sneakers also bought baseball hats.
Or, those who drive Ford Mustangs wear leather jackets.
Weighted items place more precedence on a particular item and less
importance on others. For example,
a stapler may be considered a key purchase item and carry more weight while
staples may be considered more minor (because in order to need staples, one must
first have a stapler). The rate of
decal/half-life calculation places a higher value on current transactions and
relies upon older transactions for historical purposes.
Generally this formula considers the weight a particular item carries and
decays its worth relative to the date and time it appears in a users profile.
Older transactions tend to not be considered as valid as younger ones
because a users tastes and preferences may have changed during the time
between transactions. These
formulas are relatively complex; it is better for purposes of this discussion to
leave the discussion of the mathematical formulas as such.
With time, and as
more data is collected, the profiles become more and more accurate.
Thus as the data is sifted the outcomes are also more on target.
All systems suffer from what is referred to as a cold-start
problem. Users start off with nothing in their profile and must
train a profile from scratch. . . .
[There is] a training period before the profile accurately reflects the users
preferences. During the training
period the system cant effectively filter for the user (Maltz 203).
This is understandable. With
only a sketchy picture of a user, it is difficult to make recommendations or
serve appropriate content. The
system has to learn as the user provides more and more information through
interaction with the site over time.
Mentors are drawn
from the user pool. These are users
who are considered to have the most experience with visiting the site.
They have registered more transactionspurchases, item views,
etc.than other users. Thus their
profiles are considered more robust and offer a more accurate portrayal of a
visitor to the site. The purpose is
to aid the personalization engine in serving content that more accurately
reflects a particular users tastes. Other
users, then, can be grouped in with these super users.
Comparisons are done (using filtering techniques, etc.) to lump users
together. The rules for a match are
determined by the organization; generally the more similar the profiles are, the
better the match. A user can be
assigned different mentors as his or her tastes change, more transactions are
registered, or a better mentor match emerges.
All of this is done behind the scenes and the users are almost never
aware that they have been assigned a mentor.
Mentor assignment
is often discussed with filtering.
Content and
collaborative filtering are the two major types that content personalization
engines tend to use (Balabanovic 377-378).
Content based filtering ultimately sifts through the processed profiles
and dishes recommendations based on some analysis of the profiles content.
Keywords or elements found in the users profiles are matched with site
content which contains the same or similar keywords.
Collaborative
filtering is the more complex of the two filtering techniques.
Mentor sets are often used here. The
basic premise is that people looking for information should be able to make
use of what others have already found and evaluated (Maltz 202).
The past results of others are used to calculate future content.
In other words, usage history is used to determine which items have
garnered the most user affinity. It
is those items that become marked and eventually displayed to the user.
Balabanovic points out that online content
personalization is a three-stage process. These
steps are not a one-time effort. They
are a part of a constantly intense, ongoing effort to present the most accurate
content to users.
Collection: First collect the items to be recommended.
Selection: Next
select from the collected items those best for a particular user.
Delivery: Finally
deliver the selected items to the user.
(Balabanovic 378)
These steps occur once the
profiles have been established and the users have been placed into any groups,
assigned to mentors, etc. This
phase begins by matching the sites various pieces of content to the users
preferences; usually this is done by matching keywords or descriptions.
From the collection of content, the personalization engine selects the
ones that best match the users profile.
For example, if a user profile states that the individual prefers
baseball hats, jeans, and T-shirts and does not like classical music, content
associated with formal dinner wear may not be displayed.
Once the appropriate content is determined, the selected items are
displayed on the website for the user to search, view, purchase, and the like.
What technology actually displays the content?
The content is often displayed using existing Web page development tools
such as HTML, XML, and Javascript. XML
has become of particular interest due to its dynamic nature and ability to adapt
to different types of content. Displaying
the content is relatively easy as compared to the background processing that
needs to be done to determine what content to display.
Discussed in the next section is the role that data collection takes on
the business end of things. The
data that is collected from the user helps the personalization engine become
smarter about the user when considering the rules, constraints, or formulas that
have been provided. Yet the data
contained in the user profiles come sin handy for the site designers as they can
evaluate the results and manually determine if the layout of the content (or
even the content itself) is having a positive impact on the sites visitors.
If the results show that content is not having appositive impact, the
designers have the ability to make adjustments to the sites design, how they
rate items, how they present the items, and many more business factors.
Before an organization dives in headfirst and implements content
personalization on their website, there are a few considerations that have to be
taken into account. Implementing
personalization is unique to each organizations situation.
There is no single solution available that will address the needs of all
organizations. Each business
contemplating employing a personalization solution for its website should be
sure to:
1) analyze
the business and determines the function of the website to the business;
2) plan how personalization will be used to enhance the site;
3) implement
the personalization solution by comparing and selecting the best personalization
technology provider for the situation; and
4) evaluate
the integrated personalization solution for performance, refinement, and return
on investment. (Personalization.com
FAQs)
Once a business does decide upon the appropriate personalization package
to implement, they can begin their efforts to build up and maintain a loyal
clientele. Personalization offers
an almost infinite supply of data about visitors to a website.
It is in the best interest of the organization to take the data that they
have obtained and use it effectively to reach out and meet individuals needs.
In the Internet environment,
products and services are constantly in danger of becoming commodities, shoppers
can explore competing Web sites without leaving their chairs, and bots and
agents make comparison shopping almost effortless.
Data serves two important functions.
First, it becomes possible to nurture loyalty by analyzing information
learned about customers over many visits. Secondly,
e-business intelligence, which aggregates data over many customers, allows
managers to evaluate how effective their user interface is, and continually
improve the site based on measurement feedback to keep the visitors on the site
longer. (Schonberg 53-54)
To measure success, it is important to understand what success means (Schonberg
54). An organization
cannot blindly hope to implement personalization, sell a bunch of their
services, and make lots of money. There
must be a series of clearly defined business goals from which the content
personalization engine will draw. Among
those goals is a definition of the type(s) of visitors that the site will
target, what the sites content is going to convey to the user, and what the
visitor can accomplish by visiting the site.
For example, a website may be established as a non-bias information
resource for users interested in purchasing a new or used automobile.
Or, the business goal may be to sell canoes and kayaks.
Once the goals are firmly outlined, then the technology can be
implemented.
An organization will need to utilize the client profile data that is
collected by the personalization engine as well as data from other sources to
refine their business goals and definition of the target audience. While the personalization engine adheres to formulas to serve
page content to visitors, it is the business that ultimately controls what that
site content is and how it is organized.
Metrics are important tools to determine if the content that is being
served meets the various needs of the visitors.
The measures are mathematical formulas which reflect the organizations
goals and marketing strategy. Two
categories of metrics are clickthrough and look-to-buy.
Clickthrough data measures the ratio of clicks to impressions, where
an impression is simply the display of [some component] on a Web page (Schonberg
54). Look-to-buy metrics measure
the effectiveness of a variety of design and merchandizing features,
promotions, and product displays within the Web site.
Rather than viewing a Web site as pages and hyperlinks, the Web site is
decomposed into a collection of component features, each with specific
measurable goals (Schonberg 54). Figure
3 is an example of a typical scatter plot which is often done to compare results
from look-to-buy data. Each
product, service, or piece information (depending upon the business goals) is
assigned a color and then plotted to show a comparison of product impressions
(X-axis) versus purchases (Y-axis). Clusters
denote areas with the same or similar activity.
Both of these
measures make use of clickstream technology to record a users movement around
the site. Such data is integrated
with other sources, including: call
center information, surveys, ad banner hits, and sales data:
to glean the overall
effectiveness of the site can be viewed from the perspective of the Web owner
and the perspective of the Web visitor. From
the business perspective, metrics may suggest where improvements can be made with regard to design, layout, and navigation
issues. Metrics can also be used to create visualizations that demonstrate
visitors behaviors in ways that may otherwise be missed.
For example, the number of times a product appears on the site compared
to the number of times people actually bought a product. (Schonberg 56)
In order to
remain competitive, an organization should regularly mine the data (as the
amount of data available will grow immensely with time) and determine if and
where any improvements to the site are needed.
Perhaps there were too many steps needed to arrive at a particular
catalog item; or, the user became confused or lost and needed to start the
series of steps over or simply gave up and went to another site.
The eBusiness needs to be resilient enough to note such behavior and
remedy it before further damage is done. In
the process, the organization may make discoveries that they did not know
existed; there could be certain product correlations that exist that otherwise
would go undetected. The more that
the organization knows the customer, the better the service should be.
Ultimately, loyalty results from the investment that a customer makes in
educating the business about him or herself and that a business makes in
learning about the customer (Schonberg 56).
Personalization is not a 1-2-3 plug-n-play solution to instant website
success. There is a significant
amount of monetary and time effort that must be expended into achieving the
intended goals. Content
personalization packages are not cheapthe initial cost of implementation can
be upwards of $50,000. Maintenance,
because it needs to be ongoing, can also cost a pretty penny.
Before an organization implements a personalization package, they must
sit down and outline what it is that they want to achieve.
This author has
personally witnessed organizations who jumped on the personalization bandwagon,
spent the $30,000 for the software, spent another $20,000 for consultants to
install and configure the package, and not have a clue as to what it is that
they purchased. These organizations
definitely counted their chickens before they hatched.
From what location do you
primarily access the Web?
a.
Home
c.
Mobile Environment
d.
Other
To which age group do you belong?
a.
under 18
c.
26-32
d.
33+
What is the highest level of education you have completed?
a. Grade
school
b.
Some High School
c.
High School Graduate
d.
Some College
e.
College Graduate
f. Some Graduate School
g.
Graduate School Degree
What type of music do you listen to (check all that apply)?:
Adult Contemporary (Yanni)
Classical (Mozart, Strauss)
Classic Rock (Jimi Hendrix, Pink Floyd)
Hard Rock/Heavy Metal (Sepultura, Pearl Jam)
Rap/Hip-Hop/R&B (Jay-Z, Wu-Tang Clan)
Religious
Techno/Dance (Fat Boy Slim, Webster Hall Productions)
Where do you normally purchase your music (check all that apply)?:
Mall (Sam Goody, Record Town)
Online (CDNow.com)
Individual Retailer (Tower Records, Virgin)
Other: _____________________________
I dont buy music, I download from Napster