| ..... |  |
||
 |
..... | ||
| ..... |
KDI-99
The average person feels powerless in the modern world, with its global scale and fast pace. Many long for the good old days of small close-knit towns. In a small town, there was an expert to solve your problem, typified by the town doctor. The doctor served as Local Advisor, who tracked your personal situation, and Global Analyst, who tracked trends in broader society. His global knowledge came from careful analysis of global trends. He knew many other people across a wide range of experiences and could locate similar cases to convey in striking fashion. His local knowledge came from continual interaction with your personal situation. He knew your daily lifestyle, why you had the problem, and how you had been addressing it. He behaved like a personal trainer, who was coaching you how to change your lifestyle to get better. The promise of computing and communications technology is that the good old days will be preserved as the scale of community increases — as the number of people increases and the amount of information increases — to preserve the small town flavor. We propose a demonstration project to construct technologies for "lifestyle coaches", expert advisors for everyday life. A Personal Situation Trainer (PST) can be a virtual town doctor. Although interacted with by computer, it looks and acts like a person. It has local knowledge via continual interactions with you, in your home as part of your family. It has global knowledge via recording worldwide personal sessions into a database, which can be analyzed for global trends. Healthcare is, of course, only one instance of the utility of PSTs. Any domain where personal experience needs to be captured and compared to a global database is a ready target. It would work for a layman buying a new house or using new software, or for a scientist trying a new technique or using new databases. Our proposed project would develop generic technology for PSTs and deploy an initial prototype in clinical medicine, specifically home healthcare. The purpose of PSTs is to provide local advisors who interact with users in a personalized fashion and generate a national database as a by-product. This database of personal experience can be searched by global analysts who mine it for trends. Such technology is thus an ideal realization of the promise of Knowledge and Distributed Intelligence. Wide deployment of PSTs will bring a new level of expert knowledge to the average person. It will also enable large groups of disparate people, distributed across the Net, to work together effectively. Our goal is to develop a complete prototype of a Personal Situation Trainer and deploy this technology on an experimental basis for home healthcare. On the local (personal) side, the design goal is a natural conversation with a realistic advisor for interactive consultations. On the global (community) side, the design goal is to transform the user interactions into a structured database of personal experiences, which can be usefully matched to yield similar situations. These goals will be accomplished by a prototype system to transform user interactions with talking heads driven by domain questionnaires into structured records for community repositories. We have assembled a multidisciplinary team, with expertise in image processing, information retrieval, community databases, and personal healthcare. Over the 3-year grant period, we will develop the components, then work on an integrated demonstration of a PST for arthritis, to be tested in the lab with real patients on an experimental basis. The specific components of a PST are: create domain-specific questionnaires for personal experiences, interact with a virtual person with speech and emotion, infer questionnaire answers from user interaction on a personal basis, transform questionnaire answers into feature vectors, build a knowledge network of community repositories, and locate similar experiences using the knowledge network of personal experiences. The average person feels powerless in the modern world, with its global scale and fast pace. They simply don’t know how to find the information they need to solve their problems. Many long for the "good old days" of small towns and friendly neighbors. In a small town, there was an expert who could solve your problem, typified by the Town Doctor. He served as a Local Advisor, who tracked your personal situation closely. He had book knowledge, but his advice was tailored to your personal situation. He also served as a Global Analyst, who tracked trends in broader society. He knew others in similar situations and how they had dealt with their problems. So he explained his advice in terms of case studies. The reason the town doctor was so much more effective than the current interchangeable HMO physician is that he really knew what was happening with the world and with you. You listened to the Town Doctor because of his global knowledge (Authority) but you actually followed his advice because of his local knowledge (Sympathy). His global knowledge came from careful analysis of global trends. He knew many other people across a wide range of experiences and could locate similar cases to convey in striking fashion. You might be in terrible shape, but you felt better if you knew that someone else had been in a very similar situation and had dealt with it in the following manner. His local knowledge came from continual interaction with your personal situation. He knew your daily lifestyle, why you had the problem, and how you had been addressing it. He did not behave like an impersonal oracle, making stock pronouncements based on brief encounters. He behaved like a personal trainer, who was coaching you how to change your lifestyle to get better. The promise of computing and communications technology is that the good old days will be preserved as the scale increases — as the number of people increases and the amount of information increases -- that the small town flavor of community can be preserved. We propose a demonstration project to construct technologies for "lifestyle coaches", expert advisors for everyday life. A Personal Situation Trainer can be a Virtual Town Doctor. Although interacted with on a computer screen, it looks and acts like a person. It has local knowledge for sympathy, via daily interactions with you, in your home as part of your family. It has global knowledge for authority, via recording the personal sessions all over the world into a database, which can be analyzed for global trends. A Personal Situation Trainer (PST, pronounced "psst") is your lifestyle coach, who is always there when you need him. Personal Healthcare is the obvious choice to investigate PSTs. Health is a topic that touches everyone and home healthcare is ubiquitous. For example, health sites are the most widely used reason for accessing the Internet today. A virtual doctor sitting in your home that you can always talk to and who always knows similar cases would greatly aid the public good. Home healthcare is largely lifestyle coaching, which fits well with the notion of personal trainers. Healthcare is, of course, only one instance of the utility of PSTs. Any domain where personal experience needs to be captured and compared to a larger community or global database is a ready target. It would work just as well for a layman buying a home or using new software, or for a scientist trying an experimental technique or using new databases. Our proposed project focuses on developing generic technology for PSTs and deploying initial prototypes for personal coaches and community analysts in the domain of home healthcare and clinical medicine. The purpose of PSTs is to provide local advisors who interact with users in a personalized fashion and generate a national database as a by-product. This database of personal experience can be searched by global analysts who mine it for trends. Such technology is thus an ideal realization of the promise of Knowledge and Distributed Intelligence. Wide deployment of PSTs will bring a new level of expert knowledge to the average person. It will also enable large groups of disparate people, distributed across the Net, to work together effectively. The motivation for the PST project originally arose from Schatz’s experience as a PI in the NSF National Collaboratory program. This program at NSF laid the groundwork for the KDI program. Schatz’s project was the only flagship that built a national testbed of scientists interacting with multiple databases across the Internet, unique functionality in 1991 pre-Web. The molecular biologists that used the Worm Community System (WCS) could interactively search across multiple databases representing their community knowledge in a federated fashion. Building a national database that can be effectively searched requires author creation of the database entries. In WCS, the federated database was integrated with tight interlinks between components of objects like literature abstracts and gene descriptions. It proved difficult to accomplish this after the fact, e.g. to parse a gene name out of document text so that a link could be built. WCS was a symmetric system, where any object type could be interactively created. Correct useful links could be generated, by enabling the scientists to directly author documents within the system itself. But for the general public to use such technology, a more natural interaction style than forms entry is necessary to generate the database items. Our research, as part of the NSF Digital Libraries Initiative (DLI) project with Schatz as PI, has led us to a viable scalable semantics for knowledge networks. It relies on artificial intelligence techniques for parsing text phrases with limited domain semantics and on information retrieval techniques for using document context to record semantic relationships. We have processed collections of millions of documents with the statistical co-occurrence of concept spaces. We believe abstract spaces of concepts must be constructed over the concrete networks of objects to build the Interspace on top of the Internet. Then concept switching across community repositories will provide the necessary similarity matching for knowledge networks. Schatz has been in demand in the planning of the KDI program. For example, he gave the featured technical talk at the NSF Workshop in September 1997, hosted by the National Research Council, on Advancing the Public Interest through Knowledge and Distributed Intelligence. Schatz was the only member of the scientific community invited to give a technology talk on KDI to key officials from major foundations, in support of presentations by the entire NSF hierarchy. His talk emphasized that concept switching is the primary information infrastructure necessary to enable problem solving in the Net by the general public. As we enter the age of Knowledge and Distributed Intelligence, it is imperative to remember that successful Information Technology should be human-centered, not technology-centered. PI Schatz has joined with co-PI Huang specifically to construct significantly better user interfaces to his database indexing technology. Huang organized two Workshops for NSF in 1997 on Human-Centered Systems. The first Workshop concentrated on general issues including: data overload, communication and collaboration, human-centered design, and social informatics. The second Workshop was oriented toward the solution of national challenge problems, including healthcare, education, civil infrastructure, and digital government. Final reports and position papers of panelists are available at www.ifp.uiuc.edu/nsfhci . Our goal is to develop a complete prototype of a Personal Situation Trainer and deploy this technology on an experimental basis in the subject domain of home healthcare. On the local (personal) side, the design goal is a natural reassuring conversation with a realistic looking advisor who has medical knowledge useful for interactive discussions. On the global (community) side, the design goal is to transform the structured interactions into a database of experiences, which can be usefully matched for similar situations. The technologies are generic beyond the specific application. The design goal is to build community repositories that are detailed enough to effectively data-mine for trends. A complete system transforms the daily experiences of the users (authors) into the actual detailed records of the national structured repositories (databases). What is needed is for authors to directly produce detailed records, in appropriate structured database format, as part of their daily experiences. The focus of the natural interactions is in co-PI Huang’s lab, in collaboration with co-PI Berlin as medical domain expert. Huang is Co-Chair of the Human-Computer Intelligent Interaction major research theme in the Beckman Institute, 1 of 3 themes in the largest scientific interdisciplinary facility in the country. In this capacity, he is PI of large interdisciplinary research projects, including the Army Research Laboratory's Advanced and Interactive Display Consortium (in partnership with Rockwell) and a Yamaha Motor Corporation sponsored research project on "Computer Companion" in the travel domain. The research aim of the latter grant is to construct a personable computer who interacts with the human user through multiple modalities, and who has the ability to learn about the user and respond accordingly. The steps (research tasks) in the PST interaction process are as follows. Step 0. Create domain-specific questionnaires for personal experiences. These capture the daily lifestyle experiences and can generate structured records for the community repositories. This is the only non-generic step. We will base the interactive questionnaires on lifestyle questionnaires for chronic illness such as arthritis, a likely usage for daily diagnosis and treatment. For example, we will use the MOS 36-Item Short-Form Health Survey (SF-36) and the Arithritis Impact Measurement Scales 2 (AIMS2). Our proposed work will extend these static health status indicators for dynamic usage. For example, an adaptation for daily interaction might include: "Do you have stiffness in your knees? Is it greater or lesser than yesterday?" "Could you do your errands in the neighborhood today?" "Do you have stiffness in your hands? Is it greater or lesser than yesterday?" "Could you do your housework without help today?" Berlin will use his experience in establishing HMO self-care outcomes studies here. Step 1. Interact with a realistic virtual person with speech and emotion. The head of a real person is digitized using a high-resolution 3-D scanner. We will use our co-PI doctor as the model for the virtual doctor. The head scanner captures the ground truth of the surface and texture of the face. Huang’s lab has developed software that fits a generic head model to the scanner data and generates a realistic-looking 3-D image on the computer screen. The software can analyze speech and animate the head to speak naturally. Emotion can be superimposed as required. This can now realistically produce prepared phrases with natural reactions. We propose to extend this software to generate the range of speech and emotions necessary for a PST (lifestyle coach). Next, we need to gather the user’s (patient) reaction to the system’s (doctor) output. The user is sitting in front of a computer displaying the virtual person. There is a monitor-mounted camera filming the user’s motions. Huang’s lab has developed software that identifies key features of the face (eyes, nose, mouth) and can correctly track user motion of these features in real-time. This will enable the PST to react to personalized interactions for each user, once we build an interaction database and learn how to match into it. Instead of needing a generic interaction to detect happy/sad or truth/falsehood, we have a training session for each user where they react as specified and we record templates of their facial motions for these reactions. Step 2. Infer questionnaire answers from user interaction, on personal basis. The questionnaire is the central point of the interaction. Step 0 has created the template. This Step 2 walks through a heuristic traversal of the template, generating interactions for the doctor. Each interaction (speech and emotion) is then translated using Step 1 into motions by the virtual person. The heuristic traversal is similar to the heuristics for triad nurses, who answer patient queries by telephone when called. Berlin will supervise the medical soundness of these traversals. The traversal will be guided by the user reactions to the interaction. With static questionnaires, patient responses are often difficult to interpret. They say they are happy even though they are sad, or lie to appease the doctor. The personalized interactions can move beyond simple yes or no answers to infer degrees of satisfaction from facial expressions. We also plan to experiment with dynamically-chosen questions using rule-based approaches based on the situational answers, beyond the simple decision trees of the triad heuristics. Step 3. Transform questionnaire answers into feature vectors. The interactions must now be recorded into the community database. This Step concentrates on the collaboration between Berlin and Schatz, rather than between Huang and Berlin as for the above Step. The questionnaire is a specific set of questions based on diagnostic heuristics — the actual answers are tempered by the emotional inferences from the user’s facial expressions. The answers are transformed into a set of numbers representing the severity of each condition, e.g. pain in joints for chronic arthritis. These numbers are further transformed into feature vectors, which represent the current health of the person. Feature vectors are a representation for characterizing each database item in a structured fashion that can be matched to other items. To learn more about appropriate feature vectors for personal experiences, the proposed project will use vectors specialized to home healthcare. Most individual patient perceptions of their personal physiological health can be described within 10 categories, with 10 additional categories to describe their mental health. Schatz’s lab is also developing automatic clustering techniques for transforming text documents on arbitrary subjects into document vectors. Step 4. Build knowledge network of community repositories. The community repositories must now be maintained with the feature vectors from each user. Through daily interactions, each user implicitly builds a set of vectors that form their personal collections. Each community repository contains the collections of a group of related users, e.g. the local clinic contains the records of the patients it serves. The knowledge network consists of the set of all community repositories with the interconnections between these. The information infrastructure automatically performs semantic indexing to support one level of interconnections. The semantic indexing enables searching at the concept level and the network infrastructure supports concept switching across repositories. Schatz’s lab has developed unique software to semantically index community repositories and applied this technology to collections with millions of documents, in many subject domains including clinical medicine. The sessions of user interactions implicitly generate documents, so that the document indexing can be utilized, such as co-occurrence lists. In addition, the feature vectors provide a concise indexing scheme, which enables semantic clustering techniques to be utilized, such as self-organizing maps. Having a uniform representation for the vectors across repositories greatly facilitates concept switching, since it provides a normalization between disparate representations. Thus home healthcare is a good model domain for investigation. Step 5. Locate similar experiences using the knowledge network. Once the concept switching is in place, a demonstration of a functional PST can be provided by integrating together all the individual Steps as tasks. The user can talk to the virtual doctor to describe their health status, then have the doctor suggest treatments and describe other patients with similar conditions. This Step will compose all components into a prototype system. The investigators constitute a multidisciplinary team, with the appropriate sets of expertise to carry out the Personal Situation Trainer project and an established history of collaboration. PI Schatz is the Director of the Community Architectures for Network Information Systems (CANIS) Laboratory in the Graduate School of Library and Information Science. His laboratory has been prototyping the Interspace, with analysis environments for indexing community repositories and pattern searching across repositories. CANIS is housed in its own building on campus, with state-of-the-art computing and network equipment, supported by professional staff. Co-PI Huang is the Director of the Image Laboratory in the Beckman Institute. His laboratory has been prototyping Affective Computing, conversations with natural virtual people. The Image Laboratory was established under an NSF Research Infrastructure Grant and contains state-of-the-art equipment for multimodal and multimedia information processing and visualization. A major research emphasis is integrating audio and vision in various applications, e.g. combining speech and vision-based gesture analysis to control virtual environments. Co-PI Berlin is Medical Director of HealthAlliance, the HMO that dominates south-central Illinois. His responsibility is establishing treatment guidelines for clinic physicians and management of medical services and outcome studies. As Medical Director of an HMO with 200,000 members in rural Illinois, he is well placed to understand the needs for home healthcare. Schatz is an experienced NSF PI in multidisciplinary projects. He has done several projects with Huang, including a major neuroscience application in brain-mapping with a similar databases/images responsibility split. Huang also had a supplementary grant on the DLI grant, on which Schatz was PI. Berlin has visited Schatz at CANIS weekly for a year to plan projects in information infrastructure relevant to practical healthcare in today’s world. Over the 3-year grant, we will be developing the components, then working on an integrated demonstration of a PST for arthritis. Component responsibility is: Step 0 (Berlin), Step 1 (Huang), Step 2 (Huang/Berlin), Step 3 (Schatz/Berlin), Step 4 (Schatz), Step 5 (all, Schatz coordinating). For personal interactions, we will use actual patients on an experimental basis, provided by Berlin’s practice and interacted with in Huang’s lab. To demonstrate community database matching, we will use patient records with the interactions simulated by hand. We will use the medical literature from all of MEDLINE as a large global source of patient interactions.
|
||
 |
..... | ........ | |
