CCE Theses and Dissertations

An Expandable Markov Model for the Design of Intelligent Communicative Agents in managed Health Care

Date of Award


Document Type


Degree Name

Doctor of Philosophy in Computer Science (CISD)


Graduate School of Computer and Information Sciences


Lee J. Leitner

Committee Member

Sumitra Mukherjee

Committee Member

S. Rollins Guild


In the field of medicine, decisions are often difficult to make in the absence of clear symptoms, decisive test results and adequate patient involvement. Medicine in most cases is still an art, using science for its basic foundation. Everyday practice relies on the case-study method, trial and error, and intuitive judgement. In some medical specialties there is heavy reliance on intangibles. For example, human motivation plays a significant role in complex medical decisions affected by many variables which remain unquantifiable and intangible; indeed most variables that determine outcome are hidden, including human motivation. The complexity of medical information systems demands that new ways be investigated that emphasize timeliness and efficiency. Medical information systems are no longer centralized; they are distributed over networks and the Internet. Interoperability has become a requirement in order for these heterogeneous systems be able to exchange information and work together in a cooperative manner. For this reason the design of decision processes within the general domains of medicine require further analyzation and establishing a methodology for developing a flexible agent architecture for the creation of intelligent agent systems in medicine.

Thus, this research provided the underlying theoretical framework for the design of interactive intelligent agents in the medical domain examining the design of open and flexible architectures. In the last decade rapid development of agent technologies has occurred. Research of multi-agent systems sprung from earlier works in artificial intelligence and decision sciences. Complementing the study of agent technologies is the discipline of mathematical modeling. Adapted Markov models were applied to facilitate the methodology of the study emphasizing the process by which real decisions are formalized rather than the solution to already formalized problems. Another important element of this dissertation was the use of clinical pathways; fundamental guidelines that are components of managed health care. Clinical pathways were at the core of the architecture and formed the basis of a suitable expandable and adaptive Markov model. The results of the model are a derivation of intelligent agent architecture for the medical domain.

The methodology exploited the generality, flexibility and normative power of Markov models, particularly, fully observable Markov decision processes (FOMDP) and partially observable Markov decision processes (POMDP). Based on both the FOMDP and POMDP, an expandable observable Markov decision process (EOMDP) model was formulated. The formulated model was further revised and reformulated based on the phenomenological observation and measure of clinical pathways. This approach is mathematically sound, computationally efficient, and intuitively appealing.

This document is currently not available here.

  Link to NovaCat