Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

DeepDyve requires Javascript to function. Please enable Javascript on your browser to continue.

Foundations of Average-Cost Nonhomogeneous Controlled Markov Chains

Part of the SpringerBriefs in Electrical and Computer Engineering Book Series

DOI: 10.1007/978-3-030-56678-4

ISBN: 978-3-030-56677-7

Publisher: Springer International Publishing - Springer Journals

Sep 10, 2020

Available

Introduction

Cao, Xi-Ren

Pages: 1 –12

[This book deals with the performance optimization problem related to the discrete-time discrete-state time-nonhomogeneous Markov chains (TNHMCs), in which the state spaces, transition probabilities, and reward functions depend on the time. There are two main issues: First, notions such as...
Available

Confluencity and State Classification

Cao, Xi-Ren

Pages: 13 –27

[In this chapter, we introduce the notion of confluencity and show that with confluencity, the states of a time-nonhomogeneous Markov chain can be classified into different classes of confluent states and branching states. Confluent and branching are similar to but different from recurrent and...
Available

Optimization of Average Rewards and Bias: Single Class

Cao, Xi-Ren

Pages: 29 –58

[In this chapter, we study the optimization of the long-run average and bias of single class (or uni-chain) time-nonhomogeneous Markov chains. With confluencity, we define the most central notion in performance optimization, the performance potentials, and discuss its properties. With the...
Available

Optimization of Average Rewards: Multi-Chains

Cao, Xi-Ren

Pages: 59 –78

[In this chapter, we study the optimization of the long-run average of multi-class time-nonhomogeneous Markov chains (TNHMCs). We show that with confluencity, state classification, and relative optimization, we can obtain the necessary and sufficient conditions for optimal policies of the average...
Available

The Nth-Bias and Blackwell Optimality

Cao, Xi-Ren

Pages: 79 –108

[In a Markov chain, an \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$(...

results per page

Browse All eBooks

Advanced Search

Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

Foundations of Average-Cost Nonhomogeneous Controlled Markov Chains

Our policy towards the use of cookies