n-Person Prisoner’s Dilemma Game and Bacterial Pattern Formation: A Finite Automata Modelling

n-Person Prisoner’s Dilemma Game and Bacterial Pattern Formation: A Finite Automata Modelling

n-Person Prisoners Dilemma Game and Bacterial Pattern Formation: A Finite Automata Modelling

Gaurav Srivastava1, Sudeepto Bhattacharya2*

1IIIT Allahabad, Deoghat, Jhalwa, Allahabad 211012, Uttar Pradesh, India.

2Department of Mathematics, School of Natural Sciences, Shiv Nadar University,

Chithera 203 207, District GautamBudh Nagar, Uttar Pradesh, India.

ABSTRACT

Bacterial colonies in nature are often required to evolve under harsh and hostile environmental growth conditions. In order to do so, bacteria work as a social formation and employ intricate communication capabilities to exchange information and interact cooperatively to form highly complex colonies, equipped with higher capabilities for adaptation to the environmental challenges. These colonies that could be observed as intricate spatial patterns are essentially the manifestation of bacterial self-organization resulting from such cooperative behaviour. The information required by the bacteria for giving rise to the observed self-organized complex pattern formation is generated through cooperative interactions, depending on and in response to the available growth conditions. Bacterial self-organization and colony formation, thus, appears as an instance of a social network, essentially created through communicative interactions.

In this paper, we use concepts from game theory and informatics to describe the emergence of self-organization and consequent pattern formation through communicative cooperation in Bacillus subtilis colonies. The emergence of cooperative regime is modelled as an n- personPrisoners Dilemma game, with the bacterial colonies as individual players. The game is played iteratively through cooperative communication, and mediated by exchange of information about the local environment between the different bacterial colonies comprising the system. The iteration causes the interactive system to grow and produce beautiful complex spatial patterns signaling the emergence of self-organization.

As a formal description of the above game, we model the emergence of this cooperative behaviour as finite deterministic automata, whose transition function is informed by the game pay-off.

Key words:Cooperation, Emergence, Pattern formation, n- person Prisoners Dilemma game, Deterministic finite automata, Grammar.

1. INTRODUCTION

   Interacting groups of motile organisms form complex systems that offer a rich repository for studying and learning about various interesting facets of complexity. These organisms may exhibit characteristic patterns of self-organized collective behaviour, forming spatial aggregation. Microorganisms communicate and cooperate to perform a wide range of multicellular behaviours such as dispersal, nutrient acquisition, bio-film formation and quorum sensing [1, 2, 3].

   In this paper, we study an interacting colony of bacteria as a typification of complexity. A single agent can be treated as a complex molecule, which interacts with its environment, communicates with other individuals, replicates and undergoes evolution and mutation. These interactions include competition for nutrients, cooperation by providing public goods essential for the formation and maintenance of the biofilm, communication through the secretion and detection of extracellular substances [4]. When millions of bacteria act collectively as a group (as is the case in any bacterial colony), a large variety of different, fascinating spatial patterns emerge as a result of games played by the individual bacteria colony with its neighbours as well as with the environment. These patterns are the emergent result of local interactions (interactions within the neighbourhood of each individual colony) through exchange and processing of information over discrete time, and environmental conditions. The self-organization in the colonies that is manifest as the emergent phenomenon can be viewed as being programmed by both the particular way the bacteria in question interact, as well as by the particular way these bacteria respond to environmental signals [5, 6, 7, 30, 31, 32, 33].

   In nature, bacterial colonies may often experience hostile, challenging environmental conditions that may not offer optimal factors for growth and evolution of the colonies. Evolving populations of bacteria (and therefore bacterial colonies) carry and mediate information via games played by them locally. Each colony would receive a finite set of input information or signals from its neighbours (and therefore, as well as the environment) in the form of the strategies played by the neighbouring colony in the game. This input set is the determinant for the strategies that the focal colony would play in response. Strategies in the game would decide the pay-offs for the colonies (the players) playing the game. The pay-offs would determine the transition of the focal colony to the next state in discrete time step, producing a final set of states (the next generation). The behaviour of bacterial colonies could therefore be mimicked by finite state machines.

   To cope with environmental stress, these aggregations of colonies or finite state machines which essentially form the complex system, through the mechanism of information processing develop extensive but complex cooperative behaviour. These cooperative behaviours could be modelled as pay-offs of non-cooperative games played between the colonies mentioned in the above paragraph, to share available resources usefully via transmission of information between them, and evolve [30, 31, 32]. These stressed bacterial colonies exhibit emergence and self- organization, two of the essential signatures of a complex system, and produce fascinating

   spatially complex patterns of aggregations, manifesting complexity of cellular level informatics [1, 8, 9, 10, 11, 12].

   Our objective in this paper is to model the cooperative behaviour of such stressed bacterial colonies which emerges as a collective manifestation of information processing by the interacting bacterial colonies, by using deterministic finite automata (DFA) [21, 22, 23, 24, 25].

2. PRELIMINARIES AND RESEARCH QUESTION

   The growth of bacterial colonies in harsh, challenging environment could be modelled as an instance of management of commons through collective behaviour. In the present context, such a management regime addresses three very critical issues: producing a public good by wetting the surface hardened by chemical stress, regulating proper appropriation of the limited nutrient by the growing colonies and preventing public bad by avoiding the effect of antibiotic-induced stress. Such a regime may encourage emergence of cooperation among the colonies competing for the constrained resources for growth. Thus the situation as is in this work, may be viewed as a problem of resource appropriation through coordination among the colonies.

   We apply n-person Prisoners Dilemma game together with the theory of formal language to model the above bacterial behaviour leading to coordinated cooperation among the colonies. The spatial patterns observed are the manifestation of emergent phenomena, which are in turn the consequence of essentially non-linear interactions among the individual bacteria colonies, as well as with their immediate environment [22, 23, 34].

   Then-person Prisoners Dilemma is a multi-player version of the familiar n=2-person Prisoners Dilemma and is anappropriate mathematical method for modelling collective behaviour [16, 17, 18, 19, 20]. Inthis form of the game, each of the n participants, who we shall refer to as agents, has a strategy set, comprising two pure strategies: cooperate (C), and defect (D). By choosing to play C, agents cooperate with each other for the common good, while a choice of D results in withholding cooperation, following individual short-term interests. Each player receives a payoff as a consequence of its choice of strategy from the strategy set the strategy chosen could be a pure one as also a mixed one, where the two pure strategies are randomly mixed. The payoff received by an individual agent depends on its own choice of strategy as well as on the choice of its co-players. The game is iterated over time-steps for a countable number of steps to produce the emergence of complex spatial patterns of bacterial colonies.

   For the purpose of modelling in this work, we assume that the game is played without any institutional arrangement to oversee as a central authority, and further that it progresses by exchange of information between the neighbouring colonies of bacteria. Each bacterial colony is an agent in this modeland at a given state, receives a finite number of input information from its neighbour at each time step, and makes a transition accordingly to an unambiguously determined next state at the next time step. With this assumption, we design deterministic finite automata (DFA) to model the discrete growth dynamics of the stressed bacterial colonies [13, 14, 15]. We propose that the colonies exchange information using a context free grammar, which is obtained via the automata.

   With the above agenda, we pose our research problem as:

   What are the DFA and the corresponding grammar that model the discrete-time growth dynamics and pattern formation in a system of stressed bacterial colonies engaged in an n- person Prisoners Dilemma game?

   To address the problem, we make the following assumptions[5]:

   1. The environment is a two-dimensional matrix, which represents the medium. The resources necessary for growth of bacterial population are locally (within a neighbourhood) constrained.

   2. Growth of bacterial colony is an orderly increase in the quantity of bacterial constituents.

   3. The agents fitness, which we define as a linear function of the pay-off received by the agent in a game, is the determinant of the agents growth.

   4. Availability of an appropriate biochemical and biophysical environment is the necessary condition for the bacteria to propagate. The biochemical environment is the environment of the nutrition, and is made available as the culture medium; the biophysical environment is supplied by the agar concentration that forms the substrate for the bacteria population to grow.

   5. In a single time step each agent consumes a fixed amount of nutrients to grow.

   6. The ability to reproduce depends on the amount of free space available in the agentneighbourhood.

   7. The game provides the pay-off to eachagent for computing its fitness, and growth determined by the fitness, is contingent upon the game played by the agent at the instant of discrete time.

Let an n-person Prisoners Dilemma game in the strategic form be given by G, , , where

{i }is the set of players, with i {1,2,…, n} a finite index set and n 2 , {i } where

i is the pure strategy set for each playeri , with 1 , 2 ,…, n where i i i is a

pure strategy profile of the game and

{i }, the set of pay-off functions

i : S i

where S is the set of pure strategy profiles, give the players von Neumann-Morgenstern utility function i for every profile.

Let the evolving population of bacterial colonies play the game G, iterated over time steps of t,

with t N . Assume that for each playeri ,i , the strategy set i {Ci , Di }

comprises two

pure strategies C and D.We do not assume the existence of any institutional arrangement to oversee the game, and act as a rule enforcing central authority.

The payoff function of the ithagent is given by

, x, where {C , D } and x ,the number

i i i i i

of other cooperating agents in . Notice that for being explicit, we let i : S A N , with A {1,…, n}. Following [17], we make assumptions regarding the payoff function of the agents that essentially lead to the dilemma in the game:

1. i i Di , x i Ci , x 0,x 0,…, n 1, and i is a constant for all values of x .

2. i Ci , xis monotonically increasing function of x .

   3. i Ci , n 1 i Di ,0.

   Assumption 1 implies that the payoff obtained by defection is always positive for any given agent in , irrespective of the strategy choice of the other agents, and thus the strategy vector

   D ,…, D t is a non-cooperative equilibrium of the game.Assumption 2 implies that with increase

   1 n

   in the number of other cooperating agents, the payoff of a given agent also increases. Assumption 3 implies that all agents receive a higher payoff if all cooperate than if all defect.

   The game matrix of G is obtained using the following payoff functions, which may be assumed without any loss of generality [18]:

   C 3x 1/n 1

   D 5x n x 1/n 1

   (1)

   where C

   and D

   are respectively the payoffs received by a cooperating and a defecting agent

   in the game.

   Assuming the number of players as 100, the following payoff matrix for the game could be constructed by using the formulae for payoff given in (1), and gives the percentage of cooperating agents in the population of 100:

   Player	100%	75%	50%	25%	0%
   C	3	2.24	1.48	0.72	0
   D	5	4.03	3.02	2.01	1

   Fig1. Payoff matrix for the n=100 person Prisoners Dilemma, giving the percentage of cooperators in the population

3. EXPERIMENTAL OBSERVATIONS AND INFERENCES

   This section describes the experimentally observed patterns formed in the Petri dish , by the interacting bacterial colonies subjected to different chemical stresses.

   Fig. 2 Compact structure emerges under substrate stress, due to all agents in the population playing strategy (C100,D0).

   These kinds of pattern as depicted in Fig 2 are observed when we use normal peptone, and introduce a substrate stress in the form of high agar concentration (hard surface). It may be inferred that the pattern emerge as a consequence of cooperative, communicative interaction among the agents, that makes it possible for the interacting colonies to cope with the induced surface stress and grow.We named this pattern as compact structure (CS) and propose that

   bacteria use C100, D0 , that is, all the agents in the playing population cooperate, to emerge this

   pattern.

   Fig. 3 Dense branching pattern, formed due to substrate stress, with bacterial colonies playing the strategy (C75,D25).

   This particular pattern depicted in Fig. 3, which we have named dense branching (DB), is observed on inducing nutrient stress in the growth environment of the colonies, through supply of low level of peptone, while keeping the agar concentration at normal level.In such a nutrient- deficient environment, each agent competes with the co-players for appropriating the scarce resource, often through secreting anti-bacterial compounds [26]. Single bacteria cannot secrete antibacterial compound. We claim that only some of the closely related colonies collectively cooperate to secrete such compounds, and thereby affect distantly related sibling colonies, and thus defect with a majority of the cooperating colonis. Hence, we propose that such pattern

   emerge due to playing of the mixed strategy C75, D25 in the agent population.

   Fig. 4 Sparse branching pattern formed due to antibiotic stress

   The pattern depicted in Fig. 4 was observed with normal levels of nutrient and agar concentration with an antibiotic stress in the bacterial environment. The pattern was called sparse branching (SB).We infer that antibiotic is affecting the DNA synthesis of bacteria, thus affecting its growth and multiplicity [26, 27, 28]. We propose that to reach on this state from initial state bacterial colonies are using (C50, W) strategy i.e. 50% of the agents, selected randomly, are cooperating (to cope with hard surface) and the rest 50% have withdrawn from game owing to absence of growth in these colonies.. A withdrawal by50% of the playing population means that 50% (though this number is arrived at just for the sake of modelling convenience) of the total erstwhile players are unable to co-operate, as they face antibacterial inhibition in multiplying and contributing players over newer generations to participate in the ongoing game.

   Fig. 5 Meagerly spaced branching pattern occurs due to nutrient and surface stresses, with colonies playing (C25,D75)

   This kind of pattern, depicted in Fig. 5 that we called meagerly spaced branching (MSB), is observed when we use poor nutrient level with high agar concentration. Observation suggests that to in order to manage the dual stress and grow, the agents in the game must cope with the hardness of the surface and simultaneously compete amongst themselves for appropriating the low availability of nutrient. We infer that to cope with substrate stress, the agents assess the problem through collective sensing, recall stored information of past experience, and then execute distributed information processing. The above process causes cooperation to emerge among the competing agents [5, 29, 30].At the same time, scarce availability of nutrient causes the agents to secrete anti-bacterial compounds, and thus promote defection with the sibling agents [26].Based on the foregoing discussion, we propose that the meagerly spaced branching

   pattern emerges because of the players using the strategy pair C25, D75 .

4. MODELLING

   Let the DFA that models the evolutionary dynamics of the stressed bacterial colony be

   Q,, q, , h, where Q is the set of states, is the alphabet which is the set of input symbols,

   q Q

   is the initial state, is the transition function that prescribes the mapping of the automaton

   from one state to the next in time steps of t N {0,1,…}, h is the set of final states comprising the theorems in a Turing machine.

   For our present modelling purpose, the objects of are described as below:

   Q comprises the following states, which represent the different pay-offs that the players would receive on their respective strategic moves:

   • Initial state (I)

   • Compact structure (CS)

   • Dense branching (DB)

   • Meagerly spaced branching (MSB)

   • Sparse branching (SB)

   • Dead state (DS)

The alphabet comprises the inputs, which are the strategies used by the players in playing G, given together with their corresponding codes, as below:

I is the initial state, representing the initial state of the growth of the bacterial colonies when they are inoculated in the Petri dish, at the given conditions.

We have used the symbol

C0 as a strategy to indicate that the population of cooperators has

approximately become 0%, thus causing the colony growth to eventually stop. The transition function is described by the following matrix:

DS isconsidered to be the final state in our model, since the growth of bacterial colonies ceases on reaching this state, and there does not exist any state for transition to, at any later time step.

The DFA may then be depicted as a digraph as shown in Fig. 6 below:

Fig.6. DFA for bacterial evolutionary dynamics

The above model of DFA would generate a standard context free grammar (CFG), which would, in essence, be determined by the transition function . Let the non-terminals in the DFA be denoted byV , comprising the following states:

I: Initial state

J: Compact state

K: Dense branching state L: Sparse branching state

M: Meagerly spaced branching state D: Dead state

{a,b,c, d,e}

Corresponding to such a set of non terminal states, the CFG could be written as

I aJ|bK|cL|dM|eD J aJ|bK|cL|dM|eD K aJ|bK|cL|dM|eD L aJ|bK|cL|dM|eD M aJ|bK|cL|dM|eD

* D aD|bD|cD|dD|eD|

Where, in accordance with the description of the DFA, I is the initial state and D is the final state, which, for the sake of identification, is prefixed with an asterisk.

1. CONCLUSION

   The discussions presented in this paper pertain to a modelling of the complexity that emerges due to iterated play of n-person Prisoners Dilemma game, by the bacterial colonies, or agents, subjected to grow in harsh environmental conditions. The growth of the colonies under the induced chemical stresses in the bacterial growth environment in form of nutrient depletion, augmentation of hardness of substratum and presence of antibiotic material, resulted in complex, often fractal like pattern formation. We have argued that such complex patterns are formed due to the game being repeated over generations, and proceeding via exchange and processing of information among the competing agents. We have presented a deterministic finite automata model, together with the context free grammar, for such a constrained growth scenario that leads to the beautiful complex patterns observed.

   Acknowledgement

   The authors wish to express their gratitudeto the department of Biotechnology of the ICFAI Unversity, Dehradun, India, for providing the facility for conducting allwet-lab experiments.

2. REFERENCES

1. M. Mitchell (2009) Complexity A Guided Tour, Oxford University Press.

2. C. Gros (2008) Complex and Adaptive Dynamical Systems, Springer-Verlag.

3. S.A. West, A.S. Griffin, A. Gardner, (2006) Social evolution theory for microorganisms,

   Nature, 4:597-608.

4. E. Frey, T. Reichenbach (2010) Bacterial games, InPrinciples of Evolution: From the Planck Epoch to Complex Multicellular Life, H. Meyer-Ortmanns& S. Thurner (eds), Springer-Verlag.

5. E. Ben-Jacob (1997) From snowflake formation to growth of bacterial colonies II: Cooperative formation of complex colonial patterns, Contemporary Physics 38(3): 205 241.

6. K. Krawezyk, W. Dzwinel, D. A. Yuen (2004) Nonlinear development of bacterial colony modelled with cellular automata and agent objects, International J. Mod. PhysC14(10): 1385 1404.

7. E. Ben-Jacob, Y. Ahranov, Y. Shapira (2004) Bacteria harnessing complexity, Biofilms1: 239 263.

8. E. Ben-Jacob, I. Cohen, I. Golding, D.L. Gutnick, M. Tcherapakov, D. Helbing, I.G. Ron (2000) Bacterial cooperative organization under antibiotic stress, Physica A 282: 247 282.

9. E. Ben-Jacob, I. Cohen, D.L. Gutnick (1998) Cooperative organization of bacterial colonies: from genotype to morphotype, Ann. Rev. Microbiol.52: 779 806.

10. A. M. Lacasta, I.R. Canatalapiedra, C.E. Auguet, A. Penaranda, L. Ramirez-Piscina (1999) Modeling of spatiotemporal patterns in bacterial colonies, Phys. Rev. E59(6): 7036 7041.

11. J. A. Shapiro (1988) Bacteria as multicellular organisims, Sci. Amer. 258: 62 69.

12. M. Doudoroff, R.Y. Stainer, E.A. Adelberg (1957) The Microbial World,Prentice-Hall.

13. D. R. Hofstadter (2000) Gdel, Escher, Bach: an Eternal Golden Braid, Penguin.

14. W. J. M. Levett (2008) An Introduction to the Theory of Formal Languages and Automata, John Benjamins.

15. J. E. Hopcroft, R. Motwani, J. D. Ullman (2006) Introduction to Automata Theory, Languages and Computation, Addison Wesley.

16. M.N. Szilagyi, Z.C. Szilagyi (2002) Non-trivial solutions to the N-person prisoners dilemma, Syst. Res. 19: 281 290.

17. A. Okada (1993) The possibility of cooperation in an n-person prisoners dilemma with institutional arrangements, Public Choice 77: 629 656.

18. K. Manhart, A. Diekmann (1989) Cooperation in 2- and N-person prisoners dilemma games: a simulation study, Analyse&Kritik 11: 134 153.

19. M. N. Szilagyi (2003) An investigation of N-person prisoners dilemmas, Complex Systems14:155 174.

20. R. M. Dawes (1980) Social dilemmas, Ann. Rev. Psychol. 31: 169 193.

21. K. G. Binmore, L. Samuelson (1992) Evolutionary stability in repeated games played by finite automata, J. Economic Theory57: 278 305.

22. A. Rubinstein (1986) Finite automata play the repeated Prisoners Dilemma, J. Economic Theory 39: 83 96.

23. D. Abreu, A. Rubinstein (1988) The structure of Nash equilibrium in repeated games with finite automata, Econometrica56(6): 1259 1281.

24. A. Rubinstein (1998) Modeling Bounded Rationality, The MIT Press.

25. E.B. Jacob, A. Beer (2009) Deadly competition between sibling bacterial colonies,Proc. Natl. Acad. Sci. USA106 (2): 428 433.

26. M.A. Kohanski, D. J. Dwyer, J. J. Collins (2010) How antibiotics kill bacteria: from targets to networks, Nature Reviews Microbiology 8: 423 435.

27. F.G. Rodgers, A.O. Tzianabos, T.S.J. Elliott (1990) The effect of antibioticsthat inhibit cell-wall, protein and DNA synthesis on the growth and morphology of Legionella pneumophila, J. Med. Microbiol. 31: 37 44.

28. L. Tsimring, H. Levine, I. Aranson, E. B. Jacob, I. Cohen, O. Shochet, W.N. Reynolds (1995) Aggregation Patterns in stressed bacteria,Phys. Rev. Lett., 75(9):1859 1862.

29. E.B. Jacob (2009) Learning from bacteria about natural information processing, Ann. N.Y.Acad.Sciences, 1178:78-90.

30. R. S. Xavier, N. Omar, L.N. de Castro (2011) Bacterial colony: Information processing and computational behavior, In Proc. Third World Congress on Nature and Biologically Inspired Computing.

31. E. Ben-Jacob, Y. Shapira, I. Becker, N. Richman, V. Volman, E. Hulata, I. Baruchi (2003) Communication-based regulated freedom of response in bacterial colonies, Physica A330: 218 231.

32. E. Ben-Jacob, I. Cohen, H. Levine (2000) Cooperative self-organization of microorganisms, Adv. Phys. 49:395 554.

33. K. Kawasaki, A. Mochizuki, M. Matsushita, T. Umeda, N. Shigesada (1997) Modeling spatio-temporal patterns created by bascillus-subtilis, J. Theor. Biol. 188: 177 185.

34. A. Neyman, D. Okada (2000) Two-person repeated games with finite automata, Int. J. Game Theory 29: 309 325.