Self-Organization in Femtocells: A Game Theoretic Approach to Mitigate the Interference and Enhance the Macrocell Performance

Self-Organization in Femtocells: A Game Theoretic Approach to Mitigate the Interference and Enhance the Macrocell Performance

M.A.Anisha, M.E,Sri Sairam Engineering College.

1. Brindha, Associate Professor, SeniorMember, IEEE, Sri Sai Ram Engineering College. anishamaria84@gmail.com.

   AbstractMobile operators are compelled to explore newways to improve their coverage, boost their network capacity, and lower their capital and operating expenditures. To reach these goals, small-cells are underlain macrocells which automatically creates interference (SINR) and to avoid this, short range, low power and low-cost base station which are decentralized and self-organizing by enacting the concept of game theory is implemented. In this system co-tier and cross-tier interference of macrocells and femtocells occurs which is avoided and a minimum guaranteed signal to interference plus noise ratio (SINR) is guaranteed at the macrocell user equipment. Individual performance of each femtocell is increased drastically to provide a better service to macrocell. The reinforcement learning procedure used here is fully distributed as every small cell base station requires only an observation of its instantaneous performance which can be obtained from its user equipment. Furthermore, it is shown that the proposed mechanism always converges to an epsilon Nash equilibrium when all small cells share the same interest. Finally real time example of this process is explained to validate the theoretical findings, highlighting the utility functions through which the results are arrived.

   Index Termssmall cell networks, SINR, game theory,Nash equilibrium, reinforcement learning.

   1. INTRODUCTION

      Increased demand in wireless systems and proliferated user services leads to create a greatest demand in wireless systems. New ways are to be explored to increase the coverage and boost up the performance, network capacity, lower the capital and operational expenditures and to provide a network system in interference less environment. To fix these issues, small cell networks are designed, they are the combination of picocells, femtocells and microcells. By deploying these short range, low-power, and low-cost base stations underlying the macro-cellular network increases the performance of the macrocells and small cells [4], [8].

      Though the small cells provide unprecedented network

      capacities, their deployment faces a number of technical challenges widely, among which self-organization and interference management are very crucial. Small cells are user deployed and their operation hinges on their self- organizing and self-optimization capabilities. Eventually self-organizing networks (SON) constitute a novel approach which empowers the reduction of manual intervention in network planning instead relying on self-configuration and self-analysis. SON-enabling mechanisms are used in small cell networks to sense and learn from their surrounding autonomously and tune their transmission strategies to produce an optimal performance. So naturally the deployment of self-organizing small cells is an intricate problem that calls for fully scalable, distributed and self- organizing strategies for interference management in dense small cell networks. In this regard, game theory (GT), a mathematical tool which analyzes strategic interactions among decision makers and a paradigm to study this problem due to mutual interference and coupling among small cells strategies[3], [5].

      In [1], the authors propose a clustering-based radio resource sharing method for femtocell networks, in which femtocells cluster into interference-free coalitions. In [2] and [8], the spectrum leasing as a reward mechanism for femtocell operation is proposed which acts as a helping relays. Amount of manual intervention is reduced in spectrum planning, relying on self-analysis [6], [9]. Majority of these works require information exchange among femtocells and between macro and femtocells, which is not desirable in a dense heterogeneous network due to significant signaling/overhead. In this paper, we propose self-organizing strategies for interference management in closed- access small cell networks with the minimum strategies required to learn equilibrium. Simultaneously, learning of both the strategies and the utility of players for achieving equilibria in small cell networks is recommended.

      This paper is organized as follows. In section II the design of system and models used in game are presented. Section III explains the urban scenario of an apartment blocks randomly located and their respective outputs. Section IV A and B explains the results and convergence

      properties of two utility functions. Section V explains the conclusion and future work and concludes the paper. Section VI contains references.

   2. SYSTEM MODEL

      The system model mainly comprising of the method using which signal interference plus noise ratio of the femtocells can be reduced so that the performance of macrocell is increased. For reaching this we are following Game

      channel gain between transmitter j and receiver I in the sub- carrier s at time interval n is denoted by

      (4)

      With i, j {0} and s S. The channel realization is given by , it is assumed to be the path loss and log- normal shadowing. The downlink transmit power used by transmitter k on sub carrier s is denoted by

      Theoretic approach by which each user equipment which

      are decentralized and self-organizing learn the strategies of

      (n)

      (5)

      other user equipment and enhance its own performance. We are taking an urban scenario in which due to the increases usage of data services and the need for sophisticated user terminals we need more unused and free spectrums to increase the coverage and reduce the data loss and for this purpose we are considering to establish femtocell in each apartment blocks using which the user is benefited and henceforth we face a interference problem which is actually

      With = { } a finite set of power levels. The transmit configurations of all the femtocells is determined by the channel over which the transmission takes place and the transmit power levels. The total number of transmit configuration of femtocell base station k is

      =S , and the set of possible transmission configurations is given by

      reduced by the method SON-RL. Using Shannon

      transmission rates each cells characteristics are learned

      = X { }

      after which the utility functions are used to change its own transmission rates and find an optimal output to itself.

      = { }, (6)

      Downlink of a macrocell base station (MBS) is considered to be operating over a set

      S = {1. . . S}(1)

      OfSorthogonal subcarriers.Over each sub-carrier at each time interval, the MBS serves one macrocell user equipment (MUE).For all s S, the MBS guarantees a minimum average SINR, denoted by (s). Only below a fixed threshold value the transmission strategies are considered, if it applies more than the limit then it is

      Where for all {1 }, the -th element is a vector of form

      = (7)

      Where {1 } and . Both and are indices that are unique to determine by index . The transmit configuration taken by FBS k at time n is denoted by the power allocation vector

      discarded. The cross-tier interference is denoted and generated by a set of values

      = (n) (n))

      , (8)

      K = {1. . . K}(2)

      Which are of K femtocell base stations (FBSs) under the

      And thus, the SINR levels at MUEs and FUEs over sub-

      carrier S are:

      ]

      [

      (n) = (9)

      coverage area of macrocell. Technically different femtocells dynamically chooses each subcarriers which are available at moment to serve the femtocell user equipment (FUE) as long as it does not induce a lower average SINR for the MUEs lower than the predefined thresholds

      And

      (n)=

      [ ]

      .

      (3)

      [

      ] [ ]

      ]

      (10)

      Noting the assumption it can be relaxed to accommodate the case in which arbitrary number of sub-carriers is

      [

      transmitted by the femtocells, to be in general multiple macrocell is also possible. The transmitter-receiver pairs in the macrocell (i.e., MBS-MUE) are denoted by the index 0, whereas, the transmitter-receiver pairs in the femtocells (i.e., FBS-FUE) are denoted by the indices in K. Thus, the

      Respectively. , and are the macrocell (femtocell k) noise variance on sub carrier S, respectively.

      In what follows, we consider some performance metrics for femtocells although any objective function can be considered. We consider the case in which femtocells which are selfish interested in only their individual performance and their Shannon transmission rates are found to be observed. We find the utility function as such :

      X.X R this is associated with their selfish behavior of player k as follows

      ( , ) =

      implementing the game theory pattern in all the active femtocells.

   3. URBAN SCENARIO OF APARTMENT BLOCKS The objective of this section is to study a small apartment

      urban scenario to learn the performance of each femtocell

      and how to avoid interference of the macro cells using the game theory process. We consider M=1 macrocell and S=6 subcarriers. For that purpose we consider six apartment

      blocks, each block has two apartment strips. Each block is

      considered to be separated by a 10m wide alleyway. Five

      (11)

      apartments is contained in each strips. Inside each

      In this case we note that theutility achieved by a given player is either the actual transmission rates of its own, if the mobile base station achieves its minimum SINR level or zero otherwise. The engagement of second tier operation is formed only if the performance of first tier is not degraded. Femtocells are able to estimate their interference for this purpose. The altruistic performance is the second metric. The interest of each femtocell here is to find the sum of the individual Shannon transmission rates of all the active femtocells over the sub-carriers where the mobile base station achieves its minimum SINR level. The utility function of each femtocell k is defined as such

      ( (n), ) = , (12) Where,

      , =

      apartment there is an active femtocell. The arbitrary number of power levels and femtocell user equipments straightforward.

      MUE

      Alleywa

      A.B

      (13)

      MBS

      Unlike the selfish case, the femtocells interest is altruistic and corresponds to the global performance. Each femtocell has the transmit power level of L=3, with S available sub- carriers and provides service to single femtocell users

      A main macrocell has certain scenarios for deploying femtocells. Dedicated radio channel for macro users and femtocell users, it can be used where a lot of radio channels are unused specifically in rural areas. In case of sharing of channels between femtocell and macro cells degree of freedom to manage interference is provided vastly. Whereas in this case a channel or sub-carrier is sequentially used with respect to time only by the macrocell users, but femtocells can utilize them only if any of the subcarrier is

      Fig. 1. Apartment blocks located randomly which we consider here as urban scenario. Average distance of apartment blocks: 350m, sector of MBS 120 degree. (AB- Apartment Block)

      One solution is to allow femtocells to carry out a two- level approach, in which each femtocell first selects a sub- carrier, and then the femtocells schedule their UEs accordingly. The minimum SINR for per sun-carrier of Macrocell user equipment is defined as such

      == = 3dB (14)

      available. Second case is as such some radio channels are shared by macrocell and femtocell users certainly some channels are purely for macrocell users and hence this method proves to be managing the interference through the way of setting a threshold value for specific sub-carriers that are being utilized by users commonly. This method differs from the existing method in the aspect of

      Each channel is represented as a combination of path-loss and log-normal shadowing with standard deviation of 8dB and 4dB for outdoor and indoor communications respectively.

      TABLE I

      PARAMETERS CONSIDERED IN THE PROCESS OF ITS SPECIFICATIONS

      S.NO Parameters Specifications

      1. Alleyway width 5m

      2. Macro BS power 46dBm

      3. Femto Range 3m

      4. Femto BS max power 21dB

         ISSN: 2278-0181

      5. Femto BS default power

         11dB

      6. Noise density power -174dBm/Hz

      7. Target SINR range 10m

      8. Target SINR 3dB

      9. Outdoor/Indoor walls loss

         15dB/7dB

   4. RESULTS AND DISCUSSION

      1. PLACING THE CELLS IN A 3X2 GRAPHICAL

         CONTEXT

         The fig. 2. Shows the placement of different macro and femtocells with the certain distance limits and the illustration shows the macrocell base station and their users (MU1, MU2, MU3, MU4) and also femtocell base station and their respective users (FU1FU18). Parameters such as indoor or outdoor unit, threshold, SINR, path loss and distance are studied using this program. X and Y coordinates are marked so that the cells are placed in a right order each time. This method is used to study the interference within cells and to deploy the Nash equilibrium to calculate the input for the game theory.

         FIG. 2. Implementation of Macro and Femtocell with X and

         Y co-ordinates and their unit details.

         Interference mainly is calculated by fixing a threshold point. In fig. 3. F1 base station is highlighted and unit details are gathered to get a clear study.

         FIG. 3. Femtocell F1 is considered as additional user and femtocell users FU1, FU2, FU3 and their unit details.

         The macrocell frequency environment is discussed with variety of partitioning methods as such FFR (fractional frequency reuse), IFR (integer frequency reuse) and SFR (soft frequency reuse). The power scheme is given by constant femtocell base station radius or SINR is targeted. FIG. 4. Gives a learning to F2 where it is targeted with fractional frequency reuse by the macrocell present at the center.

         FIG. 4. Femtocell F2 is considered as additional user and femtocell users FU4, FU5, FU6 and their unit details.

         Fig. 5 shows the interference suffered by femto user F3 because of the interference caused by the macrocell which is targeted by the threshold value below which the transmission is not possible.

         Soft frequency reuse divides the cell area into cell edge region and central region, the spectrum dedicated in the cell edge area which are unused can also be used in central region. Much reduced Shannon capacity in produced due to the lack of spectrum in cell edge. This is overcome by inducing high power carriers to users in these regions thus improving the SINR and Shannon capacity.

         FIG. 5. Femtocell F3 is considered as additional user and femtocell users FU7, FU8, FU9 and their unit details.

         Fig. 6 depicts the interference caused by the femtocell and the interference ratio caused because of that. Fractional frequency reuse is used in this where the utility function takes the location of femtocells into acount. The interference caused by the macrocell is reduced since the throughout put of F4 is given.

         Fig. 6. Femtocell F4 and Femtocell users FU10, FU11, FU12 and their unit details.

         Fig. 7. Femtocell F5 and Femtocell users FU13,

         FU14,FU15 and their unit details.

         Fig. 8 is given by the additional user F5 whereas the threshold and SINR values concerned with the less variations produced. For all the three users the interference is created by macrocell base station.

         By using Nash equilibrium the SINR values with respect to the fixed threshold is found out to get the overall convergence quality of the process. Game g (b) is termed to be a better potential game hence the best response is obtained through it. The assumptions of complete information in unaffordable in previous systems but here every information of each cell is learnt.FFR (Fractional Frequency Reuse) is used with this scenario to improve capital and spectral efficiency whereas the SINR varies according to the transmission rates. One cell with 25MHz bandwidth have higher capacity when compared with 7 cells having 3.5MHz and this FFR method helps in gaining a desirable amount of required bandwidth needed for femtocells and hence the concurrence of maximum power 21dBm is supported.

         Fig. 8. Femtocell F6 and Femtocell users FU16, FU17, FU18 and their unit details.

         Fig. 9 shows the overall coverage area of macrocell. The map shows different interferences created and also the threshold of certain regions. The results of every other player in the scenario and enhances its own performance and so the learning of its own transmission strategies in first case and learning the transmission strategies of all other players in case two is implemented.

         Fig. 9. Macrocell Base Station and their coverage details.

         The complete information and incomplete information to build the smooth best response which is clearly unaffordable for all the small cells is practiced in every step. The additional usage of F6 in which less interference is caused by the macrocell base station henceforth inter- channel interference are possible in these type of output. The distance between base station and user is also one of the main parameter for data loss.SINR required for the successful reception at the receiver depends on the transmission rates. Threshold indicates the relative signal level at which a new packet can disrupt an ongoing reception and we interpret it as minimum SINR.

      2. CONVERGENCE PROPERTIES

      Before proclaiming into the performance of the proposed reinforcement learning algorithm SON-RL in spectrum wide settings, first insight about the algorithms convergence behavior is studied. The convergence behavior of the reinforcement learning algorithm SON-RL for both selfish and altruistic utility functions, corresponding to the games respectively. With respect to the Best Nash Equilibrium (BNE), a benchmark result is also provided; it characterizes the Nash equilibrium maximizing the femtocell sum-rate. Altruistic utility function yields a higher average spectral efficiency when compared to the selfish case can be seen.

      This is due to the fact that when acting selfishly, femtocells are careless about the network wide performance and the interference caused onto neighboring femtocells, thus it end up obtaining lower performance. In contrast, in the altruistic

      case where players maximize the same network-wide performance objective function o higher performance is obtained.

      In both the cases it is worth noting that, the proposed algorithm convergence the logit equilibrium in both the gamesg (a) and g (b).However the convergence is always guaranteed in g (a) it is seen that in particular a non- cooperative game is formally defined by the set of players, the sets of strategies (or actions) that each player may take, and the individual player utility or cost functions. The game is non-cooperative in the sense that a given player is interested only in minimization of its individual cost function, without paying attention to how its actions affect the other players. At the optimal Nash equilibrium point of the game each users strategy is a best response function to the other users strategies, and all user code words are minimum eigenvectors of their corresponding interference

      + noise correlation matrices. Fig.6 shows the difference between altruistic and selfish behavior and convergence of the femtocells.

      Fig. 10. Convergence algorithm with respect to the Best Nash Equilibrium.

   5. CONCLUSION AND FUTURE WORK

      In this paper we proposed reinforcement learning based urban scenario and the interference management infemtocells by using SINR reduction using the method called game theory. The fully decentralized proposed self- organizing algorithm pronounces the base stations which relays solely on local information. For these conditions utility functions and their transmission strategies are learned which gives a definite output to the next level of the game which is imitating the others strategies or learning its own strategies to give a better performance than others. This shown behavioral rule leads to the coverage of network towards logit equilibrium, whereby the femtocells strike a balance between experimenting the wide-network and choosing a valid action to optimize its performance.

      Though two utility functions has been proposed in this paper in future more than two utility functions can be added to make this process much more accessible and to get a positive response.

   6. REFERENCES

1. D. Lopez-Perez, I.Guvenc and G. de la Roche, Enhanced inter-cell interference coordination challenges in heterogeneous networks, IEEE Wireless Commun. Mag., vol.18, no. 3, pp. 22-30, june2011

2. V. Chandrasekhar and J.G. Andrews, Femtocells network: a survey, IEEE Commun. Mag., vol. 46, no. 9, pp. 59-67, sep. 2008.

3. I. S. Jacobs and C. P. Bean, Fine particles, thin films and exchange anisotropy, in Magnetism, vol. III, G. T. Radio and H. Suhl, Eds. New York: Academic, 1963, pp. 271350.

4. L. Rose, S.M. Perzala, S. Laura, Learning equilibria with partial information in wireless networks, IEEE Commun. Mag., vol. 49, no. 8, pp. 136-146, Aug. 2011.

5. J.G. Andrews, H. Claussen, M. Dohler, S. Rangan and M.C. Reed, Femtocells: past, present and future, IEEE J. Sel. Areas

   Commun, vol. 30, no. 3, pp.497-508, Apr.2012.

6. 3GPP R4-092042 TGS RAN WG4 (Radio) Meeting 51: simulation assumptions and parameters for FDD HeNB RF requirements, Tech Rep., May 2009.

7. M. Bennis, S. Guruaacharya and D. Nityato, Distributed learning strategies for interference mitigation in femtocell networks, IEEE Global Commun. Conf., 2011

8. H. Li, Multi-agent Q-Learning of channel selection in multi- user cognitive radio systems: a two by two case, IEEE Conf. Syst. Man Cybern., 2009.

9. S. Rangan, Femto-macro cellular interference control with sub-band scheduling and interference cancellation, IEEE Int. Workshop Femtocell Net., 2010.