Using Linux Virtual Server to Balance Zone Load in an MMO Game

I am a developer of an MMO game, and currently, at my company, I am faced with some scalability problems, which, I think, can be solved by properly clustering the game world.

I really don't want to reinvent the wheel, so I think Linux Virtual Server might be a good choice, especially using the level 7 load balancing method.

I am currently looking at ktcpvs as a load balancing solution and wondering if it is the right choice.

The main idea is to have several zones ("locations" from the point of view of my game) running on dedicated servers. When a player decides to go to a specific place, the load balancer decides which zone server will actually serve the player (this is actually why I need a level 7 load balancer)

What do you think of all the above?

Update: I sent the same question to the LVS user mailing list http://marc.info/?l=linux-virtual-server&m=124976265209769&w=2

Update: I also started a similar topic on the gamedev.net forum http://www.gamedev.net/community/forums/topic.asp?topic_id=544386

+4
source share
4 answers

To answer your question, we need to understand if you need a volume or an answer, but it is difficult to get both at the same time.

Layer 7 load balancing is application-level load balancing, so the contents of the network packet data must be redirected to the endpoint. You can achieve volume (more users) by implementing routing at the application level, service level, or kernel level.

Scalability. I assume that you are running out of memory, processor resources, and network bandwidth.

  • Application Level - Your application logic receives the application package and routes accordingly.

  • The level of service - your system infrastructure (any front-end service) receives the packet and through the module - performs routing (think about a custom apache module, even network driver modules - for example, creating a network filter) / p>

  • Kernel level β€” Perform routing at the network packet level.

The closer you get to metal, the better your answer will be. I suggest using a dedicated linux server to perform routing - go to your native, not virtual. Use multiple or combined network adapters for the WAN and a dedicated adapter for each endpoint (one + wan, one for each connected application server).

If the response time is important, you need a kernel / supervisor solution, it will save you a few context switches, but keep in mind that you need to limit flights at any cost and better serve less, more machines and your scalability will always be limited. There is a risk of using KTCPVS, it is quite old and is not actively updated. If you judge that it works fine for you, consider writing something that looks like a surge protector while it is running in system state.

If loudness is important, but response time is secondary, implement a built-in high-speed socket switch, built-in C ++, working in a problem / user state. This is easiest to maintain and will offer better scalability.

You will need to create prototypes to find out what works best for your needs.

Final thoughts -

Before you do any of the above, make sure that you have optimized your game design. You can know most of this, I list it here for everyone.

(a) Messages should fit comfortably within a single network packet, less than 1,500 bytes for most home routers.

(b) Try to set the routing logic in your game client instead of your servers. Simply loading a small table with zones and IP addresses for the client will allow you to abandon all of the above.

(c) Try to limit the visibility of the zone to customers, they should only know about their zones and adjacent zones (if you implement point b above)

Hope this helps, sorry, I cannot be more specific regarding KTCPVS.

+9
source

You did not indicate where the bottleneck is. Network traffic? Disk IO? CPU cycles?

Assuming that you are referring to a load balancer of level 7 and do not have sufficient processor power, I think that LVS is not an optimal choice. I performed load balancing on a web server using LVS, which works directly and not quite complicated.

But I think that load balancing on MMORP in this way requires significant amounts of additional code in LVS, it may be easier to perform load balancing using a multi-threaded application distributed over some multi-core server. But it is not fully scalable, it will lead to only 16 cores without limiting the growth of cost.

+2
source

The biggest problem in something like this is what happens when players are near the border. Obviously, they should be able to see and interact with each other, but they are on separate servers. Thus, you need a pretty nice cross-server connection, and sometimes just duplication of messages on both servers. This can get complicated when someone is near the β€œcorner” and then you need to deal with 4 servers!

In the book "Multiuser game development" there is a chapter "Traps common server boundaries", which details this issue.

I still have not heard about Linux Virtual Server, so I don’t understand how it fits. I think that your actual server application should support this load balancing for a specific game, instead of trying to start a cluster and assume that it will automatically know how to split your application (which it will not). If I were you, I would write a server program to process my own piece of land, and it should connect to the parts surrounding it, and then develop a server-server protocol to transmit these messages ("here comes the player, I'm going to tell you about him! "" Be sure to tell me about messages near our border, "" it’s good that the player has left my territory and yours, his details are here, "etc.), I think this is a little more complicated than just run a different linux style and assume that you get automatically Iy load balancing.

0
source

Why are you moving the distribution logic to a loadbalancer? This is a component that is not free and may break. It seems that your customers are well aware of the zone in which they are located. It seems they could very well connect to zone<n>.example.com . Then you will handle load balancing at the DNS level.

-1
source

All Articles