Executing arbitrary unfriendly Python code on my server

I am making a game where users can write Python programs to control robots that fight each other. Each turn (in a game with a thousand revolutions) of their script will be launched on my server to determine the next step of the robot. How can I prevent these users from being average for my server?

I thought / researched the following:

  • eval their code in a restricted environment (i.e. with __builtins__ disabled)
  • using OS-specific mailboxes like chroot / ptrace
  • running code in some type of virtual machine

I have a Python program that runs a user script a thousand times. How can I ensure the maximum total duration of one minute, limit their memory resources, prevent their access to any files or network connections, etc.? What would be the perfect solution or combination of solutions?

+7
python virtual-machine ptrace chroot jail
source share
3 answers

I am working on PythonAnywhere , which is the Python platform as a service, which means that we run a lot of untrusted code on behalf of our users - the new interactive console on the front page of Python.org is ours, and its limitations are probably not too far from yours.

I recommend using OS-level virtualization; anything at the language level is likely to be less secure. We use chroot and cgroups for historical reasons, but if we start from scratch today, I think we will use Linux (LXC) or Docker containers. LXC is basically a bunch of smart wrappers around chroot and cgroups, etc., which makes them more convenient to use, so you can quickly deploy an ephemeral virtual machine. And Docker is an even easier to use wrapper around the LXC. Both are super-fast - you can get a new virtual machine and start it in less than a second.

+5
source share

I looked into your profile and decided that this is a constant labor of love! so here it goes.

Given your requirements, I assume that you are planning something that will be available through i / webs.


As for your second line of query (; this prevents duplication of Q);

I have a Python program that runs a user script a thousand times. How can I ensure the maximum total duration of one minute, limit their memory resources, prevent them from accessing any files or network connections, etc.? What would be the perfect solution or combination of solutions?

As you probably know, this is well-punched land, right up to the 1980s. Reinvention is sometimes, but not always better, from an informed pov. For ideas on how to work out a temporary breakdown of player instructions, and so on, you can take a look at the sources of some of the related ongoing projects already in the wild there. eg:

Spoiler:

- robocode
This project has a long history ; is in the hands of the community since 2005 and is actively supported. sources are java. essential reading imo.
https://github.com/robo-code/robocode

- fightcode:
It is a javascript flavor and works on the Internet. so it should be very helpful to help you get along with your online Python approach. The site now appears to be commercial / closed. however, you can see the original sources that were presented for the github 2012 game. At the very least, it can give ideas on how to present your final game content to the world :)
https://github.com/timehome/game-off-2012
(By the way, the boycode was runner-up!)


Your first query line is fully SO compliant. See here . But:

The cheapest (in every sense) option would be a chroot prison. The next cheapest option would be something like Linux-VServer . It seems that the performance is close to native, although I have no experience using this software. Otherwise, yes, full virtualization using xen or something else.

imo:

Full virtualization will lead to headaches and harmless overhead.

If you are careful, you can do this with a simple chroot prison.

  • To have a prison exclusively as a one-time "arena" in which codes can be run. make sure that it works with the minimum resources and privileges necessary to complete this work.

  • Unleash your web application completely from the prison (for example, there is no need for the prison to have network access, etc.).

So a workflow could be like this:

  • Download game requests through your web application.

  • The blessing of requests for the game. you said "arbitrary unfriendly codes", but in fact the game will have some formal parameters; you will only need a subset of python . at least check it out.

  • queues, apparently, reasonable requests for the game and sending them to the arena (aka jail).

  • If you are truly paranoid, create a new, clean arena / prison for each new game.

  • start the game in the arena. storing each discrete state of the game (in an internal inexpensive format) in a database local to the prison.

  • from outside the prison, and when the game is finished, provide the web application with a database of the game’s status, and then display it in i / webs.

  • done. go and have a drink!


Notes on setting up python jail.

This is how I got into this post. I was looking for specific instructions on how to jail python using (very fine) jailkit . but could not find much ... so having prepared my own, I then looked to see if there was a place for it on SO.

Keep in mind that some nix distributions, such as Unbuntu and Centos, use python in their work. in this case, to avoid merging your system, you may want to create any version of python from sources.

Here is my recipe for installing python 2.7 (assuming you already have jailkit installed) on Centos

First, a more formal approach to a jail kit, including creating a dedicated user, could be:

 # Once-only set-up: # as root user: ## get, build and alt-install required python onto host OS mkdir -p /usr/local/src/python cd /usr/local/src/python wget http://www.python.org/ftp/python/2.7/Python-2.7.tgz tar -vxzf Python-2.7.tgz cd Python-2.7 ./configure #default ${prefix}="/usr/local" make make altinstall #don't clobber systems' python; everythin installed under /urs/local and comes with 2.7 postfix. ## set-up a jailkit config for python 2.7 (& optionally to support some minimal devs) cat <<OOOK >> /etc/jailkit/jk_init.ini [python2.7] comment = python 2.7 interpreter and libraries paths = python2.7, /usr/local/lib/python2.7, /usr/local/include/python2.7, /etc/ld.so.conf devices = /dev/null, /dev/zero, /dev/random, /dev/urandom OOOK 

 # Ad-infinitum: # as root user: cd /home ## create a python jail by passing jk_init the name of the sction created in /etc/jailkit/jk_init.ini jk_init -v -j /home/jail_py python2.7 ## optionally create a dedicated user useradd -M -s /sbin/nologin prisoner passwd prisoner ## quick sanity-check for setuids find /home/jail_py -perm -4000 -exec ls -ldb {} \; ## test echo "import sys; print sys.version" | jk_chrootlaunch -u prisoner -j /home/jail_py -x /home/jail_py/usr/local/bin/python2.7 echo "import os; print 'cwd:{} uid:{}'.format(os.getcwd(),os.getuid())" | jk_chrootlaunch -u prisoner -j /home/jail_py -x /home/jail_py/usr/local/bin/python2.7 jk_chrootlaunch -u prisoner -j /home/jail_py -x /home/jail_py/usr/local/bin/python2.7 ## optional, if we need /proc mkdir -p /home/jail_py/proc mount -t proc /proc /home/jail_py/proc ## nuke everything umount /home/jail_py/proc rm -fr /home/jail_py userdel prisoner 


And not recommended here, a more ad-hoc approach

 # Once-only set-up: # as root user: ## get and build required python mkdir -p /usr/local/src/python cd /usr/local/src/python wget http://www.python.org/ftp/python/2.7/Python-2.7.tgz tar -vxzf Python-2.7.tgz cd Python-2.7 ./configure --prefix="/usr" # optionally override default ${prefix} ~ we don't really need /local/ indirection for our jail make ## find out minimal python lib dependencies so we can add them to the jail - see jk_cp -j below.. ldd /home/jail_py/usr/bin/python 

 # Ad-infinitum: # as root user: ## create a user useradd -M -s /sbin/nologin prisoner passwd prisoner ## manually create a jail mkdir -p /home/jail_py ## deploy python into jail # IMPORTANT: be sure to export DESTDIR to avoid stomping on your system python ;) cd /usr/local/src/python/Python-2.7 export DESTDIR="/home/jail_py" # point install to /home/jail_py/${prefix} make install # no need for altinstall since we're deploying directly to the jail cd /home ## provision jail with a copy of /etc/ld.so.conf to prevent jk_cp form chucking out warnings jk_cp -j /home/jail_py /etc/ld.so.conf ## copy minimal python lib dependencies to the jail - see ldd above jk_cp -j /home/jail_py /lib/libpthread.so.0 /lib/libdl.so.2 /lib/libutil.so.1 /lib/libm.so.6 /lib/libc.so.6 /lib/ld-linux.so.2 ## quick sanity-check for setuids find /home/jail_py -perm -4000 -exec ls -ldb {} \; ## test running jailed python as user 'prisoner' echo "import sys; print sys.version" | jk_chrootlaunch -u prisoner -j /home/jail_py -x /home/jail_py/usr/bin/python echo "import os; print 'cwd:{} uid:{}'.format(os.getcwd(),os.getuid())" | jk_chrootlaunch -u prisoner -j /home/jail_py -x /home/jail_py/usr/bin/python jk_chrootlaunch -u prisoner -j /home/jail_py -x /home/jail_py/usr/bin/python ## optional, if we need /proc mkdir -p /home/jail_py/proc mount -t proc /proc /home/jail_py/proc #optional, if we need some /dev/x jk_cp -j /home/jail_py /dev/null #nuke everything umount /home/jail_py/proc rm -fr /home/jail_py userdel prisoner 


Note. , depending on your requirements, you may also need / need to install /proc . For example, starting something like this will fail if /proc not installed.

 cat <<OOOK | jk_chrootlaunch -u prisoner -j /home/jail_py -x /home/jail_py/usr/local/bin/python2.7 - #thx: https://raw.github.com/pixelb/ps_mem/master/ps_mem.py import os import sys import errno class Proc: def __init__(self): uname = os.uname() if uname[0] == "FreeBSD": self.proc = '/compat/linux/proc' else: self.proc = '/proc' def path(self, *args): return os.path.join(self.proc, *(str(a) for a in args)) def open(self, *args): try: return open(self.path(*args)) except (IOError, OSError): val = sys.exc_info()[1] if (val.errno == errno.ENOENT or # kernel thread or process gone val.errno == errno.EPERM): raise LookupError raise def meminfo(self): fd=self.open("meminfo") for next in iter(fd.readline, ""): print next.replace('\n', '') Proc().meminfo() OOOK 


You, of course, will not need an interactive session. But it's still worth playing in the arena under a special user. But you do not need it. In any case, jailkit takes most of the pain out of creating a decent chroot prison. You can capture sources and tinker with them according to your needs (I did). or you can simply use jk_chrootlaunch to safely call your arena from outside the prison.

(NB: the reason I needed to mess around with jailkit sources is because it uses the same "/./" login hack, like pure-ftp. Lol)

Finally, you might think: http://www.unixwiz.net/techtips/chroot-practices.html


Good luck, and I hope that you are still active in this .. ~ i41 would like to see the implementation of pythonic =) we played this game obsessively on uni ~ Mat Peck c-zone stylee;)

+3
source share

There is no built-in way to run isolated code in cpython, but there is in pypy.

http://pypy.org/features.html#sandboxing

There are several other methods described in the wiki (e.g. jailkit), but they seem to have various disadvantages.

https://wiki.python.org/moin/SandboxedPython

I would send a pypy route.

0
source share

All Articles