Setuptools was never found on AWS Lambda

Question

I am trying to use AWS Lambda to do an introspection of a python package.

What I'm trying to do is press git push, check the package, install its dependencies (package dependencies for introspection) and run the toolkit. AWS does not have a protocol installed by default (you must bind your own dependencies), so I build pip using my lambda function.

Every other part of my lambda works, except for trying to install package dependencies. When trying to use my package package with an unloaded package, it does not try to run setuptools:

Collecting alembic==0.8.3 (from -r /tmp/tmpnx0tY0/requirements.txt (line 1)) Downloading alembic-0.8.3.tar.gz (935kB) Complete output from command python setup.py egg_info: Traceback (most recent call last): File "<string>", line 1, in <module> ImportError: No module named setuptools ---------------------------------------- Command "python setup.py egg_info" failed with error code 1 in /tmp/tmp30m2KN/alembic 

I tried everything: add the current path (using setuptools) to the path, copy setuptools to the temp directory, run pip install setuptools , passing parameters using --global-options , trying pOpen with env, specifying PYTHONPATH , even tried using older versions pip with: pip install -r requirements.txt --download --no-install . Nothing seems to work.

How can I get this python setup.py egg_info call python setup.py egg_info to successfully find setuptools?

Answer 1

Have you considered using Jenkins (or any other CI tool) to connect to your Git repository, build the repository on click, and then load the packed lambda with all the dependencies on S3 before running it?

In AWS Lambdas, you can only upload files to the / tmp folder, and you have a limit of 50 MB of space. This 50mb includes the space consumed by your lambda function. Lambdas are zippered and pipette-free for some reason.

Answer 2

I used Lambda a bit, and I don't think Lambda supports this. You need to use your own CI tools (Jenkins is a good choice). Otherwise, you need to create it on your local computer, and then use the AWS Lambda plugin to download the code with the installed dependencies.