Gettings settings and settings from INI file for Pyramid functional testing

Question

In a real Pyramid application, it does not work on docs http://docs.pylonsproject.org/projects/pyramid//en/latest/narr/testing.html :

class FunctionalTests(unittest.TestCase): def setUp(self): from myapp import main app = main({}) 

An exception:

 Traceback (most recent call last): File "C:\projects\myapp\tests\model\task_dispatcher_integration_test.py", line 35, in setUp app = main({}) File "C:\projects\myapp\myapp\__init__.py", line 207, in main engine = engine_from_config(settings, 'sqlalchemy.') File "C:\projects\myapp\ve\lib\site-packages\sqlalchemy\engine\__init__.py", line 407, in engine_from_config url = options.pop('url') KeyError: 'url' 

The reason is trivial: an empty password is passed to main , although it seems that when you launch a real application (from __init__.py ) it gets settings pre-populated with values ​​from [app:main] in the development.ini / production.ini section:

 settings {'ldap_port': '4032', 'sqlalchemy.url': 'postgresql://.....} 

Is there a way to easily restore settings from a .ini file for functional testing?

Answer 1

pyramid.paster.get_appsettings is the only thing you need:

 from pyramid.paster import get_appsettings settings = get_appsettings('test.ini', name='main') app = main(settings) 

---

That test.ini can include all the settings of another .ini file is easy:

 [app:main] use = config:development.ini#main 

and then you only need to override those keys that are changing (I think you need to test a separate database soon):

 [app:main] use = config:development.ini#main sqlalchemy.uri = postgresql://.... 

Answer 2

Yes, there is, although it is easy to discuss.

I use the following test fixture py.test to make the --ini option --ini for tests. However, this approach is limited to the py.test test runner, since the other test runner does not have this flexibility.

Also my test.ini has special settings, such as disabling outgoing mail and, instead, printing to the terminal and testing the available lag.

 @pytest.fixture(scope='session') def ini_settings(request): """Load INI settings for test run from py.test command line. Example: py.test yourpackage -s --ini=test.ini :return: Adictionary representing the key/value pairs in an ``app`` section within the file represented by ``config_uri`` """ if not getattr(request.config.option, "ini", None): raise RuntimeError("You need to give --ini test.ini command line option to py.test to find our test settings") # Unrelated, but if you need to poke standard Python ConfigParser do it here # from websauna.utils.configincluder import monkey_patch_paster_config_parser # monkey_patch_paster_config_parser() config_uri = os.path.abspath(request.config.option.ini) setup_logging(config_uri) config = get_appsettings(config_uri) # To pass the config filename itself forward config["_ini_file"] = config_uri return config 

Then I can configure the application (note that here pyramid.paster.bootstrap parses the configuration file again:

 @pytest.fixture(scope='session') def app(request, ini_settings, **settings_overrides): """Initialize WSGI application from INI file given on the command line. TODO: This can be run only once per testing session, as SQLAlchemy does some stupid shit on import, leaks globals and if you run it again it doesn't work. Eg trying to manually call ``app()`` twice:: Class <class 'websauna.referral.models.ReferralProgram'> already has been instrumented declaratively :param settings_overrides: Override specific settings for the test case. :return: WSGI application instance as created by ``Initializer.make_wsgi_app()``. """ if not getattr(request.config.option, "ini", None): raise RuntimeError("You need to give --ini test.ini command line option to py.test to find our test settings") data = bootstrap(ini_settings["_ini_file"]) return data["app"] 

In addition, configure a functional test server:

 import threading import time from wsgiref.simple_server import make_server from urllib.parse import urlparse from pyramid.paster import bootstrap import pytest from webtest import TestApp from backports import typing #: The URL where WSGI server is run from where Selenium browser loads the pages HOST_BASE = "http://localhost:8521" class ServerThread(threading.Thread): """ Run WSGI server on a background thread. Pass in WSGI app object and serve pages from it for Selenium browser. """ def __init__(self, app, hostbase=HOST_BASE): threading.Thread.__init__(self) self.app = app self.srv = None self.daemon = True self.hostbase = hostbase def run(self): """Open WSGI server to listen to HOST_BASE address """ parts = urlparse(self.hostbase) domain, port = parts.netloc.split(":") self.srv = make_server(domain, int(port), self.app) try: self.srv.serve_forever() except Exception as e: # We are a background thread so we have problems to interrupt tests in the case of error import traceback traceback.print_exc() # Failed to start self.srv = None def quit(self): """Stop test webserver.""" if self.srv: self.srv.shutdown() @pytest.fixture(scope='session') def web_server(request, app) -> str: """py.test fixture to create a WSGI web server for functional tests. :param app: py.test fixture for constructing a WSGI application :return: localhost URL where the web server is running. """ server = ServerThread(app) server.start() # Wait randomish time to allows SocketServer to initialize itself. # TODO: Replace this with proper event telling the server is up. time.sleep(0.1) assert server.srv is not None, "Could not start the test web server" host_base = HOST_BASE def teardown(): server.quit() request.addfinalizer(teardown) return host_base 

Answer 3

If someone else does not get @ antti-haapala's answer right away:

Create test.ini populated with:

 [app:main] use = config:development.ini#main 

(Actually this step is not required. You can also save your development.ini and use it instead of test.ini in the following code. However, a separate test.ini can be useful if you need separate settings for testing)

In your test.py add:

 from pyramid.paster import get_appsettings settings = get_appsettings('test.ini', name='main') 

and replace

 app = TestApp(main({})) 

with

 app = TestApp(main(global_config = None, **settings)) 

The following comment was appropriate for this answer: The pyramid does not start when webtest and sqlalchemy are used together