How to configure Meteor on AWS / EBS using the METEOR_SETTINGS environment variable

Question

Trying to configure Meteor on AWS / EBS (Amazon Web Services, Elastic Beanstalk).

The Meteor debugger can be passed a command line flag: --settings settings.json where settings.json is a file containing server / client / value configurations (like correctly formatted JSON).

Instead of passing the configuration file on the command line, the Meteor deployment uses the METEOR_SETTINGS environment METEOR_SETTINGS . If provided, it is expected that it will contain a json document, such as the contents of settings.json , for example:

 $ METEOR_SETTINGS=$(cat settings.json) $ echo $METEOR_SETTINGS { "public": { "s3path": "https://d2v4p3rms9rvi3.cloudfront.net" } } 

The problem is that when I set the METEOR_SETTINGS value for this value in the EBS console:

AWS / EBS discards quotes, avoids slashes (as shown in the screenshot), and sends a Meteor:

 {public:{s3path:https:\/\/d2v4p3rms9rvi3.cloudfront.net}} 

As indicated by the node launch error:

 ------------------------------------- /var/log/nodejs/nodejs.log ------------------------------------- npm WARN deprecated backwards-incompatible changes made to `npm run-script` and npm WARN deprecated semver behavior. > meteor-dev-bundle@0.0.0 start /var/app/current > node main.js /var/app/current/programs/server/boot.js:283 }).run(); ^ Error: METEOR_SETTINGS are not valid JSON: {public:{s3path:https:\/\/d2v4p3rms9rvi3.cloudfront.net}} at packages/meteor/packages/meteor.js:21:1 at Package (packages/meteor/packages/meteor.js:42:1) at /var/app/current/programs/server/packages/meteor.js:1277:4 at /var/app/current/programs/server/packages/meteor.js:1286:3 at /var/app/current/programs/server/boot.js:242:10 at Array.forEach (native) at Function._.each._.forEach (/var/app/current/node_modules/underscore/underscore.js:79:11) at /var/app/current/programs/server/boot.js:137:5 

Attacking this problem, I tried all kinds of variations for the JSON object in the value field: avoiding quotes, covering the entire json part with single quotes, replacing double quotes with single quotes and other attempts - and didn't solve it.

Question:

How to set METEOR_SETTINGS so that Meteor rcv and correctly disassemble it?

Note. One of the requirements is that the same assembly is deployed in dev, staging and production environments. Configurations must be set separately for each environment, so if there is another way to insert settings into the EBS environment without modifying the assembly, which will also solve it.

Answer 1

After discussing this issue with AWS support, I realized that AWS / EBS does not support storing JSON in environment variables. This is because environment variables are stored as key / value strings in unencrypted JSON (apparently in CloudFormation). The point here is a little disappointing:

METEOR_SETTINGS cannot be used in AWS / EBS console

This is really unsuccessful, but there are several workarounds.

First decision

Move json configs into the s3 bucket and put the following contents in the .ebextensions/app.config :

 container_commands: 01_setvariable: command: "aws s3 cp s3://<bucket-name>/nodejs.conf /tmp/deployment/config/#etc#init#nodejs.conf 

This will completely override the /etc/init/nodejs.conf contents received from your s3 bucket. Naturally, it is possible to set / uncheck individual settings using finely tuned / bizarre bash scripts.

I did not choose this method because it includes another object (S3 bucket), and iterating dev requires the deployment of a new version, which is not very fast.

2nd workaround

Note: this is a simple hack code that I came up with. It seems that all this was confusing, without requiring much effort.

My initial need was to distribute AVS / EBS env vars to the client, so I decided to bypass the METEOR_SETTINGS variable and populate Meteor.settings.public directly with env vars from node process.env . The whitelist is managed by a simple list. Add the server/lib/config.js with:

 Meteor.startup(function () { // public settings that need to be exposed to the client can be added here var publicEnvs = { S3_PATH: 's3path' }; var modified; _.each(publicEnvs, (value, key) => { let envValue = process.env[key]; if (envValue) { Meteor.settings.public[value] = envValue; modified = true; } }); if (modified) { __meteor_runtime_config__.PUBLIC_SETTINGS = Meteor.settings.public; } }); 

Hooray, your client can access the env vars of your choice!

For example, with this change, the S3_PATH environment variable defined in the EBS console may be available as Meteor.settings.public.s3path on the client. Pretty simple and without many moving parts :)

Answer 2

NEW SOLUTION

It is easier than I can come. I created a bash script that automatically generates a compressed assembly of your project and combines the settings file, so you really don't need to do anything.

BEFORE YOU SCRIPT

Create a meteor in your project. / lib / beanstalk -settings-fix.js

 /*============================================================================== * Globals *============================================================================*/ /*global process*/ /*global Meteor*/ /*global Npm*/ if (Meteor.isProduction){ var meteorFile = process.env.METEOR_SETTINGS_FILE; if(meteorFile == undefined) throw new Error( 'METEOR_SETTINGS_FILE env variable must be defined in production.') var fs = Npm.require('fs'); var pjsonBuf = fs.readFileSync( meteorFile ); Meteor.settings = JSON.parse( pjsonBuf.toString().trim()); } 

HOW TO USE IT

• Copy the following code into a text file and save it as build.sh
• Edit the constants.
• Give it permission to run and run it.

You will end up with something like project-name.zip to load it into your beanstalk environment. Hope you find this helpful!

This solution is based on AWS forums . If you want to check out old solutions, check out the change history.

 #!/bin/bash #=============================================================================== # DESCRIPTION: #=============================================================================== # This script creates a build of the project ready to be uploaded to beanstalk. # Requires pyton 2.7.x #=============================================================================== # COMMON ISSUES: #=============================================================================== # -If you upload the output to a sample application, it will fail. # -Version format must be 0.0.0 #=============================================================================== # CONSTANTS #=============================================================================== CURRENT_VERSION="1.0.0" OUTPUT_NAME="file-name-without-extension" PRODUCTION_SETTINGS_JSON="./your-project-directory/settings-prod.json" PROJECT_DIRECTORY="./your-project-directory" OUTPUT_DIRECTORY="./the-output-directory" ROOT_URL="http://www.SOMEENVIRONMENT-env.us-west-2.elasticbeanstalk.com" MONGO_URL="none" #=============================================================================== # SAY HELLO #=============================================================================== initial_directory=$(pwd) # This file local path clear echo "COOKING OUTPUT" echo "=========================================================" #=============================================================================== # RAW PROJECT BUILD #=============================================================================== echo "> BUILDING RAW PROJECT" cd $initial_directory cd $PROJECT_DIRECTORY rm -f -R "../build/bundle" meteor build --directory ../build/ #=============================================================================== # SET PRODUCTION ENVIRONMENT VARIABLES #=============================================================================== cd $initial_directory json=`cat $PRODUCTION_SETTINGS_JSON` cd $OUTPUT_DIRECTORY/bundle mkdir -p .ebextensions echo "option_settings:" >> .ebextensions/environment.config echo " - option_name: MONGO_URL" >> .ebextensions/environment.config echo " value: $MONGO_URL" >> .ebextensions/environment.config echo "option_settings:" >> .ebextensions/environment.config echo " - option_name: ROOT_URL" >> .ebextensions/environment.config echo " value: "$ROOT_URL"" >> .ebextensions/environment.config echo "files:" >> .ebextensions/environment.config echo " '/tmp/settings.json':" >> .ebextensions/environment.config echo " content : |" >> .ebextensions/environment.config echo " "$json >> .ebextensions/environment.config echo "option_settings:" >> .ebextensions/environment.config echo " - namespace: aws:elasticbeanstalk:application:environment" >> .ebextensions/environment.config echo " option_name: METEOR_SETTINGS_FILE" >> .ebextensions/environment.config echo " value: '/tmp/settings.json'" >> .ebextensions/environment.config chmod 444 .ebextensions/environment.config echo "> ADDING 'settings.json' AS ENV VAR" #=============================================================================== # CREATE PACKAGE.JSON #=============================================================================== cd $initial_directory cd $OUTPUT_DIRECTORY/bundle # Write base package.json echo '{ "name": "'$OUTPUT_NAME'", "version": "'$CURRENT_VERSION'", "scripts": { "start": "node main.js" }, "dependencies": { } }' > ./package.json # Add dependencies from meteor in packages.json # Then add extra dependencies defined by us. EXTRA_DEPENDENCIES='{"forever": "*"}' meteor_packages=$(cat ./programs/server/package.json) packages=$(cat ./package.json) packages_updated=`python <<END import json; # We cannot operate directly bash variables, so we make a copy. a = $packages b = $meteor_packages a['dependencies'] = b['dependencies'] for key, value in $EXTRA_DEPENDENCIES.iteritems(): a['dependencies'].update({key: value}) print json.dumps(a, sort_keys=False, indent=4, separators=(',', ': ')); END` echo "$packages_updated" > ./package.json chmod 444 ./package.json echo "> ADDING 'package.json'" #=============================================================================== # ZIP OUTPUT #=============================================================================== cd $initial_directory cd $OUTPUT_DIRECTORY/bundle zip -FSrq "../$OUTPUT_NAME-$CURRENT_VERSION.zip" . echo "> ZIP THE OUTPUT" #=============================================================================== # CLEAN THE HOUSE #=============================================================================== cd $initial_directory cd $OUTPUT_DIRECTORY rm -R -f ./bundle echo "> CLEAN THE HOUSE" #=============================================================================== # SAY GOODBYE #=============================================================================== echo "=========================================================" echo "YOU CAN UPLOAD THE PROJECT TO A BEANSTALK ENVIRONMENT NOW" 

Additional help:. If you want to check that everything went fine, you can find your final settings in your zipped output, in the /.ebextensions/environment.config section and in your package file in / package.json

Answer 3

Another workaround tested.

after meteor build --directory edit main.js as follows

 process.argv.splice(2, 0, 'program.json'); var settingfilename = './settings.json'; if (process.env.METEOR_SETTING_FILE) settingfilename = process.env.METEOR_SETTING_FILE; var settings = require(settingfilename); if (settings) { try { process.env.METEOR_SETTINGS = JSON.stringify(settings); } catch (e) { console.error(e); } } process.chdir(require('path').join(__dirname, 'programs', 'server')); require('./programs/server/boot.js'); 

and copy settings.json to bundle/ and eb init and eb deploy .

you can set another settings file with the addition of METEOR_SETTING_FILE on the Environment Properties tab on the Configuration tab of the EB console.

An editing file is needed after each build.

---

added patch file for use in assembly script, for example ed - ../build/bundle/main.js < main.js.patch

main.js.patch

 8a var settingfilename = './settings.json'; if (process.env.METEOR_SETTING_FILE) settingfilename = process.env.METEOR_SETTING_FILE; var settings = require(settingfilename); if (settings) { try { process.env.METEOR_SETTINGS = JSON.stringify(settings); } catch (e) { console.error(e); } } // console.log (JSON.stringify(process.env)); . w