Can I include system libraries (e.g. libxml2) that I compile into a gem (e.g. nokogiri) that I can deploy to Heroku?

Question

Nokogiri has a problem translating to and from UTF-8 characters, which, as it turned out, comes from libxml2, in particular version 2.7.6, which is the highest supported version on Ubuntu 10.04 LTS. The bug is fixed in version 2.7.7 and higher, but since our application is hosted on Heroku (bamboo-ree-1.8.7 stack based on Ubuntu 10.04), we must use version 2.7.6 and continue to experience the error if:

• Someone may / hacked nokogiri to get around the problem.
• Canonical hits the supported version of libxml2 for Ubuntu 10.04 (and / or Heroku updates libxml2 in its stack)
• I can come up with a way for nokogiri to use the version of libxml2 that I can associate with the application so that it can be deployed to Heroku.

I am happy to hear any feedback on 1 or 2, of course, but I wonder if it is possible 3. Here is what I know to be possible:

• Nokogiri can be associated with a library located in a non-standard place - http://nokogiri.org/tutorials/installing_nokogiri.html
• A gem created in this way can be installed in a vendor / gemstone and thus have an advantage over system gemstones.

My question is, can I enable a higher-level libxml2 with the application so that the compiled vendor gem uses it when I committed it and pushes it to Heroku?

Answer 1

This is not the most user-friendly solution, but here is what I did once to get the custom version of libpq. I did it on cedar, but it will probably work on bamboo if you go into tmp first

• heroku run bash
• twist the source of what you want to build
• build it
• send binaries from dynamic
• embedded binary provider in provider / other
• heroku config: add LD_LIBRARY_PATH = / app / vendor / whatever

Sorry, there is no better way to get custom binaires yet. Good luck.

Answer 2

I'm not sure about the exact answer, but I found a problem raised for Nokogiri that sounds like the problem you just described: https://github.com/tenderlove/nokogiri/issues/458

One answer says how they are hosted on Heroku and blocked to version 2.7.6 from libxml2, and demanded that their case be updated to 2.7.8 in order to fix their problem: https://github.com/tenderlove/ nokogiri / issues / 458 # issuecomment-2600583

Although it is unlikely, maybe you should contact Heroku to find out what they can do (if any), or if they can have any suggestions? I would risk assuming that they will not change the version of libxml2, as on a locked stack (Bamboo), but who knows?

Answer 3

I have a slightly better solution. The more I play with the gazebo, the more I like it. Bower is just a tool built on top of git and structures things in a bizarre manner (usually oriented to front-end JS scripts / not a requirement). One more note: bower is great for installing private dependencies as well as github.

Python is an example, but the overall goal is the same and can be adjusted for ruby.

So take this project for example 3scale_python .

This requires installing libxml2 (via its .txt requirements file). The easiest way for me is to include it in bower and then write a script to install it (shell script and procfile).

bower.json

{ "name": "someapp", "version": "0.0.1", "homepage": "http://github.com/yourusername/yourrepo", "authors": [nem], "description": "something that uses 3scale at a specific commit", "main": "./lib/clock.py", "private": true, "dependencies": { "3scale_ws_api_for_python": "http://github.com/3scale/3scale_ws_api_for_python/archive/82328aa8e7d43f7ef89e420921a4d63e025b527f.zip" } }

install script (manual_installs_python)

 #!/bin/bash set -e oldPath=$(pwd) installing(){ echo "-- installing $1 dependencies --" echo '--------------------------------' echo } #check for brew, if brew then use it to install something osx_brew() { #if brew is installed (no error) if hash brew 2 > /dev/null; then installing 'brew' brew " $@ " fi } installing 'python easy_install' easy_install figleaf cd ./bower_components/some_other_python_app python setup.py install if [[ $IS_HEROKU = 1 ]]; then #installing on ubuntu/heroku only as, so far I have not gotten libxml2-python to install successfully on OSX cd ../3scale_ws_api_for_python #requirements.txt here actually installs and compiles libxml2 pip install -r requirements.txt python setup.py install fi cd "$oldPath" echo "-- done with dependencies -- " exit 0 

PROCFILE

web: pip install -r requirements.txt && ./scripts/manual_python_installs

Done

Installing libxml2 is too easy, you might want to consult scripts. 3scale disconnects through its requirements file.