How to run py-fast-rcnn with X11 forwarding

Question

I am running py-fast-rcnn with cuDNN enabled in the g2.8xlarge instance of EC-2 with the Ubuntu 14.04 operating system. Everything compiled and seems to work fine. I enter the remote instance via:

ssh -X -i "<key.pem>" ubuntu@ <IP address> 

I also enter the command: export DISPLAY=:0

Running ./tools/demo.py The result looks good:

 Loaded network /home/ubuntu/py-faster-rcnn/data/faster_rcnn_models/VGG16_faster_rcnn_final.caffemodel ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Demo for data/demo/000456.jpg Detection took 0.543s for 300 object proposals ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Demo for data/demo/000542.jpg Detection took 0.506s for 161 object proposals ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Demo for data/demo/001150.jpg Detection took 0.507s for 194 object proposals ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Demo for data/demo/001763.jpg Detection took 0.507s for 196 object proposals ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Demo for data/demo/004545.jpg Detection took 0.541s for 300 object proposals 

But the graphical output does not appear in my XQuartz window.

Has anyone else solved this? You need to figure out how to use AWS equipment, but with local visualization capabilities. Checked here, but I could not solve my problem: BVLC / caffe # 861

EDIT

Here are links to my remote sshd_config and local ssh_config .

The following are the results of the requested tests on a remote AWS server:

 $ echo $DISPLAY localhost:10.0 

and

 $ DISPLAY=localhost:10.0 xhost && echo success access control enabled, only authorized clients can connect success 

UPDATE

Executing the xeyes and xcalc on the remote computer after applying the above steps leads to the expected result (eyes, first, the calculator in the second) on the local client. This is probably a problem with python. Start searching there.

SOLUTION - UPDATE OF PYTHON LIBRARIES

After verifying that my system was configured to support X11 forwarding with the guidance of those who responded to this post, I focused on conducting a series of python tests to see if matplotlib works matplotlib with X11. You can verify this yourself by running this script interactively . If xcalc and xeyes are working properly, but this script is causing an error, the problem lies in python / matplotlib .

Since then I fixed the problem, so I don’t have the error that appeared on hand, but the steps to fix it on Ubuntu 14.04, g2.8xlarge EC2 were as follows:

• Install gobject python: sudo apt-get install python-gobject-dev
• Install python-tk: sudo apt-get install python-tk
• Install pygtk:
  • wget http://ftp.gnome.org/pub/GNOME/sources/pygtk/2.24/pygtk-2.24.0.tar.gz
  • tar -xvzf pygtk-2.24.0.tar.gz
  • cd pygtk-2.24.0
  • ./configure
  • make
  • sudo make install
  • cd
• If matplotlib was installed using the package manager, for example. pip , uninstall it and reinstall from source:
  • sudo pip uninstall matplotlib
  • git clone https://github.com/matplotlib/matplotlib.git
  • cd matplotlib
  • sudo python setup.py install
  • cd
• Not sure if this is necessary, but sudo apt-get install xorg openbox successfully executed.

After completing the above steps, python ./tools/demo.py with the root directory py-faster-rcnn returns the expected images of the bounding fields and class probabilities.

Answer 1

I assume there is an X server on your local host, right?

There are two files:

• /etc/ssh/ssh_config : it is located on your local host; it is used with the client command: ssh .
• /etc/ssh/sshd_config : it is located on a remote instance; it is used with an sshd server running on a remote instance.

X11Forwarding yes and X11DisplayOffset 10 are 2 values ​​that must be set for sshd : the ssh daemon running on the remote instance. Modify the / etc / ssh / sshd _config file on the remote instance. Perhaps after the change, sshd in the remote instance will be restarted.

Connect again using ssh -X ... and check the value of the DISPLAY variable. The example below is correct if X11DisplayOffset 10 installed:

 # ssh -X -i "<key.pem>" ubuntu@ <IP address> # echo $DISPLAY localhost:10.0 

If the DISPLAY variable is not set or is not set to a different value (something in .bashrc.profile, etc. changed its value at login), then this is a problem.

Another way is to check with xhost and install the environment on the same line:

 # ssh -X -i "<key.pem>" ubuntu@ <IP address> # DISPLAY=localhost:10.0 xhost && echo success 

If an error message is displayed, this is a problem to fix: edit your question and add an error message. Otherwise, the status bar and the word success displayed, continue:

 # DISPLAY=localhost:10.0 xcalc 

A window should open on the local host. Finally, continue:

 # DISPLAY=localhost:10.0 ./tools/demo.py 

Answer 2

I also enter the command: export DISPLAY=:0

This step may be your problem. When you forward X through SSH, you almost never need to manually install DISPLAY on a remote system.

If ssh successfully sends X, it will set the DISPLAY environment variable in the remote session to the correct value. If DISPLAY is not installed on the remote system, this means that ssh cannot negotiate X forwarding with the server (otherwise, if something in your shell startup removes the variable). By setting DISPLAY yourself, you can overwrite the correct variable value. At least the value you set for it is unlikely to be correct for forwarding via SSH.

The DISPLAY value ": 0" means the connection through the X system by default, which can include the connection through the unix-domain socket or through the TCP port 6000. This is intended for clients to access the X server running on the same system as the client (in this case server EC2).

For an SSH session, the correct DISPLAY value will usually be something like "localhost: 10", which means that clients on the remote system must connect to localhost port 6010 (6000 + 10). The number 10 may differ if port 6010 is already in use or if the SSH server is configured to use a different range of ports.

If you find that DISPLAY is not installed at logon, despite running ssh with the "-X" or "-Y" option, it is possible that X11 forwarding was disabled on the ssh server. There are three relevant options on the OpenSSH server that can be set in sshd_config :

X11DisplayOffset
Defines the first display number available for sshd (8) X11 forwarding. This prevents sshd from interfering with real X11 servers. The default value is 10.

X11Forwarding
Indicates whether X11 forwarding is allowed. The argument must be yes or no. The default is no.

If X11 forwarding is enabled, there may be an additional exposure for the server and for displaying the client if the sshd (8) proxy screen is configured to listen for a wildcard address (see X11UseLocalhost below), although this is not the default value. In addition, authentication and authentication and data authentication occurs on the client side. The security risk of using X11 forwarding is that the X11 client mapping server may be attacked when the SSH client requests forwarding (see Warnings for ForwardX11 in ssh_config (5)). The system administrator may have a position in which they want to protect clients who can attack themselves by inadvertently requesting an X11 forwarding, which can guarantee a no.

Note that disabling X11 forwarding does not prevent users from forwarding X11 traffic, as users can always set their own forwarders. X11 forwarding is automatically disabled if UseLogin is enabled.

X11UseLocalhost
Indicates whether sshd (8) should bind the X11 forwarder with a feedback address or a wildcard address. By default, sshd associates the forwarder with the loopback address and sets the host name part of the DISPLAY environment variable "localhost". This prevents remote hosts from connecting to the proxy display. However, some older X11 clients may not work with this configuration. X11UseLocalhost can be set to "no" to indicate that the forwarding server should be bound to a wildcard address. The argument must be yes or no. The default is yes.

The OpenSSH ssh client usually connects to the server, even if the server refused to forward. This may hide the fact that you requested forwarding, but the server refused. ssh has the ability to set in ssh_config to control this:

ExitOnForwardFailure
Indicates whether ssh (1) should terminate the connection if it cannot configure all requested dynamic, tunnel, local and remote port forwardings (for example, if either end cannot communicate and listen on the specified port). Note that ExitOnForwardFailure does not apply to port forwarding connections, and, for example, will not cause ssh (1) to exit if TCP connections to the final forwarder fail. The argument must be yes or no. The default is no.