RabbitMQ plugin stuck in run status

Question

RabbitMQ starts just fine, but the state of the smooth plugin is displayed as "start".

I am using the following rabbitmq.config file:

Each broker runs on a separate instance of AWS. The remote server is Windows 2008, the local server is Amazon Linux.

[{rabbitmq_shovel, [{shovels, [{scrape_request_shovel, [{sources, [{broker,"amqp://test_user: test_password@localhost "}]}, {destinations, [{broker, "amqp://test_user: test_password@ec2- ###-##-###-###.compute-1.amazonaws.com"}]}, {queue, <<"scp_request">>}, {ack_mode, on_confirm}, {publish_properties, [{delivery_mode, 2}]}, {publish_fields, [{exchange, <<"">>}, {routing_key, <<"scp_request">>}]}, {reconnect_delay, 5} ]} ] }] }]. 

Running the following command:

sudo rabbitmqctl eval 'rabbit_shovel_status: status ().'

returns:

[{scrape_request_shovel, starting, {{2012,7,11}, {23,38,47}}}]

According to this question , this can happen if users have not been configured correctly on two brokers. However, I double-checked that I configured users correctly using rabbitmqctl user_add on both machines - I even tried it with a different set of users to make sure.

I also checked the nmap test of port 5672 on the remote host to see if this port was up and running.

UPDATE The problem is not resolved, but this appears to be the result of problems connecting to the remote server. I changed reconnect_delay to 0 in my configuration file to avoid trying to reconnect again. I strongly recommend that others with this problem do this as well, as this allows you to receive error messages from rabbit_shovel_status. In my case, I got the following error:

 [{scrape_request_shovel, {terminated, {{badmatch,{error,access_refused}}, [{rabbit_shovel_worker,make_conn_and_chan,1}, {rabbit_shovel_worker,handle_cast,2}, {gen_server2,handle_msg,2}, {proc_lib,init_p_do_apply,3}]}}, {{2012,7,12},{0,4,37}}}] 

Answer 1

Answering my own question here if others run into this problem. This error (as well as the timeout error if you received it, {{badmatch, {error, etimedout}}) is almost certainly a communication problem between the two machines, most likely due to the settings for accessing the portal / firewall.

There were a few dumb things I did here:

1) Used the wrong DNS for my remote instance of EC2 (D'oh! Really dumb - I can’t say how long I spent hitting my head against the wall on this ...). Remember that stopping and starting your instance generates a new DNS if you do not have an elastic IP address associated with the instance.

2) My remote instance is a Windows server, and I realized that you need to open port 5672 both in the Windows firewall and in the EC2 security groups. There are two overlapping levels of access control and opening a port in the EC2 management console is not enough if your computer is a Windows server on EC2, since you also need to configure the Windows Server firewall.