Rails converts empty arrays to nils in query params

Question

I have a Backbone model in my application that is not a typical flat object, it is a large nested object, and we store the nested parts in TEXT columns in the MySQL database.

I wanted to handle JSON encoding / decoding in the Rails API, so from the outside it looks like you can POST / GET this one large nested JSON object, even if parts of it are stored as compressed JSON text.

However, I ran into a problem when Rails magically converts empty arrays to nil values. For example, if I READ this:

 { name: "foo", surname: "bar", nested_json: { complicated: [] } } 

My Rails controller sees this:

 { :name => "foo", :surname => "bar", :nested_json => { :complicated => nil } } 

And so my JSON data has been changed.

Has anyone encountered this problem before? Why does Rails change my POST data?

UPDATE

This is where they do it:

https://github.com/rails/rails/blob/master/actionpack/lib/action_dispatch/http/request.rb#L288

And that’s why they do it:

https://github.com/rails/rails/pull/8862

So now the question is, what is the best way to handle this in my nested JSON API situation?

Answer 1

After a long search, I found that you are starting with Rails 4.1, you can completely skip the "deep_munge" function using

 config.action_dispatch.perform_deep_munge = false 

I could not find any documentation, but you can view the introduction of this option here: https://github.com/rails/rails/commit/e8572cf2f94872d81e7145da31d55c6e1b074247

There is a security risk described here: https://groups.google.com/forum/#!topic/rubyonrails-security/t1WFuuQyavI

Answer 2

Looks like this is a known recently released issue: https://github.com/rails/rails/issues/8832

If you know where the empty array will be, you can always params[:...][:...] ||= [] in the front filter.

Alternatively, you can change the BackBone model to a JSON method by explicitly binding the nested_json value with JSON.stringify() before it is published, and manually rendering it back using JSON.parse in the before_filter file.

Ugly, but it will work.

Answer 3

You can reanalyze the parameters yourself, for example:

 class ApiController before_filter :fix_json_params [...] protected def fix_json_params if request.content_type == "application/json" @reparsed_params = JSON.parse(request.body.string).with_indifferent_access end end private def params @reparsed_params || super end end 

This works by searching for queries with the JSON content type, parsing the request body again and then intercepting the params method to return the parsed parameters, if they exist.

Answer 4

I had a similar problem.

Fixed it by sending an empty string as part of an array.

Ideally, your options should like

 { name: "foo", surname: "bar", nested_json: { complicated: [""] } } 

Therefore, instead of sending an empty array, I always pass ("") to my request to bypass the deep search process.

Answer 5

Here (I believe) is a reasonable solution that does not require re-analysis of the body of the raw request. This may not work if your client is POSTing the form data, but in my case I am ALMOST JSON.

in application_controller.rb :

  # replace nil child params with empty list so updates occur correctly def fix_empty_child_params resource, attrs attrs.each do |attr| params[resource][attr] = [] if params[resource].include? attr and params[resource][attr].nil? end end 

Then in your controller ....

 before_action :fix_empty_child_params, only: [:update] def fix_empty_child_params super :user, [:child_ids, :foobar_ids] end 

I came across this in my situation, if the POSTed resource contains either child_ids: [] or child_ids: nil , I want this update to mean “delete all children”. If the client intends not to update the child_ids list, it should not be sent to the POST body, in which case params[:resource].include? attr params[:resource].include? attr will be false , and request parameters will not be changed.

Answer 6

I ran into a similar problem and found out that passing an array with an empty string would be correctly handled by Rails, as mentioned above. If you encounter this when submitting a form, you can include an empty hidden field that matches the array parameter:

 <input type="hidden" name="model[attribute_ids][]"/> 

When the actual parameter is empty, the controller will always see an array with an empty string, thereby saving the sending without saving.

Answer 7

You can get around this problem here.

 def fix_nils obj # fixes an issue where rails turns [] into nil in json data passed to params case obj when nil return [] when Array return obj.collect { |x| nils_to_empty_arrays x } when Hash newobj = {} obj.each do |k,v| newobj[k] = nils_to_empty_arrays v end return newobj else return obj end end 

And then just

 fixed_params = fix_nils params 

which works as long as you do not have zeros in your parameters as intended.