Convention for RESTful search in Rails?

Back in RailsConf Europe last year, at David’s Keynote, it was said that:

There are unresolved decisions with respect to the restful controllers. In particular, what should the convention be for searching? A separate action? Or parameters passed to the index action?

I don’t suppose a convention has been adopted for this yet? I’m just about to implement search in an application I’m working on and I’d rather go with the 2.0 convention now, rather than fight against it with my wrong decision later. :-)

Update Judging by the implementation of ActiveResource#find it’s parameters passed to index, isn’t it?

Update 2 OK, so what’s the elegant, reusable implementation for FooController#index, turning params into find(:all, :conditions => ...)?

Update 3 I’ve started codifying what I’m doing into a plugin: resource_search. It’s still in its infancy.

  • http://autopilotmarketing.com/ Dan Kubb

    I use FoosController#index for all the searching I do, and in general I follow the conventions used by ARs, where query string parameters are used to “scope” the results.

    I take query_parameters, and filter against Foo#content_columns to get a conditions hash I can pass into Foo#find.

    For ordering of the results by convention I use two other query string params: :by and :sort. The param :by is the attribute to sort the results by, and if :sort == ‘reverse’ the I add DESC to Foo#find’s :o rder argument.

    For pagination, I just stick to the conventions of will_paginate, using :page to denote the page of results I’m on, and :per_page to set the number of results to use per page. When there’s no per_page I just default to something reasonable (for me) like 25.

    The only thing I haven’t quite decided on is how to handle general keyword searches, like if someone wanted to search across a couple of attributes for records that match. Under the hood you’d use something like Ferret to index the results. I had just thought about using a :q parameter, and pass in whatever the user types in to the search library’s methods.

    On a related note, it would be absolutely awesome if there was a plugin that made this easy, and also added an View that returned an OpenSearch description document so that restful resources could be searched by OpenSearch compatible bots.

  • Trevor

    Exactly what I was looking for, thank you!

    I got an error about Array#unzip, so I dropped this in:

    class Array def unzip( n=2 ) (0…n).map { |i| map { |tuple| tuple[i] } } end end

  • Trevor

    Also -

    Since I’m passing to will_paginate I found it handy to replicate with_search_terms slightly so I can just extract the proper conditions string:

    def create_search_terms(params) conditions = params.blank? ? [] : params.delete_if do |key, value| !column_names.include?(key) end.collect do |key, value| ["LOWER(#{table_name}.#{key}) LIKE ?", "%#{value.downcase}%"] end.unzip #end #raise conditions.inspect

    conditions.length > 1 && conditions[0] = conditions[0].join(‘ AND ‘) conditions = conditions.flatten

    conditions.empty? ? nil : conditions end

  • http://woss.name/ Graeme Mathieson

    Trevor: Ah, silly me, Array#unzip is defined in my rubaidh_platform plugin where I keep all the generic stuff I use across projects. I should copy the definition across into this plugin to make it independent.

    As for using it with will_paginate, I already am. I can’t remember if I’ve changed something since you popped by, but I’m now doing something along the lines of:

    class FooController < ApplicationController
      resource_search

    def index @foos = Foo.paginate :page => params[:page] end #... end

    which does the right thing. resource_search, in the background, creates an around_filter (by default just on index, though you can specify additional collection methods) which yields with_search_terms which is using with_scope. All very neat, if I do say so myself. :)

  • Rails Futzer

    So you’re looking at column names – shouldn’t you look at attr_accessible/attr_protected as well? Wouldn’t just looking at column names let someone search for something they shouldn’t?

  • http://woss.name/ Graeme Mathieson

    Rails Futzer: You’re absolutely right. In the application I extracted the plugin from and the application I first pushed it into I didn’t have any protected attributes for the models I was searching. I’ll be sure to implement that when I first need it. Patches gratefully accepted, of course. :-)

  • http://highearthorbit.com Andrew Turner

    I’ve been wondering if something like OpenSearch is a good fit here for defining the types of search parameters that could be used for a Resource.

    It provides general search term, pagination, and even geo & time support. Could then extend it as necessary to including other – or generic – search criteria.

  • Andy Filer

    Great stuff. I’m glad to see someone’s talking about this. I tried the plugin, and it works for me except that I had to change “model.constantize.with_search_terms(params[model.downcase]) do” to “model.constantize.with_search_terms(params) do”.

  • Andy Filer

    Never mind about the change. I see now that the format isn’t just param=value but model[param]=value, of course.

  • Andy Filer

    Great stuff. I’m glad to see someone’s talking about this. I tried the plugin, and it works for me except that I had to change “model.constantize.with_search_terms(params[model.downcase]) do” to “model.constantize.with_search_terms(params[model.underscore]) do”.

    (This is my third post on this, and I was a bit confused — I got thrown off because multi-word model names weren’t working).

  • http://woss.name/ Graeme Mathieson

    Andy: Good catch. I’ve just committed the change to subversion. Thanks!

  • http://N/A Toby Clemson

    Hi, I seem to be having a bit of trouble getting your plugin to work for me. Am I right in thinking that all that is necessary is to include the resource_search call in the controller? And this automatically wraps any method defined in the model in the scope of the conditions? Including inherited methods like find?

    I’m using edge rails – would that make a difference? Also I’m using the restful_authentication plugin which may be conflicting(?).

  • http://N/A Toby Clemson

    Ok after reading up on with_scope, I’ve narrowed my problem down a little. It was because I was using a query string of ?key=val rather than the escaped version of model[key]=val.

    The trouble is that ActiveResource currently uses the ?key=val form if you call Resource.find(:all, :params => {:key => ‘var’}) and I’m trying to use your plugin in a project involving a remote component using ARes.

    Can you suggest a modification to your plugin that would support the current ARes way of searching? And also support the query string method and the post method? I’d write a patch but I can’t think of an easy way to do it.

    I really like the way the plugin works at the moment – very smart way of implementing restful search and it doesn’t get in the way at all.

  • http://mike.vincent.ws Mike Vincent

    Nice plugin Graeme.

    I modified it to add an OR #{table_name}.#{key} IS NULL if the value was nil to allow empty to allow matching nulls. might be mysql specific, I’m not sure. Here’s what my modified with_search_terms method looks like:

            def with_search_terms(params)
              conditions = params.blank? ? [] : params.delete_if do |key, value|
               !column_names.include?(key)
              end.collect do |key, value|
                condition = "LOWER(#{table_name}.#{key}) LIKE ?"
                condition = "(#{condition} OR #{table_name}.#{key} IS NULL)" if value.nil?
                [condition, "%#{value.downcase if value}%"]
              end.unzip
              conditions.length > 1 && conditions[0] = conditions[0].join(' AND ')
              conditions = conditions.flatten.compact
              with_scope :find => {:conditions => conditions.empty? ? nil : conditions} do
                yield
              end
            end
    

    sure that will not look pretty :)

    I also ran into issues with the page links will_paginate generated if the results ended up paginated. Seems in 1.2.3 has issues handling nested params, so the links that will_paginate generated were somewhat munged. :( Ryan Kinderman discusses the issue in greater detail and provides a plugin to fix it over at http://kinderman.net/articles/2007/02/07/passing-arrays-and-nested-params-to-url_for-in-rails

    Maybe helpful to others not using edge rails.

  • Pingback: GSIY … Ruby-Rails Portal

  • Kevin Schmeichel

    Any thoughts about extending this to also search by foreign key objects? As a concrete example, I have a Business model that has a foreign key to an Address, and I want to be able to find all of a certain kind of business within 10 miles of me.

    How would I specify that certain search parameters apply to the foreign key model?

  • Mike Burrows

    Nice!

    <

    p> I added this little helper to your form_helper.rb which (I hope) encapsulates the expected naming conventions for search field (and remembers their values)

    
            def search_field_for(model, field, options = {})
              text_field_tag "#{model}[#{field}]", params[model].nil? ? nil : params[model][field]
            end
    

    <

    p> I’m having issues with

    will_paginate

    though – it’s making a bit of a mess of the search parameters in the navigation links, losing rather than quoting the special characters. Is it something I’m doing wrong?

  • Mike Burrows

    Oops, messed the formatting up a bit there. I think I’m encountering the same problem you referred to previously in http://woss.name/2007/01/14/correct-route-generation-for-params-which-are-hashes/

    Did you find a solution?

  • Richie Vos

    @Toby I haven’t tried this plugin yet, but if all the params are nested under the model name (model[x], model[y]), you can get at all of them by just doing: find(:first, :conditions => params[:model])

    Is that what you were talking about?

  • Toby Clemson

    @Richie – The trouble I was having was that when using ActiveResource (i.e., the client API for REST web services) if I called:

    Resource.find(:all, :params => {:key => :val})

    it would generate a request to the application with the url:

    GET http://server.com/resource?key=val

    But resource_search would expect the parameter has to contain model[key] rather than just key.

    I wanted to know a way of making this plugin work regardless of the format of the query string or the POST parameters.

    I hope that makes more sense.

    Toby

  • Don Morrison

    Hi Graeme. I was looking at this plugin today. Very nice. It is exactly the kind of thing I wanted to do on models in my application. In reading the comments and in using it I noted that you have to prefix the key/val with the model name. I have a little hack to let you do it this way or the ARes way that Toby mentions above.

    Before I clean it up (it isn’t much), are you still using/maintaining/accepting patches for this plugin?

    Thanks for your hard work. Don

  • http://woss.name/ Graeme Mathieson

    Don: I am about to refactor the plugin with a number of improvements I’ve been needing for another project. Please do send your patch to me and I’ll make sure it’s incorporated. :-)

  • http://edseek.com/ Jason Boxman

    Neat plugin. For determining the model names from the controller, you might want to look at how the resource_controller plugin handles it. It opionally lets you specificy a different model from the one derived from the controller name, too.

    For the SQL, you might want to look at the squirrel plugin just for fun. It’s a neat plugin to Rubyize the conditions for AR#find. You can obtain just the SQL by using it with Model.find(:query) do foo == ‘bar’ end.conditions.to_sql

    Thanks!

  • http://edseek.com/ Jason Boxman

    I know this plugin is more intended as a one-shot deal to allow for searching a model, but it might be nice to be able to override the SQL and produce a more crafted query per model. I added the followingly quickly:

    My model (using squirrel plugin for SQL):

    def self.search_scope_conditions(params={}) return nil if params.blank? self.find(:query) do last == params[:last] end.conditions.to_sql end

    model.rb:

    def with_search_terms(params) conditions = search_scope_conditions(params) with_scope :find => {:conditions => conditions} do yield end end

    Maybe I should make it accept a block or a symbol.

    Just a thought.