Smart Listing
The smart_listing ruby gem, built by Sology developer Łukasz Jachymczyk, is a very powerful gem that I'm surprised isn't used by more developers.
Guide Tasks
  • Read Tutorial

What smart_listing Provides

The smart_listing ruby gem, built by Sology developer Łukasz Jachymczyk, is a very powerful gem that I'm surprised isn't used by more developers. I've integrated it into a number of applications that I have built and I really enjoy using it. So what does it do? smart_listing provides a full suite of table based view functionality, including:

  1. Pagination

  2. Sorting/Filtering

  3. Search

  4. In place editing (very cool feature)

  5. The ability to create new records directly from the table

Pretty feature rich, right? In addition to all of these cool features, one of my favorite aspects of the Gem is that all of the above features are all AJAX based, so they don't require a page refresh.

Drawbacks

I haven't really found very many drawbacks/limitations for this specific Gem, the only issue I had was that there wasn't great support for it in the Rails community from a training perspective (which is why I wanted to create this walk through).

Implementation / Smart Listing Example

The app we'll use for this walk through is going to be a simple Rails application that has a Movie model with the attributes of title:string and director:string. FYI, if you're following along, don't use a scaffold for this lesson, I used the resource generator, the scaffold would create the wrong type of files. Many of these view templates are going to be JavaScript based, so the scaffold ERB files would clutter everything up.

To begin, go to RubyGems.org and grab the latest stable version of the Gem and add it to the Gemfile, at the time of this post it's:

gem 'smart_listing', '~> 1.1', '>= 1.1.2'

After running bundle install make sure to update the following JavaScript call to the app/assets/javascripts/application.js file:

//= require jquery
//= require jquery_ujs
//= require smart_listing
//= require_tree .

I removed the Turbo Links call (I typically do this anytime I'm integrating outside JavaScript features to prevent conflicts) and swapped it out with the //= require smart_listing call.

Now let's create the initializer file by running the generator:

rails generate smart_listing:install

That creates config/initializers/smart_listing.rb and populates it with the following code:

SmartListing.configure do |config|
  config.global_options({
    #:param_names  => {                                              # param names
      #:page                         => :page,
      #:per_page                     => :per_page,
      #:sort                         => :sort,
    #},
    #:array                          => false,                       # controls whether smart list should be using arrays or AR collections
    #:max_count                      => nil,                         # limit number of rows
    #:unlimited_per_page             => false,                       # allow infinite page size
    #:paginate                       => true,                        # allow pagination
    #:memorize_per_page              => false,                       # save per page settings in the cookie
    #:page_sizes                     => DEFAULT_PAGE_SIZES,          # set available page sizes array
    #:kaminari_options               => {:theme => "smart_listing"}, # Kaminari's paginate helper options
    #:sort_dirs                      => [nil, "asc", "desc"],        # Default sorting directions cycle of sortables
  })

  config.constants :classes, {
    #:main                  => "smart-listing",
    #:editable              => "editable",
    #:content               => "content",
    #:loading               => "loading",
    #:status                => "smart-listing-status",
    #:item_actions          => "actions",
    #:new_item_placeholder  => "new-item-placeholder",
    #:new_item_action       => "new-item-action",
    #:new_item_button       => "btn",
    #:hidden                => "hidden",
    #:autoselect            => "autoselect",
    #:callback              => "callback",
    #:pagination_wrapper    => "text-center",
    #:pagination_container  => "pagination",
    #:pagination_per_page   => "pagination-per-page text-center",
    #:inline_editing        => "info",
    #:no_records            => "no-records",
    #:limit                 => "smart-listing-limit",
    #:limit_alert           => "smart-listing-limit-alert",
    #:controls              => "smart-listing-controls",
    #:controls_reset        => "reset",
    #:filtering             => "filter",
    #:filtering_search      => "glyphicon-search",
    #:filtering_cancel      => "glyphicon-remove",
    #:filtering_disabled    => "disabled",
    #:sortable              => "sortable",
    #:icon_new              => "glyphicon glyphicon-plus",
    #:icon_edit             => "glyphicon glyphicon-pencil",
    #:icon_trash            => "glyphicon glyphicon-trash",
    #:icon_inactive         => "glyphicon glyphicon-circle",
    #:icon_show             => "glyphicon glyphicon-share-alt",
    #:icon_sort_none        => "glyphicon glyphicon-resize-vertical",
    #:icon_sort_up          => "glyphicon glyphicon-chevron-up",
    #:icon_sort_down        => "glyphicon glyphicon-chevron-down",
    #:muted                 => "text-muted",
  }

  config.constants :data_attributes, {
    #:main                  => "smart-listing",
    #:controls_initialized  => "smart-listing-controls-initialized",
    #:confirmation          => "confirmation",
    #:id                    => "id",
    #:href                  => "href",
    #:callback_href         => "callback-href",
    #:max_count             => "max-count",
    #:item_count            => "item-count",
    #:inline_edit_backup    => "smart-listing-edit-backup",
    #:params                => "params",
    #:observed              => "observed",
    #:href                  => "href",
    #:autoshow              => "autoshow",
    #:popover               => "slpopover",
  }

  config.constants :selectors, {
    #:item_action_destroy   => "a.destroy",
    #:edit_cancel           => "button.cancel",
    #:row                   => "tr",
    #:head                  => "thead",
    #:filtering_icon        => "i"
    #:filtering_button      => "button",
    #:filtering_icon        => "button span",
    #:filtering_input       => ".filter input",
    #:pagination_count      => ".pagination-per-page .count",
  }
end

Łukasz did a nice job with the method names and they're very descriptive. Going through the file you can see it includes custom options such as:

  • The types of icons that are used - it uses Glyphicons by default, but we'll see how to customize this to show Font Awesome icons instead

  • The ability to set the default page size

  • The default CSS attributes

  • The pagination theme options - it uses Kaminari by default, so if you are adding this to an application that uses the will_paginate gem you'll need to add in some initializers to fix namespace issues. These are the two files you'll need to add config/initializers/kaminari.rb and config/initializers/will_paginate.rb. Once again, that's not necessary for this walk through, it's only if your app needs both the Kaminari and will_paginate gems.

  • And pretty much every other option you'd want to customize the behavior of the gem. Note: since this file is in the config directory you will need to restart the rails server after each change.

Now let's add in the smart listing view helpers, run this generator:

rails g smart_listing:views

This will create the following files for us:

app/views/smart_listing/_action_custom.html.erb
app/views/smart_listing/_action_delete.html.erb
app/views/smart_listing/_action_edit.html.erb
app/views/smart_listing/_action_show.html.erb
app/views/smart_listing/_item_new.html.erb
app/views/smart_listing/_pagination_per_page_link.html.erb
app/views/smart_listing/_pagination_per_page_links.html.erb
app/views/smart_listing/_sortable.html.erb

These view helpers can be shared by any/all other views in the application and they're necessary for quite a bit of the code implementation, so I'd always run this generator.

Now that we have the shared code for the views let's customize the controller so it can have access to the smart_listing methods for all CRUD functions:

# app/controllers/movies_controller.rb

class MoviesController < ApplicationController
  before_action :set_movie, only: [:show, :edit, :update, :destroy]
  include SmartListing::Helper::ControllerExtensions
  helper  SmartListing::Helper

  def index
    scope = Movie.all
    options = {}
    options = options.merge(query: params[:filter]) if params[:filter].present?
    options = options.merge(filters: params[:f]) if params[:f].present?
    scope = Movie.all_with_filter(options, scope)

    if params[:movies_smart_listing] && params[:movies_smart_listing][:page].blank?
      params[:movies_smart_listing][:page] = 1
    end

    @movies = smart_listing_create :movies, scope, partial: "movies/list", page_sizes: [10, 25, 50, 100, 250, 500]
  end

  def show
    respond_to do |format|
      format.json { render json: @movie }
    end
  end

  def new
    @movie = Movie.new
  end

  def edit
  end

  def create
    @movie = Movie.create(movie_params)
  end

  def update
    @movie.update_attributes(movie_params)
  end

  def destroy
    @movie.destroy
  end

  def search
    render json: Movie.where("LOWER(title) LIKE LOWER(?)", "%#{params[:q]}%")
  end

  private
    def set_movie
      @movie = Movie.find(params[:id])
    end

    def movie_params
      params.require(:movie).permit(:title, :director)
    end
end

Wow, that's a lot of code! Let's go through each of the actions one by one:

  • index - the index method is going to contain the majority of the connection between smart_listing and the view. It contains the initial ActiveRecord query and then creates an options hash that contains all of the query options that can customize what is sent to the view.

  • show - the show method is simply returning JSON when it's called for a specific record

  • new - the new method creates a new instance of the Movie model

  • edit - edit simply allows the edit form to render when it's called

  • create - create communicates with the database and adds the record, notice how I didn't add in the typical if/else code in the create method? This is because the create action is only going to be called via an AJAX request, so we don't want it to re-render another template. In a production application you'd want to add in some JSON responses to pass in error messages.

  • update - the update method simply takes in the AJAX request and updates the record, this makes it possible to update records in place instead of having to have their own view template page

  • destroy - deletes the record and can be called by an AJAX request

  • search - the search method can be called via AJAX and will return the queried records, in this case the method is going to query the database and return all the records that match with the title attribute

  • private methods - these are the standard private methods for setting the movie and configuring strong parameters

Now let's create the view template files and JS files that we'll need, you can run the command below in the terminal to create them all at once:

touch app/views/movies/_form.html.haml app/views/movies/_list.html.haml app/views/movies/_movie.html.haml app/views/movies/create.js.erb app/views/movies/destroy.js.erb  app/views/movies/edit.js.erb app/views/movies/index.html.haml app/views/movies/index.js.erb app/views/movies/new.js.erb app/views/movies/update.js.erb

Before we continue, a couple things to notice:

  1. We're using HAML, so if you didn't already have the haml gem make sure that you add it to the Gemfile and run bundle install

  2. Instead of the typical ERB files we're using quite a few JavaScript files, this is because our app is going to be making AJAX calls and we're simply going to render the JS responses instead of view templates.

Now let's go through the files and add the code needed one by one, as you're adding the code make sure to observe the indentation (indentation is requried by HAML):

_form.html.haml

%td
  = form_for object, url: object.new_record? ? movies_path : movie_path(object), remote: true, html: {enctype: 'multipart/form-data', class: ""} do |f|

    %div
      = f.label :title, "Title #{object.errors['title'][0]}"
      = f.text_field :title
      %br

    %div
      = f.label :director, "Director #{object.errors['director'][0]}"
      = f.text_field :director
      %br

    %hr

    = f.submit "Save"
    %button.cancel Cancel

Since this partial will be included in a table it nests the form inside of a td tag and uses the form_for form builder helper. It also uses a ternary operator (object.new_record? ? movies_path : movie_path(object)) to figure out if the form is for a new record or updating a pre-existing record since this form partial will be used for both actions.

From there it's exactly like a typical form partial, you are free to style the form in any way you want it to look, I also included the ability to show the validation errors next to the label tag.

_list.html.haml

%div.media
  %div.media-body
    %table.table
      %thead.index-haml-pages
        %th.col-md-2= smart_listing.sortable "Title", "title"
        %th.col-md-2= smart_listing.sortable "Director", "director"
      %tbody
        - smart_listing.collection.each do |o|
          %tr.editable{:data => {:id => o.id}}
            = smart_listing.render object: o, partial: "movies/movie", locals: {object: o}

        %div.smart-listing-new-button
          = smart_listing.item_new colspan: 12, link: new_movie_path

    = smart_listing.paginate
    = smart_listing.pagination_per_page_links

The list partial is what the index view is going to call, this includes the table head and body setup, along with the pagination links. Notice that it also includes the editable method which calls the actions allowing for editing each of the records.

_movie.html.haml

%td.table-text= object.title
%td.table-text= object.director
%td.actions= smart_listing_item_actions [{name: :edit, url: edit_movie_path(object)}, {name: :destroy, url: movie_path(object), confirmation: "Are you sure you want to delete this movie?"}]

With view code that's a little more complex it's smart to abstract as much of it as possible into view partials, the movie partial holds the view template code for each record. The object call is essentially calling the equivalent of self, so we can call the title and director attribute on it and it will render each of the records out in the table. The third td tag holds the edit and delete actions.

create.js.erb

<%= smart_listing_item :movies, :create, @movie, @movie.valid? ? "movies/movie" : "movies/form" %>

In the create JS file we simply have one line of code that calls the smart_listing_item view helper method and passes in the create method if the item is valid and renders the form partial again if it's not.

destroy.js.erb

<%= smart_listing_item :movies, :destroy, @movie %>

Similiar to the setup for the create JS file, the destroy JS file only needs one line of code and calls the destroy method on it.

edit.js.erb

<%= smart_listing_item :movies, :edit, @movie, "movies/form" %>

With the edit JS file, it responds to the AJAX request and renders the form.

index.html.haml

%div
  %h1= "Movie Directory"
  = smart_listing_controls_for(:movies, {class: ""}) do
    %div.filter.form-search
      .input-group
        = text_field_tag :filter, '', class: "", placeholder: "Search...", autocomplete: :off, value: "#{params[:filter]}"
        %span.input-group-btn
          = submit_tag 'Refresh', :class => "btn btn-default", :name => "do_query"
          = link_to  "Clear search", '/movies', class: "btn btn-default"

  %script(type="text/javascript")
    - if params[:f]
      - params[:f].each do |field, param|
        - param.each do |option, value|
          - if value[:o] != "skip"
            - if value[:v].is_a? Hash
              value = ["#{value[:v]['0']}", "#{value[:v]['1']}", "#{value[:v]['2']}"]
            - else
              value = "#{value[:v]}"
              $.filters.append(humanize("#{field}"), "#{field}", "#{Movie.columns_hash[field].type}", value, "#{value['o']}", "", "#{option}");

  = smart_listing_render :movies

The main index view template contains the search form field and some custom JavaScript code, I've included it in this file, however in a production application you'd want to pull this into its own JS file in the asset pipeline, however I think it's helpful to see it all on one page while you're learning how it works.

After the search form you will see that the smart_listing_render :movies call is bringing in the actual table and data.

index.js.erb

<%= smart_listing_update(:movies) %>

Since the list of movies will need to be returned via an AJAX request when a new query is made, this index JS file will handle that process for the application.

new.js.erb

<%= smart_listing_item :movies, :new, @movie, "movies/form" %>

The new JS file renders the new form partial.

update.js.erb

<%= smart_listing_item :movies, :update, @movie, @movie.valid? ? "movies/movie" : "movies/form" %>

The update JS file manages the process for updating the attributes of the record, along with checking to see if the changes are valid. As you probably notice it has a very similar setup to the create JS file.

Now that we have all of the code setup for the views we need to draw a new route for our search action, update the the routes file so it looks like this:

Rails.application.routes.draw do
  get '/movies/search', to: 'movies#search'
  resources :movies
end

Ok, we're getting close to having the system built out! When going through the controller, you may have noticed a weird looking method: all_with_filter. One roadblock I ran into the past was with the query, so I added an ActiveRecord extension, let's add that to the config/initializers/ directory:

# config/initializers/active_record_extension.rb

module ActiveRecordExtension

  extend ActiveSupport::Concern

  module ClassMethods

    class StatementBuilder

      def initialize(column, type, value, operator)
        @column = column
        @type = type
        @value = value
        @operator = operator
      end

      def to_statement

        return if [@operator, @value].any? { |v| v == '_discard' }

        unary_operators[@operator] || unary_operators[@value] ||
          build_statement_for_type_generic
      end


      protected

      def unary_operators
        {
          '_blank' => ["(#{@column} IS NULL OR #{@column} = '')"],
          '_present' => ["(#{@column} IS NOT NULL AND #{@column} != '')"],
          '_null' => ["(#{@column} IS NULL)"],
          '_not_null' => ["(#{@column} IS NOT NULL)"],
          '_empty' => ["(#{@column} = '')"],
          '_not_empty' => ["(#{@column} != '')"],
        }
      end

      private

      def range_filter(min, max)
        if min && max
          ["(#{@column} BETWEEN ? AND ?)", min, max]
        elsif min
          ["(#{@column} >= ?)", min]
        elsif max
          ["(#{@column} <= ?)", max]
        end
      end

      def build_statement_for_type
        case @type
        when :boolean                   then build_statement_for_boolean
        when :integer, :decimal, :float then build_statement_for_integer_decimal_or_float
        when :string, :text             then build_statement_for_string_or_text
        when :enum                      then build_statement_for_enum
        when :belongs_to_association    then build_statement_for_belongs_to_association
        end
      end

      def build_statement_for_boolean
        return ["(#{@column} IS NULL OR #{@column} = ?)", false] if %w(false f 0).include?(@value)
        return ["(#{@column} = ?)", true] if %w(true t 1).include?(@value)
      end

      def column_for_value(value)
        ["(#{@column} = ?)", value]
      end

      def build_statement_for_belongs_to_association
        return if @value.blank?
        ["(#{@column} = ?)", @value.to_i] if @value.to_i.to_s == @value
      end

      def build_statement_for_string_or_text
        return if @value.blank?
        @value = begin
          case @operator
          when 'default', 'like'
            "%#{@value.downcase}%"
          when 'starts_with'
            "#{@value.downcase}%"
          when 'ends_with'
            "%#{@value.downcase}"
          when 'is', '='
            "#{@value.downcase}"
          else
            return
          end
        end
        ["(LOWER(#{@column}) #{like_operator} ?)", @value]
      end

      def build_statement_for_enum
        return if @value.blank?
        ["(#{@column} IN (?))", Array.wrap(@value)]
      end

      def ar_adapter
        ::ActiveRecord::Base.connection.adapter_name.downcase
      end

      def like_operator
        ar_adapter == 'postgresql' ? 'ILIKE' : 'LIKE'
      end


      #  from parrent
      def get_filtering_duration
        FilteringDuration.new(@operator, @value).get_duration
      end

      def build_statement_for_type_generic

        build_statement_for_type || begin
          case @type
          when :date
            build_statement_for_date
          when :datetime, :timestamp
            build_statement_for_datetime_or_timestamp
          end
        end
      end

      def build_statement_for_integer_decimal_or_float
        case @value
        when Array, Hash then
          p @value
          normal_value = []
          @value.each_pair { |key, value| normal_value.push(value) }
          @value = normal_value
          p @value
          val, range_begin, range_end = *@value.collect do |v|
            next unless v.to_i.to_s == v || v.to_f.to_s == v
            @type == :integer ? v.to_i : v.to_f
          end
          case @operator
          when 'between'
            range_filter(range_begin, range_end)
          else
            column_for_value(val) if val
          end
        else
          if @value.to_i.to_s == @value || @value.to_f.to_s == @value
            @type == :integer ? column_for_value(@value.to_i) : column_for_value(@value.to_f)
          end
        end
      end

      def build_statement_for_date
        range_filter(*get_filtering_duration)
      end

      def build_statement_for_datetime_or_timestamp
        start_date, end_date = get_filtering_duration
        start_date = start_date.to_time.beginning_of_day if start_date
        end_date = end_date.to_time.end_of_day if end_date
        range_filter(start_date, end_date)
      end


      class FilteringDuration
        def initialize(operator, value)
          @value = value
          @operator = operator
        end

        def get_duration
          case @operator
          when 'between'   then between
          when 'today'     then today
          when 'yesterday' then yesterday
          when 'this_week' then this_week
          when 'last_week' then last_week
          else default
          end
        end

        def today
          [Date.today, Date.today]
        end

        def yesterday
          [Date.yesterday, Date.yesterday]
        end

        def this_week
          [Date.today.beginning_of_week, Date.today.end_of_week]
        end

        def last_week
          [1.week.ago.to_date.beginning_of_week,
            1.week.ago.to_date.end_of_week]
        end

        def between
          [convert_to_date(@value[1]), convert_to_date(@value[2])]
        end

        def default
          [default_date, default_date]
        end

        private

        def date_format
          I18n.t('admin.misc.filter_date_format',
            default: I18n.t('admin.misc.filter_date_format', locale: :en)).gsub('dd', '%d').gsub('mm', '%m').gsub('yy', '%Y')
        end

        def convert_to_date(value)
          value.present? && Date.strptime(value, date_format)
        end

        def default_date
          default_date_value = Array.wrap(@value).first
          convert_to_date(default_date_value) rescue false
        end
      end

    end


    class WhereBuilder
      def initialize(scope)
        @statements = []
        @values = []
        @tables = []
        @scope = scope
      end

      def add(field, value, operator)
        if !field.nil?
          field[:searchable_columns].each do |column_infos|
            statement, value1, value2 = StatementBuilder.new(column_infos[:column], column_infos[:type], value, operator).to_statement
            @statements << statement if statement.present?
            @values << value1 unless value1.nil?
            @values << value2 unless value2.nil?
            table, column = column_infos[:column].split('.')
            @tables.push(table) if column
          end
        end
      end

      def build
        scope = @scope.where(@statements.join(' OR '), *@values)
        scope = scope.references(*(@tables.uniq)) if @tables.any?
        scope
      end
    end

    def query_scope(scope, query, fields = config.list.fields.select(&:queryable?))
      wb = WhereBuilder.new(scope)
      fields.each do |field|
        wb.add(field, query, 'default')
      end
      # OR all query statements
      wb.build
    end

    # filters example => {"string_field"=>{"0055"=>{"o"=>"like", "v"=>"test_value"}}, ...}
    # "0055" is the filter index, no use here. o is the operator, v the value
    def filter_scope(scope, filters, fields = config.list.fields.select(&:filterable?))
      filters.each_pair do |field_name, filters_dump|
        filters_dump.each do |_, filter_dump|
          if filter_dump[:o] != 'skip'
            wb = WhereBuilder.new(scope)
            wb.add(fields.detect { |f| f[:name].to_s == field_name }, filter_dump[:v], (filter_dump[:o] || 'default'))
            # AND current filter statements to other filter statements
            scope = wb.build
          end
        end
      end
      scope
    end

    def build_statement(column, type, value, operator)
      StatementBuilder.new(column, type, value, operator).to_statement
    end

    def all_with_filter(options = {}, scope = nil)

      fields = model_fields
      scope = query_scope(scope, options[:query], fields) if options[:query]
      scope = filter_scope(scope, options[:filters], fields) if options[:filters]

      scope
    end

    def model_fields
      columns = []
      self.columns.each { |column|  
        columns.push({
            :name => column.name,
            :searchable_columns => [
              {
                :column => column.name,
                :type => column.type,
              }
            ]
          })
      }
      columns
    end



  end
end

# include the extension 
ActiveRecord::Base.send(:include, ActiveRecordExtension)

Yeahhhh, that was a lot of code, the query functionality of the gem took quite a while to work for my implementation and required some metaprogramming. Don't worry about the details of it, just know that there are quite a few models here that patch the way that the queries interact with ActiveRecord.

If you run the application and navigate to localhost:3000/movies you'll see that everything is working perfectly!! Very nice work. You can create new records, sort by column names, paginate and search. And also notice that none of the actions require a page refresh, they're all AJAX based. The only feature it seems to be missing is the ability to edit and delete records in place. And those features are actually already there, we simply need to have the icons show up.

I want to walk through how to customize the icons, so let's add the font awesome gem into the Gemfile:

gem 'font-awesome-rails', '~> 4.5'

After running bundle install add the CSS call, like below:

/* app/assets/application.css */

*= require font-awesome

You now have access to the font awesome icon classes. Opening up the smart_listing initializer, let's un-comment and swap out the glyphicon classes with the below calls:

:icon_new              => "fa fa-plus",
:icon_edit             => "fa fa-pencil",
:icon_trash            => "fa fa-times",
:icon_sort_none        => "fa fa-sort",
:icon_sort_up          => "fa fa-chevron-up",
:icon_sort_down        => "fa fa-chevron-down",

Restarting the rails server you will see that the icons are showing up and everything is working properly. Notice that in addition to the edit and delete icons we also have dynamic sorting up and down arrows, which is a nice bonus.

This was a pretty long review and smart_listing example, however I wanted to share how I got it working since this one took quite a while to get working properly, I hope that you have fun with it!

Code

Resources