Demystifying Sprockets

A.K.A. The Rails asset pipeline.

Forewords

How browsers used to work

How modern browsers works

But it's no silver bullet

HTTP Caching

The Last-Modified header

GET /ajax/libs/jquery/1.11.1/jquery.min.js HTTP/1.1
Host: ajax.googleapis.com
Accept: text/javascript

HTTP/1.1 200 OK
Content-Type: text/javascript; charset=UTF-8
Last-Modified: Tue, 13 May 2014 02:26:31 GMT
Date: Thu, 10 Jul 2014 08:51:40 GMT

<94 KB of JS>

GET /ajax/libs/jquery/1.11.1/jquery.min.js HTTP/1.1
Host: ajax.googleapis.com
Accept: text/javascript
If-Modified-Since: Tue, 13 May 2014 02:26:31 GMT

HTTP/1.1 304 Not Modified

<JS is not sent again>

HTTP Caching

But can do better.

HTTP Caching

The Expires / Cache-Control headers

GET /ajax/libs/jquery/1.11.1/jquery.min.js HTTP/1.1
Host: ajax.googleapis.com
Accept: text/javascript

HTTP/1.1 200 OK
Content-Type: text/javascript; charset=UTF-8
Cache-Control: max-age=31536000, public
Date: Thu, 10 Jul 2014 08:51:40 GMT

<94 KB of JS>

No other request will be issued for a year

Any Question at this point?

HTTP Caching

But how can I update my assets then?

Rails <= 3.0

Append the file mtime as query string

<script src="/javascripts/application.js?1405197924" ...

Drawbacks

Some proxies (squid) used to ignore the query string
Your deploy strategy need to preserve file properties
Possible race condition if you have multiple servers
Files are downloaded independently

Sprockets

Include the file MD5 in the file name

<script src="/assets/application-d3b07384d113edec49eaa6238ad5ff00.js"

Advantages

Proxies cannot serve stale assets
Hotlinking is not possible
No more race conditions, assets are guaranteed to match your HTML
Files are concatened together

Sprockets - The Directive Processor

//= require jquery
//= require_tree myapp
//= require_self
//= depend_on config
//= stub prototype

Required file are included only once
All required files are part of the MD5 sum
depend_on do not include the file, but use it to compute the MD5 sum
stub can be useful to prevent one of your dependency from requiring something
include (deprecated) like require without the unicity

Sprockets - The Engine API

# config.js.coffee.erb
@Config = 
  truth: <%= 21 * 2 %>

Engines apply transformations on assets to convert them to another format based on extensions

Sprockets - The Engine API

class Csv2Json < Tilt::Template

  def evaluate(scope, locals, &block)
    JSON.dump(CSV.parse(data))
  end

end

Sprockets.register_engine '.csv', Csv2Json

# foo.json.csv
Id,Name
1,George Abitbol

// foo-6d04991772b0ab12f0a96971ad290d68.json
[["Id","Name"],["1","George Abitbol"]]

Engines are just tilt templates
You can chain them as much as you want
See them as compilers

Sprockets - Processors

Processors are Tilt templates like engines, but they do not convert the asset to another type

Sprockets.register_preprocessor 'text/javascript', RemoveBreakPoints
Sprockets.register_postprocessor 'text/javascript', PrependSafetySemiColon
Sprockets.register_bundle_processor 'text/javascript', MinifyJS

Note that processors are registered by MIME type while engines are registered by extensions.

Preprocessors are ran before Postprocessors and Engines
Postprocessors are ran after Preprocessors and Engines
Bundle Processors are ran on concatenated assets rather than individual files.

Sprockets

Shorthand methods

environment.js_compressor  = :uglify
environment.css_compressor = :scss

Sprockets - Helpers

The sprockets context provide helpers to engines, which can eventually expose them to the transormed assets

/* main.css.erb */
.header { background-image: url(<%= image_path('header-background.png') %>);}

They are mostly available in .erb assets.

// main.css.scss
.header { background-image: image-url(header-background.png); }

But some engines like SCSS provide integrated support

Sprockets - Helpers

Most usefull ones are:

image_path, font_path, etc. Gives you the digested path of another asset.
asset_data_uri gives you the Base64 encoded asset content (for embeding images).

And you can of course add custom ones

Sprockets - Integration

Even if he is often nicknamed "The Rails asset pipeline", Sprockets is actually framework agnostic

It can even easilly be used in non Ruby projects (PHP, Node, Python, etc)

Sprockets - Integration

# lib/sprockets_environment.rb
require 'sprockets'

module MyEnvironment < Sprockets::Environment
  def initialize(production=true)
    root = File.expand_path(File.join(__dir__, '..'))
    super(root + '/public')

    append_path(root + '/app/assets/javascripts')
    append_path(root + '/app/assets/stylesheets')
    append_path(root + '/app/assets/images')

    if production
      self.css_compressor = :scss
      self.css_compressor = :uglifier
    end
  end
end

All the configuration would happen in a subclass of Sprockets::Environment

Sprockets - Integration

# config.ru
require 'sprockets_environment'

map '/assets' do
  run MyEnvironment.new
end

Sprockets::Environment acts as a rack middleware.
It will compile and bundle your assets on the fly when requested.
No need to wonder if the compilation is done or not yet, your browser will wait until sprockets is done.
This is intended for development.

Sprockets - Integration

# Rakefile
require 'rake/sprocketstask'
require 'sprockets_environment'

Rake::SprocketsTask.new do |t|
  t.environment = run MyEnvironment.new
  t.output      = "./public/assets"
  t.assets      = %w( application.js application.css )
end

rake assets:precompile will statically compile every listed bundles.
If the output directory is shared between releases, it will only compile what changed.
A manifest.json file will be generated, containing all the mapping between the source files and the compiled files.

Sprockets - Integration

# in your app helpers
ASSETS_ROOT = 'https://my.cdn.com/assets/'

def sprockets
  @sprockets ||= begin
    env = MyEnvironment.new
    ENV['RACK_ENV'] == 'production' ? env.index : env
  end
end

def script_url(name)
  ASSETS_ROOT + sprockets.find_asset(name + '.js').digested_path
end

script_url('application') # => 'https://my.cdn.com/assets/application-d3b07384d113edec49eaa6238ad5ff00.js'

You will need a bit of glue in your application to always go through sprockets to link assets.
The index is a faster version of the environment that rely only on the manifest.json.
If you use something else than ruby, you'll have to parse the manifest.json yourself.

Common complaints

Dependency management: https://rails-assets.org or bower.
Slow compilation: make sure to share the assets directory between releases.
jQuery plugins with hardcoded paths: just use the public/ directory or fix the few URLs.
Most frontend tools are in JS: Use ExecJS
Sourcemaps: yep... It's comming

Conclusion

Any Question?

Jean Boussier

Shopify

https://github.com/byroot