Crazyton - Better than module_function

ruby open-source

I created a new gem that gives you the nice syntax of module_functions but with the advantages of classes.

An example:

class ApiStub
  include Crazyton

  def sign_up(email, password, status = 200, response = default_sign_up_request)
    stub_request(:post, "https://some-service.com")
      .with(body: { email: email, password: password })
      .to_return(status: status,  body: response.to_json)
  end

  private

  def default_sign_up_request
    {
      event: :signed_up,
      user_id: 123
    }
  end
end

# Invoke

ApiStub.sign_up("demo@example.com", "password123")

You can get it here: GitHub - abuisman/crazyton

Why?

Modules with module_function and are a pretty good way to define helpers and make them available in a nice way:

module AppSettings
	module_function

	def google_analytics_enabled?
	  !!ENV.fetch('GOOGLE_ANALYTICS', false)
	end
end

## Usage:

AppSettings.google_analytics_enabled?

But they aren’t great once things get more complex.

Let’s say that I am creating a few interfaces to some websites. Let’s call them crawlers.

This is fine:

require 'uri'
require 'net/http'
require 'openssl'
require 'nokogiri'

module AbuismanCom
  module_function

  def latest_post
    get_selector_text("https://abuisman.com/", ".posts ul li:first-of-type")
  end

  def get_selector_text(url, selector)
    url = URI(url)

    http = Net::HTTP.new(url.host, url.port)
    http.use_ssl = true
    http.verify_mode = OpenSSL::SSL::VERIFY_NONE

    request = Net::HTTP::Get.new(url)

    response = http.request(request)
    response.read_body
    document = Nokogiri::HTML.parse(response.read_body)
    document.css(selector).text.split("\n").map(&:strip).join(" ")
  end
end

puts AbuismanCom.latest_post
=> Diffing Rails credentials in Rails 6.0 10/05

But when I add another site, I recognise that I want to reuse some of the code and things are going in the wrong direction:

require 'uri'
require 'net/http'
require 'openssl'
require 'nokogiri'

module AbuismanCom
  module_function

  def latest_post
    HttpHelper.get_selector_text("https://abuisman.com/", ".posts ul li:first-of-type")
  end
end

module HackerNews
  module_function

  def top_post
    HttpHelper.get_selector_text("https://news.ycombinator.com/", "#hnmain .itemlist tr:first-of-type")
  end
end

module HttpHelper
  module_function

  def get_selector_text(url, selector)
    url = URI(url)

    http = Net::HTTP.new(url.host, url.port)
    http.use_ssl = true
    http.verify_mode = OpenSSL::SSL::VERIFY_NONE

    request = Net::HTTP::Get.new(url)

    response = http.request(request)
    response.read_body
    document = Nokogiri::HTML.parse(response.read_body)
    document.css(selector).text.split("\n").map(&:strip).join(" ")
  end
end

puts AbuismanCom.latest_post
=> Diffing Rails credentials in Rails 6.0 10/05
puts HackerNews.top_post
=>  1.      Tokyo police lose 2 floppy disks containing personal info on citizens (mainichi.jp)

I don’t like the call to HttpHelper.get_selector_text. You can fix that by using extend(HttpHelper) on the other two modules, so that is also ok. But I don’t want to do that for every website module that I create.

I will want to add more helpers to the crawler and a caching feature and I don’t want to keep repeating those things in a list of extend calls. I could get a long way with something like extend CrawlerStuff which in turn extends those things, but then I’d just be avoiding inheritance for the sake of avoiding it or for that sweet sweet HackerNews.top_post syntax.

Insert include Crazyton

Instead, I can use inheritance from a base class and define whatever I need there:

require 'uri'
require 'net/http'
require 'openssl'
require 'nokogiri'
require 'crazyton'

class BaseCrawler
  include Crazyton

  def get_selector_text(url, selector)
    url = URI(url)

    http = Net::HTTP.new(url.host, url.port)
    http.use_ssl = true
    http.verify_mode = OpenSSL::SSL::VERIFY_NONE

    request = Net::HTTP::Get.new(url)

    response = http.request(request)
    response.read_body
    document = Nokogiri::HTML.parse(response.read_body)
    document.css(selector).text.split("\n").map(&:strip).join(" ")
  end
end


class AbuismanCom < BaseCrawler
  def latest_post
    get_selector_text("https://abuisman.com/", ".posts ul li:first-of-type")
  end
end

class HackerNews < BaseCrawler
  def top_post
    get_selector_text("https://news.ycombinator.com/", "#hnmain .itemlist tr:first-of-type")
  end
end

Without all the module_function and extend stuff the crawlers look a lot cleaner. The < BaseCrawler is very basic ruby stuff so I don’t need to explain to others in the team why you’d have to do some unfamiliar module setup. Even the include Crazyton happens only once inside of the BaseCrawler.

What IS strange then is that you would have to explain to the others why top_post is suddenly callable like a class method instead of it being an instance method. In the end though I like how little bootstrapping code there is in the classes.

Because we are using classes and inheritance I could also give the BaseCrawler and interface:

require 'uri'
require 'net/http'
require 'openssl'
require 'nokogiri'
require 'crazyton'

class BaseCrawler
  include Crazyton

  def get_selector_text(selector)
    url = URI(base_url)

    http = Net::HTTP.new(url.host, url.port)
    http.use_ssl = true
    http.verify_mode = OpenSSL::SSL::VERIFY_NONE

    request = Net::HTTP::Get.new(url)

    response = http.request(request)
    response.read_body
    document = Nokogiri::HTML.parse(response.read_body)
    document.css(selector).text.split("\n").map(&:strip).join(" ")
  end

	private

  def base_url
    raise "You have to implement base_url"
  end
end


class AbuismanCom < BaseCrawler
  def latest_post
    get_selector_text(".posts ul li:first-of-type")
  end

  private

  def base_url
    "https://abuisman.com/"
  end
end

class HackerNews < BaseCrawler
  def top_post
    get_selector_text("#hnmain .itemlist tr:first-of-type")
  end

  private

  def base_url
    "https://news.ycombinator.com/"
  end
end

Ah yes, did I mention private methods? Because modules don’t have those, but classes do.

In summary, with Crazyton you get:

Have fun!