Clojure from the ground up: logistics

Beyond the REPL

The REPL is an excellent tool for exploration, but it has limits as a development environment. Editing multi-line expressions is awkward, and any work done there disappears on restart unless it is retyped. To build anything substantial, we need to write programs durably, so they can be read, shared, and evaluated later.

A real program also carries more than just code. Tests verify behavior, resources provide supplementary data, and documentation explains usage. Clojure programs frequently rely on external libraries, and all of this needs to be assembled in a predictable structure. In Clojure, that structure is a project, managed by Leiningen.

Anatomy of a Leiningen project

A project created with lein new produces a directory that holds the entire application.

$ cd scratch; ls
doc  project.clj  README.md  resources  src  target  test

The core of this structure is project.clj. This file declares the project’s name, version, and dependencies. A -SNAPSHOT suffix on a version indicates a development build; these are mutable and can be updated freely. In contrast, a version without that suffix is meant to be immutable once published, allowing other projects to declare a specific, known dependency.

(defproject scratch "0.1.0-SNAPSHOT"
  :description "FIXME: write description"
  :url "http://example.com/FIXME"
  :license {:name "Eclipse Public License"
            :url "http://www.eclipse.org/legal/epl-v10.html"}
  :dependencies [[org.clojure/clojure "1.5.1"] ])

Here, the scratch project specifies it depends on org.clojure/clojure version 1.5.1. The other directories in the project follow a standard convention: src is for Clojure source, test is for corresponding test files, resources holds ancillary files like images, and doc is for documentation. The README.md is a stub—a basic outline left by Leiningen to be filled in with real documentation, copyright, and a license.

Organizing with namespaces

A new project includes a stub namespace at src/scratch/core.clj, which begins with an ns declaration. This macro associates the following code with a logical scope, such as scratch.core, preventing name collisions as the codebase grows. Just as clojure.core holds the standard library, you might split your project into separate namespaces for specific domains, like scratch.io or scratch.string.

(ns scratch.core)

(defn foo
  "I don't do a whole lot."
  [x]
  (println x "Hello, World!"))

In Clojure, def and defn always define things within a specific namespace. A function named foo in scratch.core is distinct from foo in scratch.pad. You can always refer to these by their fully qualified names, which are the namespace name, a slash, and the symbol.

scratch.foo=> (ns scratch.core)
nil
scratch.core=> (def foo "I'm in core")
#'scratch.core/foo
scratch.core=> (ns scratch.pad)
nil
scratch.pad=> (def foo "I'm in pad!")
#'scratch.pad/foo

By default, new namespaces automatically refer to clojure.core, providing access to standard functions like filter and let. However, they don’t automatically see definitions from other project namespaces. To use a function from elsewhere, you must explicitly :require the dependency. For example, a user namespace might need to access scratch.core/foo.

user=> (ns user (:require [scratch.core]))
nil
user=> scratch.core/foo
"I'm in core"

Since fully qualified names are verbose, you can provide an alias with :as for convenience, or bring specific vars into the current namespace with :refer. These directives keep inter-namespace relationships explicit, making the program’s structure clearer without tying names to data or state.

user=> (ns user (:require [scratch.core :as c]))
nil
user=> c/foo
"I'm in core"

This organization contrasts with object-oriented languages, where logic is often grouped into small classes that encapsulate state. Clojure namespaces are typically larger and purely functional in their organizing logic. This difference is a natural consequence of how state is handled: functional code can be freely grouped by topic or domain, while OO classes combine naming and state

into a single unit. Passing hundreds of functions in a single namespace is normal here, whereas an object with hundreds of methods is generally considered a design flaw.

Writing and running tests

For small, isolated experiments, the REPL is a fine testing tool. But as soon as functions begin to depend on each other, manual testing breaks down. A change to a core function can invalidate thousands of behaviors that would normally need re-checking by hand. Automating these checks by turning the tests themselves into a program—the test suite—lets the system verify itself quickly and repeatedly.

To illustrate, let’s add an exponentiation function to src/scratch/core.clj and write tests against it.

(ns scratch.core)

(defn pow
  "Raises base to the given power. For instance, (pow 3 2) returns three squared, or nine."
  [base power]
  (apply * (repeat base power)))

Running lein test first shows the built-in test stub failing. Distinguishing between failures (wrong expected values) and errors (exceptions during a test) helps clarify what to fix. The output points to the exact test name and file location, but checking the test source shows the root issue.

(ns scratch.core-test
  (:require [clojure.test :refer :all]
            [scratch.core :refer :all]))

(deftest a-test
  (testing "FIXME, I fail."
    (is (= 0 1))))

The test namespace references clojure.test for its deftest and testing macros, and the src/scratch/core namespace it needs to validate. The testing blocks within a deftest help name the specific scenario being checked.

After fixing the boilerplate test to assert that 0 equals 0, we can write proper tests for the pow function’s logic.

(deftest pow-test
  (testing "unity"
    (is (= 1 (pow 1 1)))))

Starting with the trivial case of 1^1, we verify the output is correct. As the tests get more complex, they expose problems in the implementation itself. A test for (pow 3 2) reveals a bug in the original sourcing of arguments to a helper function.

user=> (repeat 3 2)
(2 2 2)
user=> (* 2 2 2)
8

While Clojure’s repeat function generates a lazy sequence when given one argument, we need to specify both the count and the item.

(defn pow
  "Raises base to the given power. For instance, (pow 3 2) returns three
  squared, or nine."
  [base power]
  (apply * (repeat power base)))

Edge cases are where the value of careful design shows through. Testing 0^0 passes without writing a special case. This works due to a combination of (repeat 0 1) producing an empty sequence and the * function returning the multiplicative identity. When called with an empty argument list, the convention is to return 1, which keeps the mathematics consistent.

user=> (*)
1

Laying out tests as Clojure code in dedicated namespaces turns lein test into a fast feedback loop. Optimized for automatic verification, these tests can be run as frequently as you like, making refactoring a safe, mechanical process rather than a risky, manual one.

Putting libraries to work

Modern Clojure projects are built to interoperate, allowing developers to pull in libraries for parsing, math, graphics, and more. To see how this works in practice, consider a public-health question: where should an ad campaign against drunk driving be targeted?

The FBI’s Uniform Crime Reporting database tracks annual arrests by county, but the raw files are unwieldy. Matt Aliabadi has normalized the UCR data into JSON, available on Github. After downloading the 2008 dataset, a quick peek with head shows the format:

aphyr@waterhouse:~/scratch$ head 2008.json
[
  {
    "icpsr_study_number": null,
    "icpsr_edition_number": 1,
    "icpsr_part_number": 1,
    "icpsr_sequential_case_id_number": 1,
    "fips_state_code": "01",
    "fips_county_code": "001",
    "county_population": 52417,
    "number_of_agencies_in_county": 3,

JSON closely resembles Clojure data structures — here we see a vector of maps with string keys and null values. To make it usable, we parse it with a library like Cheshire, published on the Clojars repository. Adding it to a project is just a matter of editing project.clj:

(defproject scratch "0.1.0-SNAPSHOT"
  :description "Just playing around"
  :url "http://example.com/FIXME"
  :license {:name "Eclipse Public License"
            :url "http://www.eclipse.org/legal/epl-v10.html"}
  :dependencies [[org.clojure/clojure "1.5.1"]
                 [cheshire "5.3.1"]])

After restarting the REPL with lein repl, Leiningen downloads Cheshire automatically. Cheshire’s README shows the key function:

;; parse some json and get keywords back
(parse-string "{\"foo\":\"bar\"}" true)
;; => {:foo "bar"}

Combining slurp with parse-string loads the entire dataset:

user=> (use 'cheshire.core)
nil
user=> (parse-string (slurp "2008.json"))
...

That’s a substantial amount of data. Drilling into a single entry:

user=> (first (parse-string (slurp "2008.json")))
{"syntheticdrug_salemanufacture" 1, "all_other_offenses_except_traffic" 900, "arson" 3, ...}
user=> (-> "2008.json" slurp parse-string first)

String keys are awkward to work with, so the second argument to parse-string converts them to keywords:

user=> (first (parse-string (slurp "2008.json") true))
{:other_assaults 288, :gambling_all_other 0, :arson 3, ... :drunkenness 108}

Binding the result to a variable makes repeated exploration easier:

user=> (def data (parse-string (slurp "2008.json") true))
#'user/data

Mapping each county to its :driving_under_influence field extracts the relevant data:

user=> (->> data (map :driving_under_influence))
(198 1095 114 98 135 4 122 587 204 53 177 ...

The maximum reported count is striking:

user=> (->> data (map :driving_under_influence) (apply max))
45056

Sorting the list and taking the tail reveals the worst offenders:

user=> (->> data (map :driving_under_influence) sort (take-last 10))
(8589 10432 10443 10814 11439 13983 17572 18562 26235 45056)

The frequencies function builds a histogram of DUI report counts:

user=> (->> data (map :driving_under_influence) frequencies)
{0 227, 1024 1, 45056 1, 32 15, 2080 1, 64 12 ...

Sorting those key-value pairs by key with sort-by shows the distribution:

user=> (->> data (map :driving_under_influence) frequencies (sort-by key) pprint)
([0 227]
 [1 24]
 [2 17]
 [3 20]
 [4 17]
 [5 24]
 [6 23]
 [7 23]
 [8 17]
 [9 19]
 [10 29]
 [11 20]
 [12 18]
 [13 21]
 [14 25]
 [15 13]
 [16 18]
 [17 16]
 [18 17]
 [19 11]
 [20 8]
 ...

Most counties report zero DUIs, with a long tail trailing off — a shape characteristic of exponential distributions. The top 10 counties sit at the extreme end:

user=> (->> data (map :driving_under_influence) frequencies (sort-by key) (take-last 10) pprint)
([8589 1]
 [10432 1]
 [10443 1]
 [10814 1]
 [11439 1]
 [13983 1]
 [17572 1]
 [18562 1]
 [26235 1]
 [45056 1])

user=> (->> data (sort-by :driving_under_influence) (take-last 10) pprint)
({:other_assaults 3096,
  :gambling_all_other 3,
  :arson 106,
  :have_stolen_property 698,
  :syntheticdrug_salemanufacture 0,
  :icpsr_sequential_case_id_number 220,
  :drug_abuse_salemanufacture 1761,
  ...

Reading the full maps is tedious, so mapcat collects all keys into a sorted-set to see which fields are available:

user=> (->> data (sort-by :driving_under_influence) (take-last 10) (mapcat keys) (into (sorted-set)) pprint)
#{:aggravated_assaults :all_other_offenses_except_traffic :arson
  :auto_thefts :bookmaking_horsesport :burglary :county_population
  :coverage_indicator :curfew_loitering_laws :disorderly_conduct
  :driving_under_influence :drug_abuse_salemanufacture
  :drug_abuse_violationstotal :drug_possession_other
  :drug_possession_subtotal :drunkenness :embezzlement
  :fips_county_code :fips_state_code :forgerycounterfeiting :fraud
  :gambling_all_other :gambling_total :grand_total
  :have_stolen_property :icpsr_edition_number :icpsr_part_number
  :icpsr_sequential_case_id_number :icpsr_study_number :larceny
  :liquor_law_violations :marijuana_possession
  :marijuanasalemanufacture :multicounty_jurisdiction_flag :murder
  :number_of_agencies_in_county :numbers_lottery
  :offenses_against_family_child :opiumcocaine_possession
  :opiumcocainesalemanufacture :other_assaults :otherdang_nonnarcotics
  :part_1_total :property_crimes :prostitutioncomm_vice :rape :robbery
  :runaways :sex_offenses :suspicion :synthetic_narcoticspossession
  :syntheticdrug_salemanufacture :vagrancy :vandalism :violent_crimes
  :weapons_violations}

The :fips_county_code and :fips_state_code fields look useful for identifying locations. select-keys trims the dataset:

user=> (->> data (sort-by :driving_under_influence) (take-last 10) (map #(select-keys % [:driving_under_influence :fips_county_code :fips_state_code])) pprint)
({:fips_state_code "06",
  :fips_county_code "067",
  :driving_under_influence 8589}
 {:fips_state_code "48",
  :fips_county_code "201",
  :driving_under_influence 10432}
 {:fips_state_code "32",
  :fips_county_code "003",
  :driving_under_influence 10443}
 {:fips_state_code "06",
  :fips_county_code "065",
  :driving_under_influence 10814}
 {:fips_state_code "53",
  :fips_county_code "033",
  :driving_under_influence 11439}
 {:fips_state_code "06",
  :fips_county_code "071",
  :driving_under_influence 13983}
 {:fips_state_code "06",
  :fips_county_code "059",
  :driving_under_influence 17572}
 {:fips_state_code "06",
  :fips_county_code "073",
  :driving_under_influence 18562}
 {:fips_state_code "04",
  :fips_county_code "013",
  :driving_under_influence 26235}
 {:fips_state_code "06",
  :fips_county_code "037",
  :driving_under_influence 45056})

The NOAA’s ERDDAP service provides a FIPS code mapping. Saved as fips.json, that file loads the same way:

user=> (def fips (parse-string (slurp "fips.json") true))

The structure is straightforward — a list of code-name pairs:

user=> (keys fips)
(:table)
user=> (keys (:table fips))
(:columnNames :columnTypes :rows)
user=> (->> fips :table :columnNames)
["FIPS" "Name"]

user=> (->> fips :table :rows (take 3) pprint)
(["02000" "AK"]
 ["02013" "AK, Aleutians East"]
 ["02016" "AK, Aleutians West"])

Moving from REPL exploration to a reusable program, a new file src/scratch/crime.clj captures the FIPS parsing logic:

(ns scratch.crime
  (:require [cheshire.core :as json]))

(def fips
  "A map of FIPS codes to their county names."
  (->> (json/parse-string (slurp "fips.json") true)
       :table
       :rows
       (into {})))

The (into {}) call converts the sequence of pairs into a map — into repeatedly applies conj, making it a versatile tool for building collections. Verifying from the REPL:

user=> (use 'scratch.crime :reload)
nil
user=> (fips "10001")
"DE, Kent"

Since maps act as functions, the fips map now looks up county names quickly. Similarly, the UCR loading code becomes a reusable function:

(defn load-json
  "Given a filename, reads a JSON file and returns it, parsed, with keywords."
  [file]
  (json/parse-string (slurp file) true))

(def fips
  "A map of FIPS codes to their county names."
  (->> "fips.json"
       load-json
       :table
       :rows
       (into {})))

(defn most-duis
  "Given a JSON filename of UCR crime data for a particular year, finds the
  counties with the most DUIs."
  [file]
  (->> file
       load-json
       (sort-by :driving_under_influence)
       (take-last 10)
       (map #(select-keys % [:driving_under_influence
                             :fips_county_code
                             :fips_state_code]))))

user=> (use 'scratch.crime :reload) (pprint (most-duis "2008.json"))
nil
({:fips_state_code "06",
  :fips_county_code "067",
  :driving_under_influence 8589}
 {:fips_state_code "48",
  :fips_county_code "201",
  :driving_under_influence 10432}
 {:fips_state_code "32",
  :fips_county_code "003",
  :driving_under_influence 10443}
 {:fips_state_code "06",
  :fips_county_code "065",
  :driving_under_influence 10814}
 {:fips_state_code "53",
  :fips_county_code "033",
  :driving_under_influence 11439}
 {:fips_state_code "06",
  :fips_county_code "071",
  :driving_under_influence 13983}
 {:fips_state_code "06",
  :fips_county_code "059",
  :driving_under_influence 17572}
 {:fips_state_code "06",
  :fips_county_code "073",
  :driving_under_influence 18562}
 {:fips_state_code "04",
  :fips_county_code "013",
  :driving_under_influence 26235}
 {:fips_state_code "06",
  :fips_county_code "037",
  :driving_under_influence 45056})

County identification requires joining state and county FIPS codes into a single string, matching the format in fips:

(defn fips-code
  "Given a county (a map with :fips_state_code and :fips_county_code keys),
   returns the five-digit FIPS code for the county, as a string."
  [county]
  (str (:fips_state_code county) (:fips_county_code county)))

Testing that function in test/scratch/crime_test.clj:

(ns scratch.crime-test
  (:require [clojure.test :refer :all]
            [scratch.crime :refer :all]))

(deftest fips-code-test
  (is (= "12345" (fips-code {:fips_state_code "12" :fips_county_code "345"}))))

aphyr@waterhouse:~/scratch$ lein test scratch.crime-test

lein test scratch.crime-test

Ran 1 tests containing 1 assertions.
0 failures, 0 errors.

With fips-code in hand, the final step constructs a map of county names to DUI reports:

(defn most-duis
  "Given a JSON filename of UCR crime data for a particular year, finds the
  counties with the most DUIs."
  [file]
  (->> file
       load-json
       (sort-by :driving_under_influence)
       (take-last 10)
       (map (fn [county]
              [(fips (fips-code county))
               (:driving_under_influence county)]))
       (into {})))

user=> (use 'scratch.crime :reload) (pprint (most-duis "2008.json"))
nil
{"CA, Orange" 17572,
 "CA, San Bernardino" 13983,
 "CA, Los Angeles" 45056,
 "CA, Riverside" 10814,
 "NV, Clark" 10443,
 "WA, King" 11439,
 "AZ, Maricopa" 26235,
 "CA, San Diego" 18562,
 "TX, Harris" 10432,
 "CA, Sacramento" 8589}

The answer emerges: Los Angeles and Maricopa counties lead in reported drunk driving for 2008. That data reflects reports, not necessarily actual crime rates — enforcement patterns vary by state and can skew the numbers.

(ns scratch.crime
  (:require [cheshire.core :as json]))

(defn load-json
  "Given a filename, reads a JSON file and returns it, parsed, with keywords."
  [file]
  (json/parse-string (slurp file) true))

(def fips
  "A map of FIPS codes to their county names."
  (->> "fips.json"
       load-json
       :table
       :rows
       (into {})))

(defn fips-code
  "Given a county (a map with :fips_state_code and :fips_county_code keys),
  returns the five-digit FIPS code for the county, as a string."
  [county]
  (str (:fips_state_code county) (:fips_county_code county)))

(defn most-duis
  "Given a JSON filename of UCR crime data for a particular year, finds the
  counties with the most DUIs."
  [file]
  (->> file
       load-json
       (sort-by :driving_under_influence)
       (take-last 10)
       (map (fn [county]
              [(fips (fips-code county))
               (:driving_under_influence county)]))
       (into {})))

What was covered

This chapter moved beyond transient REPL scripts into structured projects combining static resources, code, and tests. The namespace system isolates code into distinct chunks, with require and use pulling definitions from elsewhere. Writing tests and running them with lein test some-namespace verifies behavior while moving fluidly between the REPL and files on disk. Cheshire, pulled from Clojars, handled JSON parsing of real-world data, tying together the core grammar, data structures, and sequence functions to answer a practical question.

Exercises

1. Prevalence vs. raw counts: most-duis ignores population size. Divide :driving_under_influence by :county_population and find the top ten counties by that ratio. Decide how to handle zero-population counties.
2. Comparing measures: Return vectors of [county-name, prevalence, report-count, population]. Examine how the prevalence ranking diverges from the raw count ranking — would a campaign target different counties under each metric?
3. Generalization: Write most-prevalent, a function taking a file and a field like :arson, returning the counties where that crime is most reported per capita.
4. Verification: Add a test confirming that most-prevalent behaves correctly.

Next: modeling.