Arbitrary Cookbook Identifiers

Use Case

The current implementation of cookbook storage in chef server identifies cookbooks with a compound identifier composed of the cookbook's name and version number, where the version number is restricted to the subset of SemVer supported by the dependency resolver subsystem.

This identifier scheme has the following deficiencies:

Arbitrary Cookbook Identifiers Proposal

This proposal addresses the use case by adding an additional set of APIs for cookbook storage that identify cookbooks by arbitrary identifiers. The endpoint is functionally identical the the current /cookbooks endpoint, with the following exceptions:

  { "name": "apache2",
    "identifier": "886757f9ae3cf2520c82b791195c27ecafd93656",
    "version": "1.10.4",
    "url": "https://chef.example.org/organizations/:org/cookbook_artifacts/apache2/886757f9ae3cf2520c82b791195c27ecafd93656",
    "origin_url": "https://supermarket.getchef.com/cookbooks/apache2/versions/1.10.4/download",
    "origin_id": "1.10.4"
  }

How Chef Client Uses Cookbooks with Arbitrary IDs

chef-client will have a new mode of operation where it loads a document that contains a list of cookbook artifact names and identifiers to use for a chef-client run. The format of the document may be more complex, but it will contain enough information to produce a list of cookbook name and identifier pairs, e.g.,

{
  "omnibus": "64b3e64306cff223206348e46af545b19032b170",
  "homebrew": "ab4ad2481e08cbb2c4874fd36a44e76f36ec91f7"
}

Given the above, chef-client would make requests to BASE_URL/cookbook_artifacts/omnibus/64b3e64306cff223206348e46af545b19032b170 and BASE_URL/cookbook_artifacts/homebrew/ab4ad2481e08cbb2c4874fd36a44e76f36ec91f7 to get cookbook objects with links to individual files it can download.

This mode is already partially implemented (using currently available APIs) here: https://github.com/opscode/chef/blob/9d277e5a4505e5e83e9c4eb30328fdc7148f15c6/lib/chef/policy_builder/policyfile.rb

What Happens to the Existing Cookbooks API

The existing cookbooks API remains as-is, for the following purposes:

When both the old and new APIs are used concurrently, cookbooks uploaded to the new end point must, by default, be invisible to the old end point. The new end point allows multiple editions of a cookbook at the same version number to exist simultaneously and provides an implicit guarantee that uploading a cookbook will not interfere with any other active cookbooks.

After a sufficient period of time (one or more major release cycles), we may remove the dependency solver from the chef-server if there is a compelling reason to do so.

Discussion

Any implementation choices in this proposal are open for discussion. The design constraints on the solution are: