Chef Style DevOps Kung Fu

Adam Jacob / CTO of Chef / @adamhjk

Chef Style DevOps Kung fu by Chef Software, Inc. is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License. A full list of incorporated sources is included.

Navigate using the left and right arrows.

Gōngfu (功夫)

功 (gōng) meaning "work", "achievement", or "merit"
夫 (fū) often treated as being a word for "man"

The excellence achieved through long practice in one's skills

Source

Wushu (武术)

Basics
Forms
Application

Devops Kung fu

A deep well of practical philosophy
Broad experience, unique to each individual
Recognizable by the basics and forms
Practically applied to the situations in your life

There is a deep well of practical philosophy
There is a broad base of experience, unique to each individual
We are recognizably similar
There is a way to teach people how to do it
It starts with the basics, and everyone must master them
It moves to Forms - predetermined behaviors that can be combined into continuous sets of actions
It ends in Application - actually doing your Kung fu in the field, using what you have learned
TMTOWTDI is true and honest and important
However - if you don't know how to do it, you could probably skip the 15+ year journey of difficulty
And practice the basics, learn the forms, and apply them in your life

DevOps is reinventing how we run our businesses

Who practices it?

Everyone

The application will be different, but the principles hold
We are not generalists - we are well connected specialists

Chef Style DevOps Kung fu

None of the below endorse this talk, or Chef Style DevOps Kung fu

Mark Burgess	Johnny Cash	John Allspaw	Sammy Hagar
Nathan Haneysmith	Ronnie James Dio	Tom Thomas	Jesse Leach
Christopher Brown	Ben Rockwood	Jesse Robbins	Jez Humble
Barry Crist	Thich Naht Hanh	Mitch Hill	Larry Wall
Paul Edelhertz	Lao Tzu	Soo Choi	Jennifer Dumas
Alex Ethier	Yukihiro Matsumoto	Jeff Hackert	Theo Schlossnagle

This list is not complete, or in order. I ran out of room.

Foundational Document

On Github: http://github.com/chef/devops-kungfu
Join by PR
We work together to develop our principles, forms, and applications
Start your own school by forking, removing the names, and changing the content

This talk is my foundational document of that Kung fu
Once you have watched it, you can join my school
We can develop the basics, our forms, and help on application
Talib Kweli explained to me that artists amplify the movement they see in the world
Marvin Gaye didn't write What's Goin On and
then
the movement happened
The movement happened, and Marvin had to respond to, and amplify it, with his art
That is my intent here - I will use my platform to raise up the voices of those who are doing the work
And we can then leverage it as a platform to help each other
And to further our aims of having our workplace be a better place

Elements of DevOps

Principles	(Universal)
Forms	(Shared)
Application	(Unique)

The principles we aspire to (How you recognize DevOps when you see it)
Wushu calls this the 'basics', but it involves a lot of pushups, and there isn't a lot of pushups in devops
This includes a definition of DevOps
The forms we use - the actual things we practice, that try and re-enforce and bolster our principles
The application - the use of those forms in our actual situation
These three things are the evolutionary arc of your practice - in the beginning, you learn from others
As you grow, you will use the forms others have used to learn about the deeper meaning and philosophy
The more you apply them in practice, the less reliant on the forms you will be - it all just becomes the natural flow
Leading you, and anyone else, to be where those of us with already strong Kung Fu are - uniquely suited to our task
(It's not about whether you are good at it - it's about the soul of our kung fu)

Principles

Universally held beliefs; the foundation of all DevOps styles

DevOps

A cultural and professional movement, focused on how we build and operate high velocity organizations, born from the experiences of its practitioners.

Cultural and professional

Cultural like Hip Hop, Heavy Metal, or Otaku
Professional like an MC, Lead Guitarist, or Animator

DevOps: A cultural and professional movement, focused on how we build and operate high velocity organizations, born from the experiences of its practitioners.

Focused on how we build and operate

Encompasses the details of how we build things
And how we operate them over time

DevOps: A cultural and professional movement, focused on how we build and operate high velocity organizations, born from the experiences of its practitioners.

High velocity organizations

Our experiences were in building high velocity organizations
The same principles applied to low velocity organizations bring instability

DevOps: A cultural and professional movement, focused on how we build and operate high velocity organizations, born from the experiences of its practitioners.

Born from the experiences of its practitioners

Most of whom were web innovators
The first explorers of DevOps

DevOps: A cultural and professional movement, focused on how we build and operate high velocity organizations, born from the experiences of its practitioners.

Design for the safety, contentment, knowledge and freedom of both your peers and your customers.

Safety

Human safety
Information safety
The ability for individuals to act without fear of unintended consequences

Safety is a slider – different systems have different thresholds

Contentment

Contentment is about being satisfied with what you have.

Happiness is not a goal – it’s a by-product of a life well lived
- Eleanor Roosevelt

Knowledge

Access to knowledge is a leading indicator of social progress.

The goal isn’t to minimize needed knowledge – it's to provide access to the wealth of it, when we need it.

The power or right to act, speak, or think as one wants without hindrance or restraint.
– The Internet

We should be empowering ourselves and others to act, speak, and think as they need to with less hindrance.

People

Products

Companies

We are Lean

Eliminate non-value-added action (Waste/Muda)
Pull over Push
Kaizen (Continuous Improvement)
Kaikaku (Disruptive Change)
Small Batch + Experimentation

Ubiquitous Workflow Automation

Diversity

Forms

Shared between many styles; emphasis varies from style to style

Make it public
Shoot for one sentence
Include the people you are trying to help
Include your product
Include the change the people you are helping will see

The most enduring and transformative companies use Chef to become fast, efficient, and innovative software driven organizations

What brings about good outcomes in your environment
Write them down, make them public
Include the principles of DevOps
Mix in the things that are unique to your industry/problem

We build software for our users; we do what is right for them.
(The Right Hard Thing)

Empowered teams

Permission to act
Paired with the context to make good decisions
With leaders who care about your purpose
And share your beliefs

Form diverse bonds

Take people to lunch, or have meetings, with people outside your specialty
Ask them what they do, and try and understand their problems and perspective
Legal, Finance, Sales, Marketing, Business Development, Software Development, Systems Administrators, Security Professionals, Product

Take people to lunch, or have meetings, outside your specialty
Ask them what they do, and try and understand
You need to talk to:
Legal (Ask them what the role of a lawyer is, and how you can help make any interactions you have smooth)
Finance (Ask them what the role of a finance is, what key metrics they watch, and how you can help)
Sales (Ask them what their quota is, what do they struggle with at customers, and how can you help them make their number)
Marketing (Ask them what message is resonating, why, and how they see the market evolving)
Business Development (Ask them what partners are moving the needle, how they think about those arrangements, and whats on the horizon)
Software Development (Ask them what they are building, and why it matters, and what they are excited about in the future)
Systems Administrators (Ask them about availability, what do they think is the biggest limit to your organizational efficiency)
Security Professionals (Ask them how they evaluate your business and technology in terms of risk, how you can help)
Product (Ask them what they think the gaps in your product are, and to talk about the last 5 customers they've met)

Build consensus on important decisions

Use your bonds to circulate plans with stakeholders
Incorporate their criticism and feedback into the plan
Present to the group
Ask for consensus

Strong value propositions

Pain Killers, not vitamins
Make something people ♡
Needs not wants (one customer wants a feature; many customers need a feature)

Make products that have strong value propositions

Product/Pain Relievers/Gain Creators
Map to Customer Jobs, Customer Pains, and Customer Gains
Pain Killers not vitamins
Make something people LOVE
Needs not wants (customer wants a feature, customer(s) need features)
How to identify the opportunities
Call out customer pain through engagement
Interview customers to learn about their problems
Solve the pain you experience yourself
Write down your ideas for products What is it? (product) Who is it for? (target customers) What problem does it solves and why does it matter? (problem & pain) How does it work? (pain relievers & gain creators) What is unique about it? (uniqueness) What is the value proposition? (value) How do we make money? (cost & revenue)

Build a roadmap

Start with your vision
Align with customer feedback
Balance innovation with customer needs
Group them into themes, with an outcome for each
Distill those into features, and validate with customers

Your THEMES should hold
Your OUTCOMES might hold
Your FEATURES will change

Always have delighters

Build features iteratively

	1	2	3
Paint the Mona Lisa Incremental
Woman in pastoral setting Iterative

Awesome analogy from Jeff Patton

Manage Risk

Small batches, near term hypothesis
Validation comes from customers
Introduce near-term volatility to gain decreased long-term risk

Do not worry about scale

Until you should

Way later than you think

Solve theory arguments with execution

Weekly
Anyone with anything to show
Invite everyone
Record it and post it internally

Choose languages and tools that fit the job

We are all polyglots
Learning new languages and tools is one of the great joys of our industry
Small batch + iterative development protects you from risk

Have a bug database

Triage and prioritize the bugs
Be nice to your customers

Continuous Integration

Always integrate branches to master
They should be short lived, iterative branches
Fix the build when it goes red.

The four-eye rule

Write tests

Unit test (a single function)
Integration tests (multiple classes/units)
Functional tests (user-oriented, high level, full stack)
Smoke tests (quickly determine if the system is "working")

One path for change

The way change moves through your organization is fixed
Designed to re-enforce your principles and aid flow
Flexible at the level of execution

Code goes through the same workflow

Applications are code
Infrastructure is code

Focus on Availability

Availability = uptime/(uptime+downtime)
Focus your efforts on reducing mean time to diagnose and mean time to repair
Failure is inevitable; it's how you detect and react that matter most

Collect metrics

From the operating system, network, applications, and your process
High resolution matters
Have as few systems as possible

Plan for capacity

Identify key metrics
Put them on a graph
Set a limit
Plot a trend line
Expand your time horizon

Only alert on what is actionable

Get the attention of the right humans
As few alerts as possible
Routed to the people who can take action
Start with the is it up alert

Practice incident response

Orient is the step that matters most

Practice incident response

The First Responder is the default Incident Commander
Decides what to do next
Coordinates resources
Communicates status
Not about rank
There is only ONE Incident Commander.

Incidents lead to Post Mortems

Invoke the space: we are here to learn, not to blame
Describe the incident
Establish the timeline
Identify contributing factors
Describe customer impact
Describe remediation tasks
Describe improvement tasks for response process

Use scalable systems design

Autonomous actors, responsible for themselves
Making progress toward their goal
Making clear promises to other actors
Having those actors evaluate the quality of those promises

About your code
About your position
About your knowledge

Simplicity

Extensibility

Re-use

Unique to the individual

Safety in technique

Remember your principles
Practice your forms
Use your discernment

Stage 1 - Problem Selection

Small enough to have a meaningful iteration in 8 weeks
Vertical not Horizontal

Stage 2 - Purpose, Beliefs and Teams

Write down your purpose
Write down your beliefs
Empower the team to act
Be available for context

Stage 3 - Product Development

Write down your value proposition
Build a roadmap (Themes, Outcomes, Features)
Include delighters
Simplicity, Extensibility, Re-use

Stage 4 - Iterate on Features

Iterate on features
Manage risk through small batches, validated with customers
Choose languages and tools that fit the job
Ignore scale
When arguing about theory, re-focus on execution
Demo every week

Stage 4 - Iterate on Features

Use source control
Have a bug database
One workflow for change
Four eyes

Stage 4 - Iterate on Features

Continuous Integration
Continuous Delivery
One test at a time (Unit, Integration, Functional, Smoke)
Use scalable systems design

Stage 5 - Operate

Focus on availability
Collect metrics
Plan for capacity
Alert on what is actionable
Run incident response
Hold post mortems

Stage 6 - Deliver

Final demo
Retrospective

Chef Style DevOps Kung Fu

On Github: http://github.com/chef/devops-kungfu
Join by PR
We work together to develop our principles, forms, and applications
Start your own school by forking, removing the names, and changing the content

Find your own way

DevOps is real
I know it by its universal principles
I practice it by its shared forms
I apply it to my daily work
My practice is unique - and so is yours