Shef
Shef, a wordplay on “shell” and “chef”, is a powerful CLI framework for cooking up dynamic shell recipes.
At its core, imagine that Make, GitHub Actions,
and CyberChef had a weird little <moira-rose>bea-by</>.
Shef allows you to chain multiple commands together, add interactive user prompts, loop using complex control structures,
easily run and manage background tasks, and build reusable workflows with advanced logic and conditionals.
Quick Start Examples
Hello World
This first example demonstrates a simple recipe that echos “Hello World!” to the terminal.
recipes:
- name: "hello-world"
description: "A simple hello world recipe"
operations:
- name: "Run echo command"
command: echo "Hello, World!"
Progress Bars
This next example demonstrates progress
bars and loops, where a recipe creates a number of temporary files, and then loops through the directory to clean up
each file it just created.
Background Tasks
This example demonstrates
a more advanced recipe where a user selects which background tasks to run and then monitors each task’s progress in
real-time.
[!TIP]
Want to see more before diving
deeper? Check out the demo recipes.
Table of Contents
Features
- Command Piping: Chain multiple commands together, passing output from one command to the next
- Transformations: Transform command output with powerful templating
- Interactive Prompts: Add user input, selections, confirmations, and more
- Conditional Logic: Use if/else branching based on command results
- Control Flow: Create dynamic workflows with loops and control structures
- Background Task Management: Easily monitor and control background tasks
- Progress Mode: Inline updates for clean status updates and progress bars
- Multiple Sources and Contexts: Use local, user, or public recipes
- Public Recipes: Common, useful recipes anyone can use. Browse
the public recipes.
A Note on Bash Scripts, YAML, and Shef
Bash Scripts
Bash scripting is a powerful and valid approach for shell automation. Shef isn’t designed to replace bash scripts, but
rather provides a toolkit that compliments bash when you need specific features.
Shef implements some common tooling like built-in support for interactive prompts, background task management, and CLI process
standardization. This structured approach can simplify certain tasks that might require more verbose code in bash.
Consider Shef as another tool in your automation toolkit. Absolutely use bash scripts when they’re the right fit, and
reach for Shef when its features align with your specific needs.
YAML
I considered several options, and YAML emerged as the most practical choice for this particular use case. JSON lacks
comments and multiline string support, which are essential when defining shell commands and documenting workflows. XML
would have been unnecessarily verbose. TOML, while nice, doesn’t handle nested structures as elegantly for complex workflows.
Installation
Homebrew
brew tap eduardoagarcia/tap
brew install shef
shef -v
Advanced Installation (Linux and Windows)
For detailed installation instructions on Linux and Windows platforms, please refer to the comprehensive installation
guide.
Quick Start
Once Shef is installed, you are ready to begin using it.
# Sync all public recipes locally
shef sync
# Run the Hello World recipe
shef demo hello-world
# List available recipes (demo recipes are excluded by default)
shef ls
# List all recipes within a category
shef ls demo
# View help information about a recipe
shef demo arguments -h
Command Reference
Basic Command Structure
Simple
shef [recipe-name]
Advanced
shef [global-flags] [category] [recipe-name] [input-text] [recipe-flags...]
Global Flags
| Flag |
Description |
-h, --help |
Show help information |
-v, --version |
Show version information |
-d, --debug |
Enable debug output |
--debug-file |
Save debug logs to specified file path |
-c, --category |
Specify a category |
-L, --local |
Force local recipes first |
-U, --user |
Force user recipes first |
-P, --public |
Force public recipes first |
-r, --recipe-file |
Path to the recipe file |
Utility Commands
| Command |
Description |
sync s |
Sync public recipes locally |
list ls l |
List available recipes (note: demo recipes are excluded by default) |
which w [category] [recipe-name] |
Show the location of a recipe file |
Recipe Sources
Shef looks for recipes in multiple locations and contexts within your system:
- Local Recipes:
./.shef/*.yaml in the current directory (only shown when you’re in the directory, like Make)
- User Recipes:
~/.shef/user/*.yaml in your home directory (always shown)
- Public Recipes:
~/.shef/public/*.yaml in your home directory (always shown)
XDG Base Directory Paths (Linux Only)
On Linux systems, Shef also supports the XDG Base Directory Specification:
- User Recipes:
$XDG_CONFIG_HOME/shef/user/*.yaml (defaults to ~/.config/shef/user/*.yaml)
- Public Recipes:
$XDG_DATA_HOME/shef/public/*.yaml (defaults to ~/.local/share/shef/public/*.yaml)
Shef includes both standard and XDG paths on Linux systems.
Source Priority
If you have recipes with the same name and category in different locations, you can prioritize a specific source:
shef -L git version # Prioritize local recipes
shef -U git version # Prioritize user recipes
shef -P git version # Prioritize public recipes
Recipe Structure
Recipes are defined in YAML files:
recipes:
- name: "example"
description: "An example recipe"
category: "demo"
vars:
name: "World"
workdir: "./my-project" # optional
help: |
This is detailed help text for the example recipe.
It can include multiple paragraphs and shows when users run:
shef demo example -h
operations:
- name: "First Operation"
id: "first_op"
command: echo "Hello, {{ .name }}!"
- name: "Second Operation"
id: "second_op"
command: ls -la
Key Recipe Components
- name: Unique identifier for the recipe
- description: Human-readable description
- category: Used for organization and filtering
- author: Optional author attribution
- help: Detailed help documentation shown when using
-h or --help flags
- vars: Optional pre-defined variables available to all operations in the recipe
- workdir: Optional working directory where all recipe commands will be executed (the directory will be created if it does not already exist)
- operations: List of operations to execute in sequence
Operations
Operations are the building blocks of recipes:
- name: "Operation Name" # Operation name
id: "var_id" # [Optional] Identifier for referencing the variable for the operation
command: echo "Hello" # [Optional] Shell command to execute
execution_mode: "standard" # [Optional] How the command runs (standard, interactive, stream, or background)
output_format: "raw" # [Optional] How to format command output (raw [default], trim, or lines)
silent: false # [Optional] Flag whether to suppress output to stdout. Default is false.
exit: false # [Optional] When set to true, the recipe will exit after the operation completes. Default is false.
condition: .var == "true" # [Optional] Condition for execution
on_success: "success_op" # [Optional] Operation to run on success (if not defined, Shef continues to next operation)
on_failure: "failure_op" # [Optional] Operation to run on failure
transform: "{{ trim .output }}" # [Optional] Transform output
raw_command: false # [Optional] When true, bypasses template rendering for the command. Default is false.
user_shell: false # [Optional] When true, runs command in user's interactive shell. Default is false.
prompts: # [Optional] Interactive prompts (can include one or more prompts)
- name: "Prompt Name"
id: "var_id"
type: "input"
message: "Enter value:"
control_flow: # [Optional] Control flow structure
type: "foreach" # Type of control flow (foreach, for, while)
operations: # [Optional] Sub-operations for control flows
- name: "Sub Operation"
command: echo "Processing " {{ .item }}
break: false # [Optional] When true, break out of a control flow and resume recipe
Execution Modes
- standard: Default mode (used when no
execution_mode is specified). Output is captured and can be used by
subsequent operations.
- interactive: The command has direct access to the terminal’s stdin, stdout, and stderr. Useful for commands that
require direct terminal interaction, but output cannot be captured for use in subsequent operations.
- stream: Similar to interactive mode but optimized for long-running processes that produce continuous output. The
command’s output streams to the terminal in real-time, but output cannot be captured for use in subsequent operations.
- background: Executes the command asynchronously in a separate process. The recipe execution continues immediately
without waiting for the command to complete. Useful for long-running tasks that don’t need to block recipe execution.
Shef provides options for controlling how whitespace and empty lines are handled in command output:
- raw: The default mode. Preserves all whitespace and newlines exactly as produced by the command, behaving like a
standard bash script.
- trim: Removes leading and trailing whitespace and newlines from the command output, useful for cleaner output when
exact formatting isn’t needed.
- lines: Splits the output by newlines, trims each line, removes empty lines, and joins them back together. Useful
for processing lists where empty lines and whitespace should be ignored.
Example Usage
operations:
- name: "Get file content with preserved formatting"
id: "read_file"
command: cat config.yaml
output_format: "raw" # Preserves all whitespace and newlines
- name: "Get version number"
id: "get_version"
command: echo " v1.2.3\n\n"
output_format: "trim" # Result: "v1.2.3"
- name: "Get clean list of items"
id: "list_items"
command: echo " item1 \n\n\n\n\n item2 \n\n item3 "
output_format: "lines" # Result: "item1\nitem2\nitem3"
Operation Execution Order
Each operation in a Shef recipe is executed in a specific order to ensure consistent behavior and proper flow control.
Execution Flow
Operations are executed in the following order:
- Condition Check: The condition (if specified) is evaluated first. If the condition is not met, the operation is
skipped entirely.
- Prompts: All prompts are processed next, collecting user input before any other execution occurs.
- Control Flow: If the operation has a control flow structure (foreach, while, for), it is executed after prompts
are collected.
- Command: The command is executed after the control flow completes.
- Transforms: Any transformations are applied to the command output.
- Success/Failure Handlers: Based on the operation result, either the on_success or on_failure handlers are
executed.
This order ensures that user input is collected before any execution, control flow structures are processed completely
before running commands, and transformations are applied to command outputs.
For a detailed flow chart, check out the Operation Flow Diagram.
Interactive User Prompts
Shef supports the following types of user prompts:
# Text Input
- name: "Username Input"
id: "username"
type: "input"
message: "Enter your username:"
default: "admin"
help_text: "This will be used for authentication"
# Selection
- name: "Environment Select"
id: "environment"
type: "select"
message: "Select environment:"
options:
- "dev"
- "staging"
- "production"
descriptions:
"dev": "Dev Environment"
"staging": "Staging Environment"
"production": "Production Environment"
default: "dev"
help_text: "Choose the deployment environment"
# Confirmation (yes/no)
- name: "Confirm Deploy"
id: "confirm_deploy"
type: "confirm"
message: "Deploy to production?"
default: "false"
help_text: "This will start the deployment process"
# Password (input is masked)
- name: "Password Input"
id: "password"
type: "password"
message: "Enter your password:"
help_text: "Your input will be hidden"
# Multi-select
- name: "Features Select"
id: "features"
type: "multiselect"
message: "Select features to enable:"
options:
- "logging"
- "metrics"
- "debugging"
default: "logging,metrics"
help_text: "Use space to toggle, enter to confirm"
# Numeric Input
- name: "Count Input"
id: "count"
type: "number"
message: "Enter number of instances:"
default: "3"
min_value: 1
max_value: 10
help_text: "Value must be between 1 and 10"
# File Path Input
- name: "Config File"
id: "config_file"
type: "path"
message: "Select configuration file:"
default: "./config.json"
file_extensions:
- "json"
- "yaml"
- "yml"
required: true
help_text: "File must exist and have the right extension"
# Text Editor
- name: "Description"
id: "description"
type: "editor"
message: "Enter a detailed description:"
default: "# Project Description\n\nEnter details here..."
editor_cmd: "vim" # Uses $EDITOR env var if not specified
help_text: "This will open your text editor"
# Autocomplete Selection
- name: "Service"
id: "service"
type: "autocomplete"
message: "Select a service:"
options:
- "authentication"
- "database"
- "storage"
- "analytics"
help_text: "Type to filter options"
Dynamic Options
You can generate selection options from a previous operation’s output:
- name: "List Files"
id: "files_list"
command: find . -name "*.go"
- name: "Select File"
command: cat {{ .file }}
prompts:
- name: "File Select"
id: "file"
type: "select"
message: "Select a file:"
source_operation: "files_list"
source_transform: "{{ trim .input }}"
Transformations let you modify a command’s output before it’s passed to the next operation.
[!IMPORTANT]
In Go templates (which Shef uses), parameters are passed in the order they appear in the function call. When using
pipe
syntax (|), the piped value becomes the last parameter of the function, not the first as in many template
engines.
Basic Syntax
Direct function call (recommended for clarity)
transform: "{{ function1 .output param1 }}"
Pipe syntax (parameters flipped - piped value becomes last parameter)
transform: "{{ param1 | function1 .output }}"
| Function |
Description |
Parameters |
Direct Example |
Pipe Example |
Input |
Output |
trim |
Remove whitespace |
(string) |
{{ trim .output }} |
{{ .output | trim }} |
" hello " |
"hello" |
split |
Split string by delimiter |
(string, delimiter) |
{{ split .output "," }} |
{{ "," | split .output }} |
"a,b,c" |
["a", "b", "c"] |
join |
Join array with delimiter |
(array, delimiter) |
{{ join .array "," }} |
{{ "," | join .array }} |
["a", "b", "c"] |
"a,b,c" |
joinArray |
Join any array type with delimiter |
(array, delimiter) |
{{ joinArray .array ":" }} |
{{ ":" | joinArray .array }} |
[1, 2, 3] |
"1:2:3" |
trimPrefix |
Remove prefix from string |
(string, prefix) |
{{ trimPrefix .output "pre" }} |
{{ "pre" | trimPrefix .output }} |
"prefix" |
"fix" |
trimSuffix |
Remove suffix from string |
(string, suffix) |
{{ trimSuffix .output "fix" }} |
{{ "fix" | trimSuffix .output }} |
"prefix" |
"pre" |
contains |
Check if string contains pattern |
(string, substring) |
{{ contains .output "pat" }} |
{{ "pat" | contains .output }} |
"pattern" |
true |
replace |
Replace text |
(string, old, new) |
{{ replace .output "old" "new" }} |
{{ "old" | replace .output "new" }} |
"oldtext" |
"newtext" |
filter, grep |
Keep lines containing a pattern |
(string, pattern) |
{{ filter .output "err" }} |
{{ "err" | filter .output }} |
"error\nok\nerror2" |
"error\nerror2" |
count |
Count array elements or lines in a string |
(value) |
{{ count .array }} or {{ count .output }} |
{{ .array | count }} |
["a", "b", "c"] or "line1\nline2" |
3 or 2 |
cut |
Extract field from each line |
(string, delimiter, field) |
{{ cut .output ":" 1 }} |
{{ ":" | cut .output 1 }} |
"name:value" |
"value" |
atoi |
Convert string to int |
(string) |
{{ atoi .output }} |
{{ .output | atoi }} |
"42" |
42 |
add |
Add numbers |
(num1, num2) |
{{ add 5 3 }} or {{ add .num 5 }} |
{{ 5 | add 3 }} |
5, 3 |
8 |
sub |
Subtract numbers |
(num1, num2) |
{{ sub 10 4 }} |
{{ 4 | sub 10 }} |
10, 4 |
6 |
div |
Divide numbers |
(num1, num2) |
{{ div 10 2 }} |
{{ 2 | div 10 }} |
10, 2 |
5 |
mul |
Multiply numbers |
(num1, num2) |
{{ mul 6 7 }} |
{{ 7 | mul 6 }} |
6, 7 |
42 |
exec |
Execute command |
(command) |
{{ exec "date" }} |
N/A |
"date" |
Output of date command |
color |
Add color to text |
(color, text) |
{{ color "green" "Success!" }} |
{{ "Success!" | color "green" }} |
"green", "Success!" |
Green-colored “Success!” |
style |
Add styling to text |
(style, text) |
{{ style "bold" "Important!" }} |
{{ "Important!" | style "bold" }} |
"bold", "Important!" |
Bold “Important!” |
resetFormat |
Reset colors and styles |
() |
{{ resetFormat }} |
N/A |
N/A |
ANSI reset code |
Available Math Functions
| Function |
Description |
Parameters |
Direct Example |
Pipe Example |
Input |
Output |
mod |
Modulo operation |
(a, b) |
{{ mod 10 3 }} |
{{ 3 | mod 10 }} |
10, 3 |
1 |
round |
Round to nearest integer |
(value) |
{{ round 3.7 }} |
{{ 3.7 | round }} |
3.7 |
4 |
rand |
Generate random integer in range |
(min, max) |
{{ rand 1 10 }} |
N/A |
1, 10 |
Random number 1-10 |
percent |
Calculate percentage |
(part, total) |
{{ percent 25 100 }} |
{{ 100 | percent 25 }} |
25, 100 |
25.0 |
formatPercent |
Format percentage with decimals |
(value, decimals) |
{{ formatPercent 33.333 1 }} |
N/A |
33.333, 1 |
"33.3%" |
ceil |
Round up to next integer |
(value) |
{{ ceil 3.1 }} |
{{ 3.1 | ceil }} |
3.1 |
4 |
floor |
Round down to integer |
(value) |
{{ floor 3.9 }} |
{{ 3.9 | floor }} |
3.9 |
3 |
abs |
Absolute value for floats |
(value) |
{{ abs -3.5 }} |
{{ -3.5 | abs }} |
-3.5 |
3.5 |
max |
Maximum of two integers |
(a, b) |
{{ max 5 10 }} |
{{ 10 | max 5 }} |
5, 10 |
10 |
min |
Minimum of two integers |
(a, b) |
{{ min 5 10 }} |
{{ 10 | min 5 }} |
5, 10 |
5 |
pow |
Power function |
(base, exponent) |
{{ pow 2 3 }} |
{{ 3 | pow 2 }} |
2, 3 |
8.0 |
sqrt |
Square root |
(value) |
{{ sqrt 9 }} |
{{ 9 | sqrt }} |
9 |
3.0 |
log |
Natural logarithm |
(value) |
{{ log 2.718 }} |
{{ 2.718 | log }} |
2.718 |
1.0 |
log10 |
Base-10 logarithm |
(value) |
{{ log10 100 }} |
{{ 100 | log10 }} |
100 |
2.0 |
formatNumber |
Format numbers with pattern |
(format, args…) |
{{ formatNumber "%.2f" 3.14159 }} |
N/A |
"%.2f", 3.14159 |
"3.14" |
roundTo |
Round to specified decimal places |
(value, decimals) |
{{ roundTo 3.14159 2 }} |
{{ 2 | roundTo 3.14159 }} |
3.14159, 2 |
3.14 |
Recommended Practices
- Use direct function calls for clarity rather than pipe syntax, especially for functions that take multiple
parameters
- For standard Go functions like
trimPrefix, remember they follow Go’s parameter ordering:// In standard Go
strings.TrimPrefix(str, prefix)
// In templates (direct call)
{{ trimPrefix .output "[" }} // Correct
// In templates (pipe syntax) parameters are reversed
{{ "[" | trimPrefix .output }} // Correct, but confusing
{{ .output | trimPrefix "[" }} // Incorrect as it will try to remove .output from "["
Function Groups
String Manipulation (Standard Go functions)
trim, trimPrefix, trimSuffix, split, join, contains, replace
Array Operations
joinArray (works with arrays of any type, unlike join which is for string arrays)
Text Processing
Numeric Operations
Math Operations
mod, round, ceil, floor, abs, max, min, pow, sqrt, log, log10, percent, formatPercent,
rand, roundTo, formatNumber
Shell Integration
color, style, resetFormat
Tables
Shef includes built-in functions for rendering tables:
table: Create tables with headers, rows, and stylingtableJSON: Create tables from JSON configurationmakeHeaders: Create a header row for a tablemakeRow: Create a data row for a tabletableStyleRounded, tableStyleLight, tableStyleDouble, tableStyleASCII: Style constants
Simple Table Example:
command: |
echo '{{ table
(makeHeaders "Name" "Age" "City")
(list
(makeRow "John" "34" "Chicago")
(makeRow "Jane" "28" "Seattle")
)
"rounded"
}}'
Table with Custom Style:
command: |
echo '{{ table
(makeHeaders "Product" "Price")
(list
(makeRow "Apple" "$1.25")
(makeRow "Orange" "$0.90")
)
(tableStyleDouble)
}}'
Table with Column Alignment:
"left": Left-align the column (default for text)"center": Center-align the column"right": Right-align the column (default for numbers)
command: |
echo '{{ table
(makeHeaders "Name" "Value" "Percentage")
(list
(makeRow "Item A" "150" "75%")
(makeRow "Item B" "42" "21%")
)
"rounded"
(list "left" "right" "center") # Left-align first column, right-align second, center-align third
}}'
command: |
echo '{{ tableJSON `{
"headers": ["Name", "Value", "Percentage"],
"rows": [
["Item A", "150", "75%"],
["Item B", "42", "21%"]
],
"align": ["left", "right", "center"],
"style": "rounded"
}` }}'
Terminal Colors and Styles
You can make your recipe outputs more readable by adding colors and styles. These are automatically disabled when using
the NO_COLOR environment variable.
Available Colors
| Color Type |
Available Colors |
| Text Colors |
black, red, green, yellow, blue, magenta, cyan, white |
| Background Colors |
bg-black, bg-red, bg-green, bg-yellow, bg-blue, bg-magenta, bg-cyan, bg-white |
Available Styles
| Style |
Description |
bold |
Bold text |
dim |
Dimmed text |
italic |
Italic text |
underline |
Underlined text |
Using Colors and Styles
Colors and styles can be used in commands, transformations, and anywhere templates are rendered:
Basic Color Usage
command: echo {{ color "green" "Success!" }}
Basic Style Usage
command: echo {{ style "bold" "Important!" }}
Combine Color and Style
command: echo {{ style "bold" (color "red" "Error!") }}
transform: |
{{ if contains .output "error" }}
{{ color "red" (style "bold" "✗ Operation failed") }}
{{ else }}
{{ color "green" (style "bold" "✓ Operation succeeded") }}
{{ end }}
Accessing Variables
You can access all context variables in transformations:
transform: "{{ if eq .format `json` }}{{ .output }}{{ else }}{{ .output | cut ` ` 0 }}{{ end }}"
Variables available in templates:
.output: The output to the current transformation (output from the command).input: The input to the current command (input from previous operation or the string input from the user running
the recipe).error: If an operation fails, the error will be captured here. Resets every operation..{variable_name}: Any variable, argument, or flag.{prompt_name}: Any variable from defined prompts.{operation_id}: The output of a specific operation by ID.operationOutputs: Map of all operation outputs by ID.operationResults: Map of operation success/failure results by ID
Undefined variables will always evaluate to the string value of "false"
Conditional Execution
Operations can be conditionally executed:
Basic Conditions
condition: .confirm == "true" # Run only if confirm prompt is true
Operation Result Conditions
condition: build_op.success # Run if build_op succeeded
condition: test_op.failure # Run if test_op failed
Variable Comparison
condition: .environment == "production" # Equality check
condition: .count != 0 # Inequality check
Numeric Comparison
condition: .count > 5
condition: .memory <= 512
condition: .errors >= 10
condition: .progress < 100
Complex Logic
condition: build_op.success && .confirm_deploy == "true"
condition: test_op.failure || lint_op.failure
condition: !.skip_validation
Branching Workflows
You can create branching workflows based on success or failure:
- name: "Build App"
id: "build_op"
command: make build
on_success: "deploy_op" # Go to deploy_op on success
on_failure: "fix_op" # Go to fix_op on failure
- name: "Deploy"
id: "deploy_op"
command: make deploy
- name: "Fix Issues"
id: "fix_op"
command: make lint
Data Flow Between Operations
Each operation’s output is automatically piped to the next operation’s input. You can also access any operation’s output
by its ID:
- name: "Get Hostname"
id: "hostname_op"
command: hostname
- name: "Show Info"
command: echo "Running on {{ .hostname_op }}"
Control Flow Structures
Shef supports advanced control flow structures that let you create dynamic, iterative workflows.
Foreach Loops
You can iterate over a collection of items and perform a flow of operations on each item.
Key Foreach Components
- control_flow
- type: foreach
- collection: The list of items to iterate over (string with items separated by newlines)
- as: The variable name to use for the current item in each iteration
- operations: The sub-operations to perform for each item (all sub-operations have access to the
as loop variable)
Mechanics of the Foreach Loop
- Parse the collection into separate items (splitting by newlines)
- For each item, set the variable specified in “as”
- Execute all operations in the foreach block for each item
- Clean up the loop variable when done
Common Uses
- Processing multiple files
- Handling lists of servers, containers, or resources
- Applying the same transformation to multiple inputs
- Building dynamic workflows based on discovered items
Within a foreach loop, you can use conditional operations, transformations, and all other Shef features.
Example Foreach Recipes
- name: "Process Each Item"
control_flow:
type: "foreach"
collection: "🍎 Apples\n🍌 Bananas\n🍒 Cherries\n🍊 Oranges"
as: "fruit" # Each item will be available as .fruit
operations:
- name: "Process Fruit"
command: echo "Processing {{ .fruit }}"
You can also generate the collection dynamically:
- name: "List Files"
id: "files"
command: find . -type f -name "*.txt"
- name: "Process Each File"
control_flow:
type: "foreach"
collection: "{{ .files }}" # Using output from previous operation
as: "file" # Each item will be available as .file
operations:
- name: "Process File"
command: cat {{ .file }} | wc -l
For Loops
You can execute a set of operations a fixed number of times.
Key For Loop Components
- control_flow
- type: for
- count: The number of iterations to execute
- variable: (Optional) The variable name to use for the current iteration index (defaults to “i”)
- operations: The sub-operations to perform for each iteration
Mechanics of the For Loop
- Parse and evaluate the count value to determine the number of iterations
- For each iteration, set the loop variable to the current index (starting from 0)
- Also set the
.iteration variable to the 1-based iteration number
- Execute all operations in the operations block for each iteration
- Clean up the loop variables when done
Common Uses
- Repeating an operation a fixed number of times
- Creating numbered resources or items
- Running tests multiple times
- Implementing retry logic with a maximum attempt limit
Within a for loop, both the specified variable (zero-based index) and .iteration (one-based counter) are available.
Example For Loop Recipes
- name: "Run a For Loop"
control_flow:
type: "for"
count: 5
variable: "i"
operations:
- name: "Print Iteration"
command: 'echo "Running iteration {{ .iteration }} (zero-based index: {{ .i }})"'
You can also use a dynamic count:
- name: "Get Count"
id: "count"
command: echo "3"
transform: "{{ trim .input }}"
- name: "Dynamic For Loop"
control_flow:
type: "for"
count: "{{ .count }}"
variable: "step"
operations:
- name: "Execute Step"
command: echo "Executing step {{ .step }} of {{ .count }}"
While Loops
You can repeatedly execute operations as long as a condition remains true.
Key While Loop Components
- control_flow
- type: while
- condition: The condition to evaluate before each iteration
- operations: The sub-operations to perform for each iteration
Mechanics of the While Loop
- Evaluate the condition before each iteration
- If the condition is true, execute the operations and repeat
- If the condition is false, immediately exit the loop
- An
.iteration variable is automatically set to track the current iteration (starting from 1)
Common Uses
- Polling for a condition (e.g., waiting for a service to be ready)
- Processing data until a certain state is reached
- Implementing retry logic with conditional termination
- Continuously monitoring resources until a specific event occurs
Within a while loop, the .iteration variable lets you track how many iterations have occurred.
Example While Loop Recipes
- name: "Initialize Status"
id: "status"
command: echo "running"
transform: "{{ trim .input }}"
silent: true
- name: "Wait For Completion"
control_flow:
type: "while"
condition: .status == "running"
operations:
- name: "Check Status"
command: echo "Checking status (iteration {{ .iteration }})..."
id: "status"
transform: "{{ if eq .iteration 5 }}completed{{ else }}running{{ end }}"
Real-world polling example:
- name: "Poll Service Until Ready"
control_flow:
type: "while"
condition: .status != "ready"
operations:
- name: "Check Service Status"
id: "status"
command: curl -s http://service/status
transform: "{{ trim .output }}"
Duration Time Tracking in Loops
All loop types in Shef (for, foreach, and while) automatically track duration. This allows you to measure
execution time, implement timeouts, or provide progress feedback in your recipes.
Available Duration Variables
Inside any loop, the following variables are available:
| Variable |
Type |
Description |
Example |
duration_ms |
String |
Total milliseconds elapsed |
“12345” |
duration_s |
String |
Total seconds elapsed (as whole number) |
“12” |
duration |
String |
Formatted time (MM:SS or HH:MM:SS) |
“00:12” or “1:23:45” |
duration_ms_fmt |
String |
Formatted time with milliseconds |
“00:12.345” or “1:23:45.678” |
Usage Examples
Displaying progress with timing:
operations:
- name: "Process Files with Duration Tracking"
control_flow:
type: foreach
collection: "{{ exec `find . -type f -name '*.txt' | sort` }}"
as: file
operations:
- name: "Process file with duration info"
command: "echo 'Processing {{ .file }} (elapsed: {{ .duration }})'"
Implementing a timeout condition:
operations:
- name: "Loop with timeout"
control_flow:
type: while
condition: .duration_s < 30 # Timeout after 30 seconds
operations:
- name: "Do something until timeout"
command: "echo 'Working... ({{ .duration }} elapsed)'"
- name: "Wait a bit"
command: "sleep 1"
operations:
- name: "Performance test"
id: perf_test
control_flow:
type: for
count: "100"
variable: i
operations:
- name: "Run test iteration"
command: "your_command --iteration {{ i }}"
- name: "Display progress"
command: "echo 'Completed {{ .iteration }}/100 in {{ .duration }}'"
- name: "Show results"
command: "echo 'Test completed in {{ perf_test.duration_ms_fmt }}'"
[!NOTE]
Duration variables persist after the loop completes, allowing you to access the total execution time of the loop in
subsequent operations.
Progress Mode
Progress mode allows operations within loops to update in-place on a single line, creating a cleaner interface for
status updates and progress indicators.
Key Progress Mode Features
- control_flow
- progress_mode: When set to
true, enables single-line updates for all operations in the loop
- Behavior: Each operation’s output replaces the previous output on the same line rather than printing new lines
- Limitations: Only the first line of output is displayed; additional lines are ignored
Mechanics of Progress Mode
- When a loop has
progress_mode: true, all operations within the loop display their output on a single line
- Each new output overwrites the previous output (returning to the start of the line)
- At the end of the loop, a newline is automatically added to maintain clean formatting
Common Uses
- Displaying progress counters (e.g., “Processing 5/100…”)
- Showing status updates for long-running operations
- Creating animated loading indicators
- Providing real-time feedback without cluttering the terminal
[!TIP]
Progress mode works best for simple, concise status messages. For complex multi-line output, standard mode is more
appropriate.
Example Progress Mode Recipes
With a for loop:
- name: "Download Progress Example"
control_flow:
type: "for"
count: 100
variable: "i"
progress_mode: true
operations:
- name: "Update Progress"
command: echo "Downloading... {{ .i }}% complete"
With a while loop:
- name: "Service Monitor Example"
control_flow:
type: "while"
condition: .duration_s < 60 # Monitor for 60 seconds
progress_mode: true
operations:
- name: "Check Service Status"
command: echo "Monitoring service... ({{ .duration_fmt }} elapsed)"
- name: "Wait"
command: sleep 1
With a foreach loop:
- name: "Batch Processing Example"
control_flow:
type: "foreach"
collection: "{{ .files }}"
as: "file"
progress_mode: true
operations:
- name: "Process File"
command: echo "Processing {{ .file }}"
Background Task Management
When using execution_mode: "background", Shef provides template functions to monitor and interact with background
tasks:
- bgTaskStatus: Returns the current status of a background task (
pending, complete, or failed)
- bgTaskComplete: Returns
true if the task has completed successfully, false otherwise
- bgTaskFailed: Returns
true if the task has failed, false otherwise
- allTasksComplete: Returns
true if all tasks are complete, false if one or more tasks are pending.
- anyTasksFailed: Returns
true if one ore more tasks fail, false if all tasks completed successfully.
- taskStatusMessage: Returns a string based on the status of the task.
{{ taskStatusMessage "task_id" "complete message" "pending message" "failed message" "unknown task message" }}
Requirements for Background Tasks
- Each background task must have a unique
id specified
- The task’s output will be available as a variable using the task’s ID once completed
Task Status Checking
Check a task’s status
- name: "Check Status"
command:
echo 'Task status: {{ bgTaskStatus "task_id" }}'
Notice we reference the task id by a string when checking status, complete, and failed states.
Wait for one task to complete
- name: "Wait For One Task"
control_flow:
type: "while"
condition: '{{ not (bgTaskComplete "task_id") }}'
operations:
- name: "Poll Status"
command: echo "Waiting for task to complete..."
Wait for all tasks to complete
- name: "Wait For All Tasks"
control_flow:
type: "while"
condition: .allTasksComplete != "true"
operations:
- name: "Poll Status"
command: echo "Waiting for all tasks to complete..."
Task Output Access
Once a background task completes, its output is available like any other operation:
- name: "Echo Task Output"
command: echo "Task result {{ .task_id }}"
condition: '{{ bgTaskComplete "task_id" }}'
Background Task Completion Behavior
When you start a background task with execution_mode: "background", it’s important to understand how Shef handles task
completion:
- Implicit Waiting: Shef automatically waits for all background tasks to complete before terminating the recipe
execution, even if you don’t explicitly wait for them in your operations.
- Output Capture: All background tasks will have their outputs captured and made available as variables, regardless
of whether you explicitly check their status.
- Completion Order: Background tasks complete in the order determined by their execution time, not the order they
were started.
- Recipe Exit: The recipe won’t exit until all background tasks have completed, which could cause the recipe to
appear to “hang” if a background task takes a very long time.
Example of implicit waiting:
- name: "Start Long Task"
id: "long_task"
command: "sleep 30 && echo 'Done!'"
execution_mode: "background"
- name: "Immediate Feedback"
command: echo "Started background task! Recipe will wait for it to complete before exiting."
Arguments and Flags
Shef allows you to pass arguments and flags directly to your recipes from the command line.
Basic Argument Syntax
shef [category] [recipe-name] [input-text] [flags...]
or without category:
shef [recipe-name] [input-text] [flags...]
Available Flag Types
| Flag Type |
Example |
Variable |
Value |
| Input Text |
"Hello World" |
.input |
"Hello World" |
| Short Flag (boolean only) |
-f |
.f |
true |
| Long Flag |
--name=John |
.name |
"John" |
| Long Flag with dash |
--user-agent=Chrome |
.user_agent |
"Chrome" |
| Multi-short (boolean only) |
-abc |
.a, .b, .c |
true |
Usage Examples
# Pass text input to a recipe
shef demo arguments "Hello World"
# Pass a boolean flag
shef demo arguments -f
# Pass a value flag
shef demo arguments --name=John
# Combine multiple types
shef demo arguments "My input text" -vf -a --count=5 --verbose
Accessing Arguments in Recipes
You can access these values in your recipe operations:
- name: "Display Arguments"
command: |
echo "Input: {{ .input }}"
echo "Flag f: {{ .f }}"
echo "Name: {{ .name }}"
- name: "Conditional based on flag"
command: echo "Flag was set!"
condition: .f == true
Recipe Help Documentation
Shef provides a built-in help system for recipes, allowing users to get detailed information about a recipe’s purpose,
requirements, and usage without having to read the recipe code.
Accessing Recipe Help
Help for any recipe can be accessed in two ways:
# Using the -h flag
shef demo hello-world -h
# Using the --help flag
shef demo hello-world --help
When a user requests help for a recipe, they receive formatted information:
NAME:
recipe-name - short description from the description field
CATEGORY:
recipe-category
USAGE:
shef recipe-name [input] [options]
shef category recipe-name [input] [options]
OVERVIEW:
Detailed help text from the help field.
If no help field is provided, Shef shows a default message.
Writing Effective Help Documentation
When creating recipes, consider including comprehensive help text that covers:
- Purpose: What the recipe does and when to use it
- Requirements: Any prerequisites or dependencies
- Examples: Sample usages with different options
- Parameters: Available flags and arguments with explanations
- Expected Output: What users should expect to see
- Troubleshooting: Common issues and how to resolve them
Example Recipe with Help Documentation
recipes:
- name: "database-backup"
description: "Backup a MySQL database"
category: "database"
help: |
This recipe creates a backup of a MySQL database and stores it in a
timestamped file.
Requirements:
- MySQL client tools must be installed
- Database credentials with read permissions
Examples:
shef database-backup # Uses interactive prompts
shef database-backup --db=mydb # Specify database name
shef database-backup --no-compress # Skip compression
shef database-backup --output=/backups # Custom output directory
Flags:
--db=NAME # Database name
--user=USER # Database username
--pass=PASS # Database password
--host=HOST # Database host (default: localhost)
--no-compress # Skip compression
--output=DIR # Output directory (default: ./backups)
operations:
# Operations follow here
Components
Components are reusable modules that can be incorporated into any recipe. Components enable you to create building
blocks that can be shared across one or multiple recipes.
Component Overview
Components provide the following benefits:
- Reusability: Define operations once, use them in multiple recipes
- Maintainability: Update a single component rather than modifying multiple recipes
- Modularity: Break complex workflows into logical, reusable blocks
- Standardization: Create consistent patterns across your organization
- Parameterization: Pass inputs to components for flexibility and customization
Defining Components
Components can be defined in the same file as recipes, or in separate files entirely.
components:
- id: "backup-database"
name: "Database Backup Component"
description: "Creates a timestamped database backup"
inputs:
- id: "db_name"
name: "Database Name"
description: "The name of the database to back up"
required: true
- id: "db_user"
name: "Database Username"
description: "Username for database connection"
required: true
- id: "db_password"
name: "Database Password"
description: "Password for database connection"
required: true
- id: "output_dir"
name: "Output Directory"
description: "Directory to store backups"
default: "./backups"
operations:
- name: "Get Current Timestamp"
id: "timestamp"
command: "date +%Y%m%d_%H%M%S"
output_format: "trim"
- name: "Create Backup"
id: "backup_file"
command: "mysqldump --user={{ .db_user }} --password={{ .db_password }} {{ .db_name }} > {{ .output_dir }}/backup_{{ .timestamp }}.sql"
Key Components
- id: Unique identifier for referencing the component (required)
- name: Human-readable name for the component
- description: Purpose and functionality of the component
- inputs: Define inputs that the component accepts
- id: Variable name for the input (required)
- name: Display name for the input
- description: Purpose of the input parameter
- required: Whether the input must be provided
- default: Default value if input is not provided
- operations: List of operations to execute when the component is used
Using Components in Recipes
Include components in your recipes using the uses field:
recipes:
- name: "nightly_backup"
description: "Performs database backup every night"
operations:
- name: "Prompt for Database Credentials"
prompts:
- name: "Database Name"
id: "db_name"
type: "input"
message: "Enter database name:"
- name: "Database User"
id: "db_user"
type: "input"
message: "Enter database username:"
- name: "Database Password"
id: "db_password"
type: "password"
message: "Enter database password:"
- name: "Backup Database"
uses: "backup_database"
with:
db_name: "{{ .db_name }}"
db_user: "{{ .db_user }}"
db_password: "{{ .db_password }}"
output_dir: "/backups/nightly"
id: "db_backup"
- name: "Show Backup Complete"
command: "echo 'Backup completed: {{ .backup_file }}'"
Components can access variables from multiple sources:
-
Component Inputs: Values provided through the with field
- Each input is available within the component using its ID
- Required inputs must be provided, while optional inputs can use their default values
- Inputs have their own namespace to prevent collisions
-
Recipe Context: If not provided through with, components can access:
- Recipe Variables: Variables defined in the recipe’s
vars section
- Prompt Results: Values from recipe prompts
- Operation Outputs: Outputs from operations before the component
Using the with Parameter
The with parameter allows you to pass values to component inputs:
- name: "Use Component With Parameters"
uses: "my_component"
with:
param1: "value1" # Static value
param2: "{{ .user_input }}" # Dynamic value from a prompt
param3: 42 # Numeric value
Capturing Component Outputs
You can capture outputs from components in two ways:
-
Component-level ID: Assign an ID to the uses operation to capture its output
- name: "Backup Database"
uses: "backup_database"
id: "db_backup" # Will contain the output of the last operation in the component
-
Operation-level IDs: Access individual operation outputs from within the component
# After using the backup-database component
- name: "Show Timestamp"
command: "echo 'Backup created at: {{ .timestamp }}'"
Component Sources
Components follow the same source priority as recipes:
- Local Components:
./.shef/*.yaml in the current directory
- User Components:
~/.shef/user/*.yaml in your home directory
- Public Components:
~/.shef/public/*.yaml in your home directory
Advanced Component Usage
Nesting Components
Components can use other components, allowing you to build more complex structures:
components:
- id: "setup-environment"
name: "Environment Setup"
operations:
- name: "Create Directory"
command: "mkdir -p workspace"
- id: "deploy-application"
name: "Application Deployment"
operations:
- name: "Setup Environment First"
uses: "setup-environment"
- name: "Deploy Code"
command: "cp -r ./app/* workspace/"
Component Conditions
Just like operations, component usage can be conditional:
- name: "Optional Database Backup"
uses: "backup-database"
condition: .perform_backup == "true"
Component Error Handling
Handle component failures using the same success/failure patterns as regular operations:
- name: "Backup Database"
uses: "backup-database"
on_success: "notify_success"
on_failure: "notify_failure"
Progress Bars
You can utilize progress bars for long-running operations within loops,
offering visual feedback about iteration progress, timing, and completion status.
Basics
Progress bars can be added to any for or foreach loop in your recipes:
- name: "Process With Progress Bar"
control_flow:
type: "for"
count: 100
variable: "i"
progress_bar: true # Enable the progress bar
operations:
- name: "Process Item"
command: sleep 0.1
Customization
Progress bars can be extensively customized using the progress_bar_options field:
progress_bar_options:
description: "Processing Files" # Text shown at the start of the bar (optional)
width: 50 # Width in characters (default: terminal width)
show_count: true # Show "5/100" counts (default: true)
show_percentage: true # Show percentage (default: true)
show_elapsed_time: true # Show elapsed time (default: true)
show_iteration_speed: true # Show iterations/second (default: false)
refresh_rate: 0.1 # Update rate in seconds (default: every iteration)
message_template: "Custom message here" # Dynamic message template with variable replacement (optional)
theme: # Visual appearance customization
saucer: "[green]=[reset]" # Bar fill character
saucer_head: "[green]>[reset]" # Leading character
saucer_padding: " " # Empty bar character
bar_start: "[" # Left bracket
bar_end: "]" # Right bracket
Theme Customization
Progress bars can be styled with ANSI colors and custom characters:
theme:
saucer: "[green]=[reset]"
saucer_head: "[green]>[reset]"
saucer_padding: " "
bar_start: "["
bar_end: "]"
Color Options
All text elements support ANSI colors:
| Color Values |
Description |
[black] |
Black text |
[red] |
Red text |
[green] |
Green text |
[yellow] |
Yellow text |
[blue] |
Blue text |
[magenta] |
Magenta text |
[cyan] |
Cyan text |
[white] |
White text |
[reset] |
Reset to default color |
Example Recipes
For even more examples, check out the demo recipes.
Hello World
recipes:
- name: "hello-world"
description: "A simple hello world recipe"
category: "demo"
operations:
- name: "Greet User"
command: |
echo "Hello, {{ .name }}!"
echo "Current time: $(date)"
echo "Welcome to Shef, the shell recipe tool."
prompts:
- name: "Name Input"
id: "name"
type: "input"
message: "What is your name?"
default: "World"
Success and Error Handling
recipes:
- name: "handlers"
description: "A simple demo of error and success handlers"
category: "demo"
operations:
- name: "Help"
command: echo {{ color "magenta" "Please provide a valid or invalid directory argument to test" }}
condition: .input == "false"
- name: "Test if directory exists"
command: cd {{ .input }}
on_failure: "handle_error"
on_success: "handle_success"
condition: .input != "false"
- name: "Handle error"
id: "handle_error"
command: echo {{ color "red" (printf "Directory %s does NOT exist!" .input) }}
- name: "Handle success"
id: "handle_success"
command: echo {{ color "green" (printf "Directory %s exists!" .input) }}
recipes:
- name: "transform"
description: "A simple demo of data transformation and pipeline flow"
category: "demo"
operations:
- name: "Generate a Simple List"
id: "generate"
command: |
echo "apple
banana
cherry
dragonfruit
eggplant"
- name: "Filter 'a' Items"
id: "filter_a"
command: cat
transform: '{{ filter .generate "a" }}'
silent: true
- name: "Display 'a' Results"
id: "display_a"
command: echo "Items containing 'a':\n{{ color "green" .filter_a }}"
Creating Recipes
To create your own recipes:
- Create your user directory:
mkdir -p ~/.shef/user (if it does not already exist)
- Create a new YAML file:
touch ~/.shef/user/my-recipes.yaml
- Build and develop your recipes following the instructions above
- Run
shef ls to see your new recipes
AI-Assisted Recipe Creation
You can generate powerful Shef recipes quickly using AI tools like ChatGPT, Claude, or other large language models.
Using AI to Create Recipes
- Copy the prompt below
- Paste it into your AI assistant of choice
- Replace
[DESCRIBE YOUR WORKFLOW IN DETAIL] with a detailed description of your recipe’s workflow
- The AI will generate a complete Shef recipe based on your description
- Test and iterate until the recipe works as expected
Example Usage
Here’s an example of how to fill in your workflow description:
I need a Docker management workflow that helps me clean up unused containers and images to free up disk space.
The workflow should:
1. Show the current Docker disk usage
2. List all stopped containers and allow me to select which ones to remove
3. Confirm before removing the selected containers
4. List dangling images (unused) and allow me to select which ones to remove
5. Offer an option to perform a more aggressive cleanup (removing all unused images)
6. Show before/after disk usage comparison
7. Include error handling in case any operation fails
The recipe should be interactive and safe, requiring confirmation before any destructive operations.
Tips for Better Results
- Use the latest AI models with advanced reasoning capabilities
- Be specific about what commands should be executed at each step
- Mention if you need interactive prompts, conditions, or transformations
- For complex workflows, break down your requirements into clear, logical steps
- Include any specific error handling or conditional branches you need
- Request comments in the generated recipe to explain complex sections
- Ask the AI to analyze and iterate on its recipe solution, considering edge cases and improvements
- If the first recipe doesn’t fully meet your needs, refine your requirements and ask for adjustments
Remember, the AI-generated recipes can provide an excellent starting point that you can further customize to fit your
exact needs.
The Prompt
I need help creating a Shef recipe. Shef is a CLI tool that combines shell commands into reusable recipes defined in YAML.
[DESCRIBE YOUR WORKFLOW IN DETAIL]
A Shef recipe is defined in YAML with this structure:
recipes:
- name: "short-name"
description: "Human-readable description"
category: "category"
author: "optional author"
operations:
- name: "Operation Name"
id: "unique_id"
command: "shell command"
execution_mode: "standard|interactive|stream" # How the command runs
silent: true|false # Whether to suppress the command's output. Default is false
exit: true|false # Whether to exit the recipe after operation completes. Default is false
condition: "optional condition"
on_success: "next_op_id"
on_failure: "fallback_op_id"
transform: "{{ .input | transformation }}"
prompts:
- name: "Input Name"
id: "variable_name"
type: "input|select|confirm|password|multiselect|number|editor|path|autocomplete"
message: "Prompt message"
default: "Default value"
help_text: "Additional help information"
required: true|false # Whether input is required
options: ["option1", "option2"] # For select/multiselect/autocomplete types
source_operation: "operation_id" # For dynamic options
source_transform: "{{ .input | transform }}" # For processing source options
min_value: 0 # For number type
max_value: 100 # For number type
file_extensions: ["txt", "json"] # For path type
multiple_limit: 3 # For multiselect type
editor_cmd: "vim" # For editor type
control_flow:
type: "foreach|for|while" # Type of control flow
collection: "Item 1\nItem 2\nItem 3</moira-rose>
> README 内容过长，已截取前 64 KB。完整内容请前往 GitHub 仓库查看。
评论