Restructure and enhance cheat sheets for improved readability (#566)

* docs(cheatsheets): expand and align the chmod, git, docker and swarm memos

Unify the four cheat sheets on one layout: short intro, emoji section
headings, grouped command blocks with aligned inline comments, reference
tables and a resources section. Placeholders are consistently <angled>.

- chmod: ls -l breakdown, octal and symbolic tables, special bits, umask,
  recursive/selective changes, recipes, related commands and gotchas;
  fix the directory access matrix (r alone does not grant file access)
  and the --preserve-root default
- git: config and aliases, staging, commit, branching, merge/rebase with
  conflict flow, remotes, stashing, inspection, cherry-pick, tags, undo
  with a reset-mode table, cleaning, bisect, submodules and worktrees
- docker: contexts and events, build flags, registries, run/exec/inspect,
  volumes and mounts, networks, the full compose workflow, cleanup and
  one-liners
- swarm: required firewall ports, node and manager operations, live
  service updates, job modes, placement expressions, publish modes,
  health checks, secret and config rotation, task states and a
  troubleshooting table

* docs(cheatsheets): put descriptions above commands so each line copies clean

Trailing inline comments meant selecting a single command also picked up
the padding and its description. Move every description onto its own line
above the command it documents, keeping one code block per topic.

- each command now stands alone on its line: triple-click copies exactly
  the command, nothing else
- no comment column to re-pad when a longer command is added later, so
  adding a line no longer reindents the whole block
- code lines drop from a median of 78 columns to ~22 (p90 42), so blocks
  no longer scroll sideways on narrow screens
- wrap the three remaining long commands over several lines
- reword the descriptions that only made sense trailing a command

* docs(cheatsheets): rework the tmux, screen, zellij, nano, jq, zpool, CSS and JSONPath memos

Bring eight more cheat sheets onto the same layout as the Docker and Git
ones: short intro, emoji section headings, one code block per topic with
the description above each command, reference tables, resources at the end.

- tmux: session/window/pane/copy-mode tables, scripting with send-keys and
  capture-pane, a starting .tmux.conf
- screen: shared and detached attach modes, split regions, copy mode,
  logging and monitoring, nested-screen prefix
- zellij: correct the pane-mode bindings (d/r split, not v/h), add tab,
  resize, move, search and session modes, layouts, zellij run/edit/action
- nano: add the command-line options and .nanorc settings the sheet was
  missing, drop the stray H1 and broken headings
- jq: flags, core syntax, object/array/string tables, conditionals,
  variables, reduce, and runnable recipes; drop the stale AWS AMI note
  and the HTML pipe entities
- zpool: vdev and support-vdev types, import/export, properties, dataset
  and snapshot/replication commands, tuning notes
- CSS selectors: split one 60-row table into basic, combinator, attribute,
  structural, state, form, functional and pseudo-element groups; add
  :is()/:where()/:has() and a specificity table
- JSONPath: RFC 9535 functions, worked example document, filter operators

* docs(cheatsheets): rework the Dockerfile, Compose and kubectl memos

- Dockerfile: instruction reference table, BuildKit cache/secret/bind
  mounts and heredocs, COPY --chown/--chmod/--from, HEALTHCHECK,
  STOPSIGNAL, a CMD vs ENTRYPOINT table, build commands and a worked
  multi-stage example
- Compose: drop the obsolete version: framing, merge the two duplicated
  "Compose vs Swarm" sections into one table, correct the claim that
  file-based secrets are Swarm-only, and add depends_on conditions,
  profiles, watch mode, include/extends/anchors, env interpolation and
  the full CLI section
- kubectl: replace the removed rolling-update and the retired Heapster
  section, add contexts, custom-columns and JSONPath output, rollouts,
  debug containers, port-forward, secrets, labels, node maintenance and
  one-liners; tag the code blocks so they highlight

* docs(cheatsheets): rework the sed, regex and common-regex memos

- sed: how the pattern space works, an addressing table, the full command
  reference including hold-space and branching, multi-line recipes, and a
  warning about BSD/macOS -i; merge the duplicated deletion and addressing
  sections
- regex: add flags, lazy quantifiers, named groups and backreferences,
  Unicode property escapes, replacement tokens, the JavaScript methods
  that expose groups, and a gotchas section (stateful /g, catastrophic
  backtracking, escaping user input)
- common-regex: rewrite the patterns that markdown escaping had mangled
  into unusable text, group them by subject, and add UUID, semver, CIDR,
  MAC, ISO 8601, duration, log-line and CSV patterns

Every pattern in common-regex now compiles, and the ones people copy most
(IPv4, UUID, semver, E.164, ISO dates, password rules, slug, duration,
MAC) were checked against matching and non-matching input.

* docs(cheatsheets): rework the XPath, conventional-commits and RJ45 memos

- XPath: give every table a CSS-equivalent column, fix the broken
  "only children of type" expression, flag which functions are 2.0+ and
  give the 1.0 workarounds for ends-with and case folding, add the axes
  short forms, worked examples, and how to run XPath from devtools,
  xmllint, lxml, Selenium and Playwright
- conventional commits: types table with the release effect each one
  has, description do/don't table, footer reference, both ways to mark a
  breaking change, a version-bump table, and husky v9 hook syntax
- RJ45: keep the colour swatches and the pinout image, add what each pin
  carries on 10/100 vs Gigabit, the PoE standards table (correcting the
  claim that the blue pair is the PoE pair), gigabit crossover, Cat8,
  length and solid-vs-stranded rules, and a fault-finding table

* docs(cheatsheets): rework the nginx and PowerShell memos

- nginx: the whole sheet was a single 218-line code block; split it into
  sections with a service/CLI block, a file-layout table, location
  matching priority, root vs alias, reverse proxy and WebSocket headers,
  load-balancing methods, TLS, rate limiting, caching and gzip, a
  variable reference and a 502/504 troubleshooting table
- PowerShell: add discovery (Get-Help/Get-Command/Get-Member), functions
  and param blocks, try/catch and error preferences, Invoke-RestMethod
  and JSON/CSV conversion, services, jobs, remoting, modules and
  execution policy, PowerShell 7 operators, an alias table, and repair
  the mangled escape-character rows

* docs(cheatsheets): rework the vim and bash memos

Both sheets were a single unbroken code block with ASCII banners, so the
whole page rendered as one <pre> with nothing to scan. Rebuild them as
sections with key/action tables and runnable blocks.

- vim: explain the count + operator + motion grammar, add the text-object
  table (ciw, ci", da(), registers and why "0p exists, the :g and :v
  global commands, folds, completion, diff mode, undo by time, a :set
  reference and a starter vimrc
- bash: parameter expansion, arrays including associative ones, the test
  operator tables, redirection and process substitution, job control and
  traps, set -euo pipefail with its caveat, shellcheck, and an ANSI
  colour table in place of sixty lines of colour variables

The vim sheet dropped roughly a hundred lines of one person's leader-key
mappings and plugin bindings (NERDTree, Taglist, sparkup, PHP DOC), which
only worked with their dotfiles; a note now explains that leader mappings
come from your own vimrc.
This commit is contained in:
Shaquille_Oatmeal authored and GitHub committed 2026-08-30 16:38:56 +02:00
1 parent a5607e04ad
commit 5bf94bc567
25 files changed
+6206 -4556

No files matched your search

File diff suppressed because it is too large. Load diff
+197 -71
View File
@@ -1,119 +1,245 @@
The `chmod` command changes file and directory permissions in Unix/Linux.
**`chmod`** (*change mode*) sets who may read, write, and execute a file or directory on Unix-like systems. The mode can be written **symbolically** (`u+x`) or **numerically** (`755`) — both reach the same result.
## 🧑 User Classes
```bash
chmod [options] <mode> <file>...
```
| Symbol | Meaning |
|--------|----------------|
| `u` | User (owner) |
| `g` | Group |
| `o` | Others |
| `a` | All (u+g+o) |
> 💡 Inspect before and after with `ls -l <file>`, or get the octal value directly with `stat -c '%a %n' <file>`.
## 🔐 Permission Types
## 🗂 Reading `ls -l` Output
| Symbol | Meaning | Numeric |
|--------|----------|---------|
| `r` | Read | 4 |
| `w` | Write | 2 |
| `x` | Execute | 1 |
```text
- rwx r-x r-- → 754
│ │ │ │
│ │ │ └── others : r-- = 4
│ │ └─────── group : r-x = 5
│ └──────────── user : rwx = 7
└─────────────── type : - file · d directory · l symlink · c/b device
```
## 🧑 User Classes (*who*)
| Symbol | Class | Applies to |
|--------|--------|-------------------------------------------------|
| `u` | User | The file's owner |
| `g` | Group | Members of the file's group |
| `o` | Others | Everyone else |
| `a` | All | `u`, `g` and `o` at once (implied when omitted) |
## 🔐 Permission Bits (*what*)
| Symbol | Permission | Octal | On a file | On a directory |
|--------|------------|-------|---------------------|-------------------------------------------|
| `r` | Read | `4` | Read the contents | List entry names (`ls`) |
| `w` | Write | `2` | Modify the contents | Create, rename and delete entries |
| `x` | Execute | `1` | Run as a program | Enter it (`cd`) and reach entries by name |
| `-` | None | `0` | — | — |
## 📝 Symbolic Mode
Use `+`, `-`, or `=` to modify permissions:
| Operator | Effect |
|----------|-------------------------------------------------------------------|
| `+` | Add the listed permissions, leave the rest untouched |
| `-` | Remove the listed permissions, leave the rest untouched |
| `=` | Set exactly these permissions, clearing the others for that class |
```bash
chmod u+x file # Add execute to user
chmod go-w file # Remove write from group and others
chmod a=r file # Set read-only for all
# add execute for the owner
chmod u+x <file>
# remove write from the group
chmod g-w <file>
# cut off group and others entirely
chmod go-rwx <file>
# everyone gets read only, nothing else
chmod a=r <file>
# several clauses, comma separated
chmod u=rwx,go=rx <file>
# 'a' is implied, then filtered by the umask
chmod +x <file>
# regular and special bits in one call
chmod u+x,g+s <file>
# copy the mode of another file
chmod --reference=<ref> <file>
```
> 💡 Use `X` (capital) instead of `x` to add execute **only** to directories and to files that already have an execute bit — the safe way to fix a tree: `chmod -R u=rwX,go=rX <dir>`.
## 🔢 Numeric (Octal) Mode
Combine values for user, group, and others:
Each digit is the sum of `r` (4), `w` (2) and `x` (1):
| Mode | Permissions | Meaning |
|------|-------------|---------------|
| 777 | rwxrwxrwx | Full access |
| 755 | rwxr-xr-x | Common for scripts |
| 644 | rw-r--r-- | Common for text files |
| Octal | Sum | Bits |
|-------|-------|-------|
| `7` | 4+2+1 | `rwx` |
| `6` | 4+2 | `rw-` |
| `5` | 4+1 | `r-x` |
| `4` | 4 | `r--` |
| `3` | 2+1 | `-wx` |
| `2` | 2 | `-w-` |
| `1` | 1 | `--x` |
| `0` | 0 | `---` |
Three digits map to **user, group, others**. An optional fourth *leading* digit sets the special bits (`chmod 4755`, see below).
### Common modes
| Mode | Symbolic | Typical use |
|-------|-------------|-------------------------------------------------------------|
| `777` | `rwxrwxrwx` | Full access for everyone — ⚠️ almost never the right answer |
| `755` | `rwxr-xr-x` | Scripts, binaries and directories |
| `750` | `rwxr-x---` | Executable for the owner and their group only |
| `700` | `rwx------` | Private directory (`~/.ssh`, `~/.gnupg`) |
| `664` | `rw-rw-r--` | Files shared read-write inside a group |
| `644` | `rw-r--r--` | Regular files, web assets, configs |
| `640` | `rw-r-----` | Config with secrets, readable by a service group |
| `600` | `rw-------` | Private files (`~/.ssh/id_ed25519`, `.env`) |
| `400` | `r--------` | Read-only key material |
```bash
chmod 755 script.sh
chmod 644 notes.txt
chmod 600 ~/.ssh/id_ed25519
```
## 🧪 Special Permission Flags
## 🧪 Special Permission Bits
These flags go beyond basic read/write/execute and control execution context and deletion behavior.
These go beyond read/write/execute and control the execution context and deletion rules.
### 🔸 `setuid` (Set User ID on Execution)
- **Octal prefix**: `4`
- **Applies to**: Executable files
- **Effect**: When executed, the process runs with the file owner's privileges (not the user's).
- **Use case**: Programs like `passwd` need elevated privileges to modify system files.
| Bit | Octal | Symbolic | Applies to | Effect | Shown in `ls -l` as |
|----------|-------|----------|--------------------------|----------------------------------------------------------------------|-------------------------|
| `setuid` | `4` | `u+s` | Executable files | Runs with the **file owner's** privileges | `s` in the user field |
| `setgid` | `2` | `g+s` | Executables, directories | Runs as the file's group / new entries inherit the directory's group | `s` in the group field |
| `sticky` | `1` | `+t` | Directories | Only the entry's owner (or root) may delete or rename it | `t` in the others field |
```bash
# setuid → -rwsr-xr-x
chmod 4755 /usr/bin/somebinary
# -rwsr-xr-x → 's' replaces 'x' in user field
# setgid → drwxrwsr-x: one group for the whole tree
chmod 2775 /srv/shared
# sticky → drwxrwxrwt: a shared drop box, like /tmp
chmod 1777 /srv/tmp
# strip every special bit again
chmod u-s,g-s,o-t <file>
```
> ⚠️ Security risk if misused — avoid setting on scripts or user-controlled binaries.
> ⚠️ `setuid` is a privilege-escalation vector — never set it on scripts or on binaries a user can replace. Linux ignores it on interpreted scripts anyway.
>
> ℹ️ A capital `S` or `T` in `ls -l` (`-rwSr--r--`) means the special bit is set while the matching `x` bit is **not** — usually a mistake.
### 🔸 `setgid` (Set Group ID)
## 📂 Directory Access Matrix
- **Octal prefix**: `2`
- **Applies to**:
- **Files**: Runs with the file group’s privileges.
- **Directories**: New files inherit the directory’s group.
- **Use case**: Shared project folders where all files should belong to the same group.
`x` on a directory means *traverse*, not *execute* — without it nothing below the directory can be reached, no matter how permissive the files themselves are.
| Mode | List names (`ls`) | Details (`ls -l`, `stat`) | Enter (`cd`) | Open a known file | Create / delete entries |
|-------|-------------------|---------------------------|--------------|-------------------|-------------------------|
| `r--` | ✅ | ❌ | ❌ | ❌ | ❌ |
| `--x` | ❌ | ✅ (by exact name) | ✅ | ✅ | ❌ |
| `r-x` | ✅ | ✅ | ✅ | ✅ | ❌ |
| `rwx` | ✅ | ✅ | ✅ | ✅ | ✅ |
| `-w-` | ❌ | ❌ | ❌ | ❌ | ❌ (write needs `x`) |
## 🔁 Recursive & Selective Changes
```bash
chmod 2755 /shared/folder
# drwxr-sr-x → 's' in group field
# blunt: also marks every file executable
chmod -R 755 <dir>
# preferred: X only touches dirs + real executables
chmod -R u=rwX,go=rX <dir>
# directories only
find <dir> -type d -exec chmod 755 {} +
# files only
find <dir> -type f -exec chmod 644 {} +
# make every shell script executable
find <dir> -type f -name '*.sh' -exec chmod +x {} +
```
> 📁 On directories, this ensures group consistency for collaborative environments.
## ⚙️ Useful Options
### 🔸 `sticky` Bit
| Option | Description |
|-----------------------------|-----------------------------------------------------------------|
| `-R`, `--recursive` | Apply to directories and everything inside them |
| `-v`, `--verbose` | Print a line for every file processed |
| `-c`, `--changes` | Print a line only when a mode actually changed |
| `-f`, `--silent`, `--quiet` | Suppress most error messages |
| `--reference=<file>` | Reuse the mode of another file instead of spelling it out |
| `--preserve-root` | Refuse to run recursively on `/` — not the default, unlike `rm` |
| `-h` (BSD/macOS) | Act on the symlink itself instead of its target |
- **Octal prefix**: `1`
- **Applies to**: Directories only
- **Effect**: Only the file owner (or root) can delete or rename files inside.
- **Use case**: `/tmp` directory — prevents users from deleting each other’s files.
## 🎭 `umask` — Default Permissions for New Files
New files start from `666` and new directories from `777`; the umask bits are removed from that base.
```bash
chmod 1777 /tmp
# drwxrwxrwt → 't' in others field
# show the current mask, e.g. 0022
umask
# show it symbolically, e.g. u=rwx,g=rx,o=rx
umask -S
# stricter default for this shell session
umask 027
```
> 🧷 Think of it as a “delete lock” for shared spaces.
| umask | New files | New directories |
|-------|-----------|-----------------|
| `022` | `644` | `755` |
| `002` | `664` | `775` |
| `027` | `640` | `750` |
| `077` | `600` | `700` |
## 📂 Execute (`x`) Flag: Files vs Directories
> 💡 Put `umask 027` in `~/.bashrc` (or a systemd unit's `UMask=`) to make the default stick.
| Context | Effect of `x` Permission |
|-------------|----------------------------------------|
| **File** | Allows execution as a program/script |
| **Directory** | Allows entering the directory (`cd`) and accessing contents by name |
## 🚑 Common Recipes
### 🔍 Directory Access Matrix
| Goal | Command |
|-------------------------------------|------------------------------------------------------------------------------|
| Make a script runnable | `chmod +x deploy.sh` |
| Lock down an SSH key pair | `chmod 700 ~/.ssh && chmod 600 ~/.ssh/id_* && chmod 644 ~/.ssh/*.pub` |
| Serve web content read-only | `chmod -R u=rwX,go=rX /var/www/site` |
| Shared team directory (one group) | `chmod 2775 /srv/shared` |
| Public scratch directory | `chmod 1777 /srv/tmp` |
| Clone the mode of a known-good file | `chmod --reference=good.conf new.conf` |
| Undo an accidental `chmod -R 777 .` | `find . -type d -exec chmod 755 {} + && find . -type f -exec chmod 644 {} +` |
| Permission | Can List (`ls`) | Can Enter (`cd`) | Can Access Files |
|------------|------------------|------------------|------------------|
| `r` | ✅ | ❌ | ✅ (if name known) |
| `x` | ❌ | ✅ | ✅ (if name known) |
| `r+x` | ✅ | ✅ | ✅ |
## 🧰 Related Commands
> 🧠 Without `x` on a directory, you can't `cd` into it — even if you can list its contents.
| Command | Purpose |
|-----------------------------------|--------------------------------------------------------------------|
| `chown <user>:<group> <file>` | Change the owner and/or group |
| `chgrp <group> <file>` | Change the group only |
| `ls -l` / `ls -ld <dir>` | Show permissions (`-d` for the directory itself, not its contents) |
| `stat -c '%a %A %U:%G %n' <file>` | Octal mode, symbolic mode, owner and group in one line |
| `namei -l <path>` | Show the permissions of every component of a path |
| `getfacl` / `setfacl` | Fine-grained ACLs when `u/g/o` is not expressive enough |
| `id` / `groups` | Check which groups you actually belong to |
## 🔍 Quick Reference
## ⚠️ Gotchas
| Command | Description |
|---------------------|--------------------------------------|
| `chmod +x file` | Make file executable |
| `chmod -R 755 dir` | Recursively set permissions |
| `chmod u=rwx file` | Set user to rwx only |
| `chmod a-x file` | Remove execute from all |
- **`chmod 777` is not a fix.** It grants every local user write access; the real problem is almost always ownership (`chown`) or a missing `x` on a parent directory.
- **"Permission denied" on a file with `rw-`?** Every directory in the path needs `x`. Debug the whole chain with `namei -l /path/to/file`.
- **Interpreted scripts need `r` too** — the kernel hands the file to the interpreter, which has to read it.
- **Group changes need a new login** (or `newgrp <group>`) before the shell picks them up.
- **FAT/NTFS/exFAT mounts ignore `chmod`** — set permissions with the `umask=`, `fmask=` and `dmask=` mount options instead.
- **Removing `x` from a directory hides everything below it**, even world-readable files.
💡 Tip: Use `ls -l` to view current permissions.
## 📚 Resources
- [GNU coreutils — `chmod` invocation](https://www.gnu.org/software/coreutils/manual/html_node/chmod-invocation.html)
- [GNU coreutils — file permissions](https://www.gnu.org/software/coreutils/manual/html_node/File-permissions.html)
- Man pages: `man 1 chmod` · `man 1 umask` · `man 1 chown` · `man 5 acl`
+205 -126
View File
@@ -1,174 +1,253 @@
Patterns you reach for again and again, written for the **JavaScript/PCRE** flavour. Drop the surrounding anchors (`^`…`$`) when you want to *find* matches inside a longer text instead of validating a whole string.
### Commonly used Regex
#### 1\. Digits
> ⚠️ Validation by regex is always an approximation. For emails, phone numbers, URLs and credit cards, a regex is a cheap first filter — the real check is a library, or sending the confirmation.
* [Whole Numbers](https://www.regexpal.com/?fam=104020) – `/^\d+$/`
* [Decimal Numbers](https://www.regexpal.com/?fam=104021) – `/^\d*\.\d+$/`
* [Whole + Decimal Numbers](https://www.regexpal.com/?fam=104022) – `/^\d*(\.\d+)?$/`
* [Negative, Positive Whole + Decimal Numbers](https://www.regexpal.com/?fam=104023) – `/^-?\d*(\.\d+)?$/`
* [Whole + Decimal + Fractions](https://www.regexpal.com/94462) – `/[-]?[0-9]+[,.]?[0-9]*([\/][0-9]+[,.]?[0-9]*)*/`
## 🔢 Numbers
#### 2\. Alphanumeric Characters
```regex
# whole number
^\d+$
* [Alphanumeric without space](https://www.regexpal.com/?fam=104024) – `/^[a-zA-Z0-9]*$/`
* [Alphanumeric with space](https://www.regexpal.com/?fam=104025) – `/^[a-zA-Z0-9 ]*$/`
# decimal number
^\d*\.\d+$
#### 3\. Email
# whole or decimal
^\d*(\.\d+)?$
* [Common email Ids](https://www.regexpal.com/?fam=104026) – `/^([a-zA-Z0-9._%-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,6})*$/`
* [Uncommon email ids](https://www.regexpal.com/?fam=104027) – `/^([a-z0-9_\.\+-]+)@([\da-z\.-]+)\.([a-z\.]{2,6})$/`
# optional sign, whole or decimal
^[+-]?\d*(\.\d+)?$
#### 4\. Password Strength
# thousands separators: 1,234,567.89
^\d{1,3}(,\d{3})*(\.\d+)?$
* [Complex](https://www.regexpal.com/?fam=104028): Should have 1 lowercase letter, 1 uppercase letter, 1 number, 1 special character and be at least 8 characters long
# percentage, 0–100 with optional decimals
^(100(\.0+)?|\d{1,2}(\.\d+)?)%?$
/(?=(.\*\[0-9\]))(?=.\*\[\\!@#$%^&\*()\\\\\[\\\]{}\\-\_+=~\`|:;"'<>,./?\])(?=.\*\[a-z\])(?=(.\*\[A-Z\]))(?=(.\*)).{8,}/
# hexadecimal number
^(0[xX])?[0-9a-fA-F]+$
* [Moderate](https://www.regexpal.com/?fam=104029): Should have 1 lowercase letter, 1 uppercase letter, 1 number, and be at least 8 characters long
/(?=(.\*\[0-9\]))((?=.\*\[A-Za-z0-9\])(?=.\*\[A-Z\])(?=.\*\[a-z\]))^.{8,}$/
#### 5\. Username
* [Alphanumeric string](https://www.regexpal.com/?fam=104030) that may include \_ and – having a length of 3 to 16 characters – `/^[a-z0-9_-]{3,16}$/`
#### 6\. URL
* Include [http(s) Protocol](https://www.regexpal.com/?fam=104034)
```
/https?:\\/\\/(www\\.)?\[-a-zA-Z0-9@:%.\_\\+~#=\]{2,256}\\.\[a-z\]{2,6}\\b(\[-a-zA-Z0-9@:%\_\\+.~#()?&//=\]\*)/
# currency amount: $1,234.56
^\$?\d{1,3}(,?\d{3})*(\.\d{2})?$
```
* [Protocol Optional](https://www.regexpal.com/?fam=104035)
## 🔤 Text & Identifiers
```
/(https?:\\/\\/)?(www\\.)?\[-a-zA-Z0-9@:%.\_\\+~#=\]{2,256}\\.\[a-z\]{2,6}\\b(\[-a-zA-Z0-9@:%\_\\+.~#?&//=\]\*)/
```regex
# alphanumeric, no spaces
^[a-zA-Z0-9]+$
# alphanumeric with spaces
^[a-zA-Z0-9 ]+$
# username: 3–16 chars, letters, digits, underscore, dash
^[a-z0-9_-]{3,16}$
# slug: lowercase words joined by single dashes
^[a-z0-9]+(?:-[a-z0-9]+)*$
# hex colour, 3 or 6 digits
^#?([0-9a-fA-F]{3}|[0-9a-fA-F]{6})$
# UUID v4
^[0-9a-f]{8}-[0-9a-f]{4}-4[0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}$
# semantic version (major.minor.patch with optional pre-release and build)
^\d+\.\d+\.\d+(-[0-9A-Za-z.-]+)?(\+[0-9A-Za-z.-]+)?$
# leading and trailing whitespace
^\s+|\s+$
# blank or whitespace-only line
^\s*$
```
#### 7\. IP Address
## 📧 Email
* [IPv4 address](https://www.regexpal.com/?fam=104036)
```
/^(([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])\.){3}([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])$/
```
* [IPv6 address](https://www.regexpal.com/?fam=104037)
```
/(([0-9a-fA-F]{1,4}:){7,7}[0-9a-fA-F]{1,4}|([0-9a-fA-F]{1,4}:){1,7}:|([0-9a-fA-F]{1,4}:){1,6}:[0-9a-fA-F]{1,4}|([0-9a-fA-F]{1,4}:){1,5}(:[0-9a-fA-F]{1,4}){1,2}|([0-9a-fA-F]{1,4}:){1,4}(:[0-9a-fA-F]{1,4}){1,3}|([0-9a-fA-F]{1,4}:){1,3}(:[0-9a-fA-F]{1,4}){1,4}|([0-9a-fA-F]{1,4}:){1,2}(:[0-9a-fA-F]{1,4}){1,5}|[0-9a-fA-F]{1,4}:((:[0-9a-fA-F]{1,4}){1,6})|:((:[0-9a-fA-F]{1,4}){1,7}|:)|fe80:(:[0-9a-fA-F]{0,4}){0,4}%[0-9a-zA-Z]{1,}|::(ffff(:0{1,4}){0,1}:){0,1}((25[0-5]|(2[0-4]|1{0,1}[0-9]){0,1}[0-9])\.){3,3}(25[0-5]|(2[0-4]|1{0,1}[0-9]){0,1}[0-9])|([0-9a-fA-F]{1,4}:){1,4}:((25[0-5]|(2[0-4]|1{0,1}[0-9]){0,1}[0-9])\.){3,3}(25[0-5]|(2[0-4]|1{0,1}[0-9]){0,1}[0-9]))/
```regex
# pragmatic: good enough for a form field
^[^\s@]+@[^\s@]+\.[^\s@]+$
# stricter on the allowed characters
^[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}$
# find addresses inside a body of text
[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}
```
* Both [IPv4, IPv6 addresses](https://www.regexpal.com/?fam=104038)
```
/((^\s*((([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])\.){3}([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5]))\s*$)|(^\s*((([0-9A-Fa-f]{1,4}:){7}([0-9A-Fa-f]{1,4}|:))|(([0-9A-Fa-f]{1,4}:){6}(:[0-9A-Fa-f]{1,4}|((25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)(\.(25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)){3})|:))|(([0-9A-Fa-f]{1,4}:){5}(((:[0-9A-Fa-f]{1,4}){1,2})|:((25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)(\.(25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)){3})|:))|(([0-9A-Fa-f]{1,4}:){4}(((:[0-9A-Fa-f]{1,4}){1,3})|((:[0-9A-Fa-f]{1,4})?:((25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)(\.(25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)){3}))|:))|(([0-9A-Fa-f]{1,4}:){3}(((:[0-9A-Fa-f]{1,4}){1,4})|((:[0-9A-Fa-f]{1,4}){0,2}:((25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)(\.(25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)){3}))|:))|(([0-9A-Fa-f]{1,4}:){2}(((:[0-9A-Fa-f]{1,4}){1,5})|((:[0-9A-Fa-f]{1,4}){0,3}:((25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)(\.(25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)){3}))|:))|(([0-9A-Fa-f]{1,4}:){1}(((:[0-9A-Fa-f]{1,4}){1,6})|((:[0-9A-Fa-f]{1,4}){0,4}:((25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)(\.(25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)){3}))|:))|(:(((:[0-9A-Fa-f]{1,4}){1,7})|((:[0-9A-Fa-f]{1,4}){0,5}:((25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)(\.(25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)){3}))|:)))(%.+)?\s*$))/`
## 🔗 URLs
```regex
# http(s) URL
^https?:\/\/[^\s/$.?#].[^\s]*$
# protocol optional
^(https?:\/\/)?([\w-]+\.)+[a-z]{2,}(\/[^\s]*)?$
# domain name only
^([a-zA-Z0-9-]+\.)+[a-zA-Z]{2,}$
# extract every link from text
https?:\/\/[^\s<>"']+
# capture the parts: protocol, host, path
^(https?):\/\/([^\/\s:]+)(?::(\d+))?(\/[^\s?#]*)?
```
#### 8\. Dates
## 🌐 Network
* Date Format [YYYY-MM-dd](https://www.regexpal.com/?fam=104039) using separator `-`
`/([12]\d{3}-(0[1-9]|1[0-2])-(0[1-9]|[12]\d|3[01]))/`
* Date Format [dd-MM-YYYY](https://regexr.com/?346hf) using separators `-` or `.` or `/`
`/^(?:(?:31(\/|-|\.)(?:0?[13578]|1[02]))\1|(?:(?:29|30)(\/|-|\.)(?:0?[1,3-9]|1[0-2])\2))(?:(?:1[6-9]|[2-9]\d)?\d{2})$|^(?:29(\/|-|\.)0?2\3(?:(?:(?:1[6-9]|[2-9]\d)?(?:0[48]|[2468][048]|[13579][26])|(?:(?:16|[2468][048]|[3579][26])00))))$|^(?:0?[1-9]|1\d|2[0-8])(\/|-|\.)(?:(?:0?[1-9])|(?:1[0-2]))\4(?:(?:1[6-9]|[2-9]\d)?\d{2})$/`
* Date Format [dd-mmm-YYYY](https://regexr.com/39tr1) using separators `-` or `.` or `/`
`/^(?:(?:31(\/|-|\.)(?:0?[13578]|1[02]|(?:Jan|Mar|May|Jul|Aug|Oct|Dec)))\1|(?:(?:29|30)(\/|-|\.)(?:0?[1,3-9]|1[0-2]|(?:Jan|Mar|Apr|May|Jun|Jul|Aug|Sep|Oct|Nov|Dec))\2))(?:(?:1[6-9]|[2-9]\d)?\d{2})$|^(?:29(\/|-|\.)(?:0?2|(?:Feb))\3(?:(?:(?:1[6-9]|[2-9]\d)?(?:0[48]|[2468][048]|[13579][26])|(?:(?:16|[2468][048]|[3579][26])00))))$|^(?:0?[1-9]|1\d|2[0-8])(\/|-|\.)(?:(?:0?[1-9]|(?:Jan|Feb|Mar|Apr|May|Jun|Jul|Aug|Sep))|(?:1[0-2]|(?:Oct|Nov|Dec)))\4(?:(?:1[6-9]|[2-9]\d)?\d{2})$/`
#### 9\. Time
```regex
# IPv4 address
^((25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)\.){3}(25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)$
* Time Format [HH:MM 12-hour](https://www.regexpal.com/?fam=104040), optional leading 0
# IPv4 with CIDR prefix
^((25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)\.){3}(25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)\/(3[0-2]|[12]?\d)$
`/^(0?[1-9]|1[0-2]):[0-5][0-9]$/`
# IPv6, full or compressed
^(([0-9a-fA-F]{1,4}:){7}[0-9a-fA-F]{1,4}|([0-9a-fA-F]{1,4}:){1,7}:|([0-9a-fA-F]{1,4}:){1,6}:[0-9a-fA-F]{1,4}|([0-9a-fA-F]{1,4}:){1,5}(:[0-9a-fA-F]{1,4}){1,2}|([0-9a-fA-F]{1,4}:){1,4}(:[0-9a-fA-F]{1,4}){1,3}|([0-9a-fA-F]{1,4}:){1,3}(:[0-9a-fA-F]{1,4}){1,4}|([0-9a-fA-F]{1,4}:){1,2}(:[0-9a-fA-F]{1,4}){1,5}|[0-9a-fA-F]{1,4}:(:[0-9a-fA-F]{1,4}){1,6}|:((:[0-9a-fA-F]{1,4}){1,7}|:))$
* Time Format HH:MM 12-hour, optional leading 0, **[Meridiems (AM/PM)](https://www.regexpal.com/?fam=104041)**
# MAC address, colon- or dash-separated
^([0-9A-Fa-f]{2}[:-]){5}[0-9A-Fa-f]{2}$
`/((1[0-2]|0?[1-9]):([0-5][0-9]) ?([AaPp][Mm]))/`
* Time Format [HH:MM 24-hour](https://www.regexpal.com/?fam=104042) with leading 0
`/^(0[0-9]|1[0-9]|2[0-3]):[0-5][0-9]$/`
* Time Format [HH:MM 24-hour, optional leading 0](https://www.regexpal.com/?fam=104043)
`/^([0-9]|0[0-9]|1[0-9]|2[0-3]):[0-5][0-9]$/`
* Time Format [HH:MM:SS 24-hour](https://www.regexpal.com/?fam=104044)
`/(?:[01]\d|2[0123]):(?:[012345]\d):(?:[012345]\d)/`
#### 10\. HTML Tags
* [Elements with Attributes](https://www.regexpal.com/95941) `/<\/?[\w\s]*>|<.+[\W]>/`
#### 11\. Javascript Handlers
* [Inline JS handler](https://www.regexpal.com/?fam=104055) `/\bon\w+=\S+(?=.*>)/`
* [Inline JS handler with element](https://www.regexpal.com/94641) `/(?:<[^>]+\s)(on\S+)=["']?((?:.(?!["']?\s+(?:\S+)=|[>"']))+.)["']?/`
#### 12\. Slug
* [Slug](https://www.regexpal.com/?fam=104056) `/^[a-z0-9]+(?:-[a-z0-9]+)*$/`
#### 13\. Match Duplicates in a String
* [Search Duplicates](https://www.regexpal.com/?fam=104060) `/(\b\w+\b)(?=.*\b\1\b)/`
#### 14\. Phone Numbers
* [International Phone Numbers](https://www.regexpal.com/?fam=99127) – with optional country code/extension
```
`/* International Phone Numbers */`
/^(?:(?:\(?(?:00|\+)([1-4]\d\d|[1-9]\d?)\)?)?[\-\.\ \\\/]?)?((?:\(?\d{1,}\)?[\-\.\ \\\/]?){0,})(?:[\-\.\ \\\/]?(?:``#|ext\.?|extension|x)[\-\.\ \\\/]?(\d+))?$/
# port number, 0–65535
^([0-9]{1,4}|[1-5][0-9]{4}|6[0-4][0-9]{3}|65[0-4][0-9]{2}|655[0-2][0-9]|6553[0-5])$
```
_Note:_ Use regex for validating phone numbers **only** if you don’t have the choice to use a library. There are [several libraries](https://stackoverflow.com/a/15644461/4717533) that handle phone numbers more accurately and should be used instead.
## 📅 Dates & Times
#### 15\. File Path
```regex
# ISO 8601 date: 2026-08-22
^\d{4}-(0[1-9]|1[0-2])-(0[1-9]|[12]\d|3[01])$
* [File Path with Filename and extension](https://www.regexpal.com/?fam=104047)
# ISO 8601 date and time with optional zone
^\d{4}-\d{2}-\d{2}[T ]\d{2}:\d{2}(:\d{2}(\.\d+)?)?(Z|[+-]\d{2}:?\d{2})?$
```
/((\\/|\\\\|\\/\\/|https?:\\\\\\\\|https?:\\/\\/)\[a-z0-9 \_@\\-^!#$%&+={}.\\/\\\\\\\[\\\]\]+)+\\.\[a-z\]+$/
# day-first with - . or / as separator
^(0?[1-9]|[12]\d|3[01])([-./])(0?[1-9]|1[0-2])\2\d{4}$
# 24-hour time, optional seconds
^([01]\d|2[0-3]):[0-5]\d(:[0-5]\d)?$
# 12-hour time with meridiem
^(0?[1-9]|1[0-2]):[0-5]\d\s?([AaPp][Mm])$
# duration in ISO 8601: P1DT2H30M
^P(?!$)(\d+Y)?(\d+M)?(\d+D)?(T(?=\d)(\d+H)?(\d+M)?(\d+S)?)?$
```
* File Path with optional Filename, extension
> ℹ️ These check the *shape*, not the calendar — `2026-02-31` passes. Parse the value if the date has to be real.
```
/^(.+)/(\[^/\]+)$/
## 🔐 Password Rules
Each requirement is a lookahead; combine only the ones you actually need.
```regex
# at least 8 characters, one lower, one upper, one digit
^(?=.*[a-z])(?=.*[A-Z])(?=.*\d).{8,}$
# the same, plus one special character
^(?=.*[a-z])(?=.*[A-Z])(?=.*\d)(?=.*[^\w\s]).{8,}$
# at least 12 characters, no other rules (the modern recommendation)
^.{12,}$
# individual checks, to report which rule failed
[a-z] # has a lower-case letter
[A-Z] # has an upper-case letter
\d # has a digit
[^\w\s] # has a symbol
```
* [File Name with extension](https://www.regexpal.com/?fam=104048) having 3 chars
## ☎️ Phone Numbers
```
/^\[\\w,\\s-\]+\\.\[A-Za-z\]{3}$/
```regex
# E.164, the format to store numbers in
^\+[1-9]\d{1,14}$
# lenient international, allowing spaces, dashes, dots and parentheses
^\+?[\d\s().-]{7,20}$
# North American number, with or without country code
^(\+?1[-. ]?)?\(?\d{3}\)?[-. ]?\d{3}[-. ]?\d{4}$
```
### Additional Regexes
> 💡 Use a library (libphonenumber and its ports) for anything beyond a smoke test — numbering plans change, and regex cannot know which prefixes exist.
#### 1\. Zip codes
## 📄 Files & Paths
There is **NO** single Regex that can handle all zip codes given that zip codes around the world do not follow a common pattern. Here is a [list](https://stackoverflow.com/a/7185241/4717533) that contains Regex specific to each country.
```regex
# file name with an extension
^[\w,\s-]+\.[A-Za-z0-9]+$
#### 2\. Payment Validation
# capture the name and the extension separately
^(.+?)\.([A-Za-z0-9]+)$
Here is a [link](https://www.regular-expressions.info/creditcard.html) that contains regex for validating leading Credit cards like Visa, Mastercard and so on.
# extension only
\.[0-9a-z]+$
#### 3\. Identity Documents
# absolute Unix path
^(\/[^\/\0]+)+\/?$
* Social Security Number – [Ref](https://www.codeproject.com/Articles/651609/Validating-Social-Security-Numbers-through-Regular)
# Windows path
^[a-zA-Z]:\\(?:[^\\/:*?"<>|\r\n]+\\)*[^\\/:*?"<>|\r\n]*$
```
/\* can use either hypen(-) or space( ) character as separator \*/
/^((?!219-09-9999|078-05-1120)(?!666|000|9\\d{2})\\d{3}-(?!00)\\d{2}-(?!0{4})\\d{4})|((?!219 09 9999|078 05 1120)(?!666|000|9\\d{2})\\d{3} (?!00)\\d{2} (?!0{4})\\d{4})|((?!219099999|078051120)(?!666|000|9\\d{2})\\d{3}(?!00)\\d{2}(?!0{4})\\d{4})$/
# image file
\.(jpe?g|png|gif|webp|avif|svg)$
```
* Passport – `/^[A-PR-WY][1-9]\d\s?\d{4}[1-9]$/`
## 🏷 Markup & Logs
#### References
* [https://www.codeproject.com/validate-passport-number](https://www.codeproject.com/Questions/1046445/How-to-validate-passport-number-using-javascript)
* [https://gist.github.com/](https://gist.github.com/nerdsrescueme/1237767)
* [https://code.tutsplus.com/regular-expressions](https://code.tutsplus.com/tutorials/8-regular-expressions-you-should-know--net-6149)
* [https://www.regular-expressions.info](https://www.regular-expressions.info/)
* [http://www.regexlib.com/](http://www.regexlib.com/)
* [https://projects.lukehaas.me/regexhub/](https://projects.lukehaas.me/regexhub/)
* [http://wiki.zoolz.com/commonly-used-regular-expressions/](http://wiki.zoolz.com/commonly-used-regular-expressions/)
* [https://www.smashingmagazine.com/advanced-regular-expressions/](https://www.smashingmagazine.com/2009/05/introduction-to-advanced-regular-expressions/)
```regex
# any HTML tag
<\/?[a-zA-Z][\w-]*(\s[^>]*)?>
# HTML comment
<!--[\s\S]*?-->
# inline JavaScript handler: onclick=, onload=, ...
\bon\w+\s*=\s*["'][^"']*["']
# content between double quotes (non-greedy)
"([^"\\]*(\\.[^"\\]*)*)"
# an @mention or #hashtag
[@#][\w-]+
# common log line: IP, timestamp, method, path, status
^(\S+) \S+ \S+ \[([^\]]+)\] "(\w+) ([^"]*)" (\d{3})
```
## 🧰 Handy Tricks
```regex
# a word repeated later in the same text
(\b\w+\b)(?=.*\b\1\b)
# two consecutive identical words
\b(\w+)\s+\1\b
# a line that does NOT contain a word
^(?!.*forbidden).*$
# everything between two markers, non-greedy
BEGIN([\s\S]*?)END
# strip ANSI colour codes
\x1B\[[0-9;]*[A-Za-z]
# split CSV, respecting quoted fields
("([^"]|"")*"|[^,]*)(,|$)
```
## ⚠️ Notes
- **Postal codes have no universal pattern** — every country differs. Look up the one you need.
- **Credit cards**: check the shape, then verify with the Luhn algorithm; brand prefixes are documented at [regular-expressions.info](https://www.regular-expressions.info/creditcard.html).
- **National identifiers** (SSN, passport, tax numbers) carry checksums and legal edge cases a regex cannot express.
- **Watch out for catastrophic backtracking** in patterns with nested quantifiers — test any user-facing regex against long, nearly-matching input.
## 📚 Resources
- [regex101 — test and explain](https://regex101.com)
- [Regular-Expressions.info](https://www.regular-expressions.info/)
- [RegExHub — common patterns](https://projects.lukehaas.me/regexhub/)
- [MDN — regular expressions](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Guide/Regular_expressions)
+147 -63
View File
@@ -1,63 +1,147 @@
| Selector | Example | Example description |
|-----------------------|-------------------------|----------------------------------------------------------------------------------------------------------|
| `.class` | `.intro` | Selects all elements with class="intro" |
| `.class1.class2` | `.name1.name2` | Selects all elements with both name1 and name2 set within its class attribute |
| `.class1 .class2` | `.name1 .name2` | Selects all elements with name2 that is a descendant of an element with name1 |
| `#id` | `#firstname` | Selects the element with id="firstname" |
| `*` | `*` | Selects all elements |
| `element` | `p` | Selects all \<p\> elements |
| `element.class` | `p.intro` | Selects all \<p\> elements with class="intro" |
| `element,element` | `div, p` | Selects all \<div\> elements and all \<p\> elements |
| `element element` | `div p` | Selects all \<p\> elements inside \<div\> elements |
| `element>element` | `div > p` | Selects all \<p\> elements where the parent is a \<div\> element |
| `element+element` | `div + p` | Selects the first \<p\> element that is placed immediately after \<div\> elements |
| `element1~element2` | `p ~ ul` | Selects every \<ul\> element that is preceded by a \<p\> element |
| `[attribute]` | `[target]` | Selects all elements with a target attribute |
| `[attribute=value]` | `[target="_blank"]` | Selects all elements with target="_blank" |
| `[attribute~=value]` | `[title~="flower"]` | Selects all elements with a title attribute containing the word "flower" |
| `[attribute\|=value]` | `[lang\|="en"]` | Selects all elements with a lang attribute value equal to "en" or starting with "en-" |
| `[attribute^=value]` | `a[href^="https"]` | Selects every \<a\> element whose href attribute value begins with "https" |
| `[attribute$=value]` | `a[href$=".pdf"]` | Selects every \<a\> element whose href attribute value ends with ".pdf" |
| `[attribute*=value]` | `a[href*="w3schools"]` | Selects every \<a\> element whose href attribute value contains the substring "w3schools" |
| `:active` | `a:active` | Selects the active link |
| `::after` | `p::after` | Insert something after the content of each \<p\> element |
| `::before` | `p::before` | Insert something before the content of each \<p\> element |
| `:checked` | `input:checked` | Selects every checked \<input\> element |
| `:default` | `input:default` | Selects the default \<input\> element |
| `:disabled` | `input:disabled` | Selects every disabled \<input\> element |
| `:empty` | `p:empty` | Selects every \<p\> element that has no children (including text nodes) |
| `:enabled` | `input:enabled` | Selects every enabled \<input\> element |
| `:first-child` | `p:first-child` | Selects every \<p\> element that is the first child of its parent |
| `::first-letter` | `p::first-letter` | Selects the first letter of every \<p\> element |
| `::first-line` | `p::first-line` | Selects the first line of every \<p\> element |
| `:first-of-type` | `p:first-of-type` | Selects every \<p\> element that is the first \<p\> element of its parent |
| `:focus` | `input:focus` | Selects the input element which has focus |
| `:fullscreen` | `:fullscreen` | Selects the element that is in full-screen mode |
| `:has()` | `h2:has(+p)` | Selects h2 elements that are immediately followed by a p element, and applies the style to h2 |
| `:hover` | `a:hover` | Selects links on mouse over |
| `:in-range` | `input:in-range` | Selects input elements with a value within a specified range |
| `:indeterminate` | `input:indeterminate` | Selects input elements that are in an indeterminate state |
| `:invalid` | `input:invalid` | Selects all input elements with an invalid value |
| `:lang()` | `p:lang(it)` | Selects every \<p\> element with a lang attribute equal to "it" (Italian) |
| `:last-child` | `p:last-child` | Selects every \<p\> element that is the last child of its parent |
| `:last-of-type` | `p:last-of-type` | Selects every \<p\> element that is the last \<p\> element of its parent |
| `:link` | `a:link` | Selects all unvisited links |
| `::marker` | `::marker` | Selects the markers of list items |
| `:not()` | `:not(p)` | Selects every element that is not a \<p\> element |
| `:nth-child()` | `p:nth-child(2)` | Selects every \<p\> element that is the second child of its parent |
| `:nth-last-child()` | `p:nth-last-child(2)` | Selects every \<p\> element that is the second child of its parent, counting from the last child |
| `:nth-last-of-type()` | `p:nth-last-of-type(2)` | Selects every \<p\> element that is the second \<p\> element of its parent, counting from the last child |
| `:nth-of-type()` | `p:nth-of-type(2)` | Selects every \<p\> element that is the second \<p\> element of its parent |
| `:only-of-type` | `p:only-of-type` | Selects every \<p\> element that is the only \<p\> element of its parent |
| `:only-child` | `p:only-child` | Selects every \<p\> element that is the only child of its parent |
| `:optional` | `input:optional` | Selects input elements with no "required" attribute |
| `:out-of-range` | `input:out-of-range` | Selects input elements with a value outside a specified range |
| `::placeholder` | `input::placeholder` | Selects input elements with the "placeholder" attribute specified |
| `:read-only` | `input:read-only` | Selects input elements with the "readonly" attribute specified |
| `:read-write` | `input:read-write` | Selects input elements with the "readonly" attribute NOT specified |
| `:required` | `input:required` | Selects input elements with the "required" attribute specified |
| `:root` | `:root` | Selects the document's root element |
| `::selection` | `::selection` | Selects the portion of an element that is selected by a user |
| `:target` | `#news:target` | Selects the current active #news element (clicked on a URL containing that anchor name) |
| `:valid` | `input:valid` | Selects all input elements with a valid value |
| `:visited` | `a:visited` | Selects all visited links |
**CSS selectors** pick the elements a rule applies to. They are also what `document.querySelectorAll()`, Playwright, Puppeteer and most scrapers use, so the same syntax works far beyond stylesheets.
```css
/* every <a> inside a .card that opens in a new tab */
.card a[target="_blank"] { color: rebeccapurple; }
```
## 🎯 Basic Selectors
| Selector | Example | Selects |
|------------------|----------------|--------------------------------------------|
| `*` | `*` | Every element |
| `element` | `p` | Every `<p>` element |
| `.class` | `.intro` | Every element with `class="intro"` |
| `#id` | `#firstname` | The element with `id="firstname"` |
| `.class1.class2` | `.name1.name2` | Elements carrying **both** classes |
| `element.class` | `p.intro` | Every `<p>` with `class="intro"` |
| `a, b` | `div, p` | Every `<div>` **and** every `<p>` (a list) |
## 🔗 Combinators
| Selector | Example | Selects |
|----------|-----------|---------------------------------------------------------------------------|
| `a b` | `div p` | Every `<p>` anywhere inside a `<div>` (descendant) |
| `a > b` | `div > p` | Every `<p>` whose direct parent is a `<div>` (child) |
| `a + b` | `div + p` | The `<p>` immediately after a `<div>` (adjacent sibling) |
| `a ~ b` | `p ~ ul` | Every `<ul>` that follows a `<p>` under the same parent (general sibling) |
## 🏷 Attribute Selectors
| Selector | Example | Selects |
|--------------------|---------------------|--------------------------------------------------------------------|
| `[attr]` | `[target]` | Elements that have the attribute at all |
| `[attr="value"]` | `[target="_blank"]` | Exact value |
| `[attr~="value"]` | `[title~="flower"]` | Value is one of a space-separated list |
| `[attr\|="value"]` | `[lang\|="en"]` | Value is `en` or starts with `en-` |
| `[attr^="value"]` | `a[href^="https"]` | Value starts with |
| `[attr$="value"]` | `a[href$=".pdf"]` | Value ends with |
| `[attr*="value"]` | `a[href*="docs"]` | Value contains |
| `[attr="value" i]` | `[type="TEXT" i]` | Case-insensitive match (add `s` instead to force case-sensitivity) |
## 🌳 Structural Pseudo-Classes
| Selector | Example | Selects |
|------------------------|-------------------------|----------------------------------------------------------|
| `:root` | `:root` | The document root — where custom properties usually live |
| `:first-child` | `p:first-child` | A `<p>` that is the first child of its parent |
| `:last-child` | `p:last-child` | A `<p>` that is the last child of its parent |
| `:only-child` | `p:only-child` | A `<p>` that is the only child |
| `:nth-child(n)` | `li:nth-child(2)` | The second child — also `odd`, `even`, `3n+1` |
| `:nth-last-child(n)` | `li:nth-last-child(2)` | Counting from the end |
| `:first-of-type` | `p:first-of-type` | The first `<p>` among its siblings |
| `:last-of-type` | `p:last-of-type` | The last `<p>` among its siblings |
| `:only-of-type` | `p:only-of-type` | The only `<p>` among its siblings |
| `:nth-of-type(n)` | `p:nth-of-type(2)` | The second `<p>` among its siblings |
| `:nth-last-of-type(n)` | `p:nth-last-of-type(2)` | Same, counting from the end |
| `:empty` | `p:empty` | No children at all, text included |
## 🖱 State & Interaction
| Selector | Example | Selects |
|------------------|------------------------|------------------------------------------------------------------|
| `:link` | `a:link` | Unvisited links |
| `:visited` | `a:visited` | Visited links |
| `:hover` | `a:hover` | The element under the pointer |
| `:active` | `a:active` | The element being clicked |
| `:focus` | `input:focus` | The focused element |
| `:focus-visible` | `button:focus-visible` | Focused *and* the browser thinks a ring belongs there (keyboard) |
| `:focus-within` | `form:focus-within` | An element containing the focused element |
| `:target` | `#news:target` | The element the URL fragment points at |
| `:fullscreen` | `:fullscreen` | The element displayed full-screen |
## 📝 Form Pseudo-Classes
| Selector | Example | Selects |
|----------------------|---------------------------|-------------------------------------------------|
| `:checked` | `input:checked` | Checked checkboxes, radios and selected options |
| `:default` | `input:default` | The default control in a form |
| `:disabled` | `input:disabled` | Disabled controls |
| `:enabled` | `input:enabled` | Controls that are not disabled |
| `:required` | `input:required` | Controls carrying `required` |
| `:optional` | `input:optional` | Controls without `required` |
| `:valid` | `input:valid` | Controls whose value passes validation |
| `:invalid` | `input:invalid` | Controls whose value fails validation |
| `:user-valid` | `input:user-valid` | Valid, but only after the user has interacted |
| `:user-invalid` | `input:user-invalid` | Invalid, but only after the user has interacted |
| `:in-range` | `input:in-range` | Value inside `min`/`max` |
| `:out-of-range` | `input:out-of-range` | Value outside `min`/`max` |
| `:indeterminate` | `input:indeterminate` | Neither checked nor unchecked |
| `:read-only` | `input:read-only` | Not editable by the user |
| `:read-write` | `input:read-write` | Editable by the user |
| `:placeholder-shown` | `input:placeholder-shown` | The placeholder is currently visible |
## 🧩 Functional Pseudo-Classes
| Selector | Example | Selects |
|-------------------------|--------------------------------|-------------------------------------------------------------------------|
| `:is()` | `:is(h1, h2, h3) > code` | Any of the listed selectors — takes the highest specificity of the list |
| `:where()` | `:where(h1, h2) a` | Same matching as `:is()`, but contributes **zero** specificity |
| `:not()` | `li:not(.done, .hidden)` | Elements that match none of the listed selectors |
| `:has()` | `article:has(> img)` | A parent that contains a match — the "parent selector" |
| `:lang()` | `p:lang(it)` | Elements in a given language |
| `:nth-child(An+B of S)` | `li:nth-child(2n of .visible)` | Counts only the siblings matching `S` |
## ✨ Pseudo-Elements
Pseudo-elements use `::` (double colon) and target a *part* of an element rather than the element itself.
| Selector | Example | Targets |
|--------------------------|-------------------------------|-------------------------------------------------|
| `::before` | `p::before` | Generated content in front of the content |
| `::after` | `p::after` | Generated content after the content |
| `::first-letter` | `p::first-letter` | The first letter |
| `::first-line` | `p::first-line` | The first rendered line |
| `::marker` | `li::marker` | The bullet or number of a list item |
| `::selection` | `::selection` | The part the user has selected |
| `::placeholder` | `input::placeholder` | The placeholder text |
| `::backdrop` | `dialog::backdrop` | The layer behind a modal or full-screen element |
| `::file-selector-button` | `input::file-selector-button` | The button of a file input |
> ℹ️ `::before` and `::after` need a `content` property — even `content: ""` — before they render.
## 📊 Specificity
Specificity is compared as three numbers, **ID – class – type**, highest wins; a later rule only wins against an equal score.
| Selector kind | Weight | Example |
|--------------------------------|----------------------------|-----------------------------|
| Inline `style` attribute | overrides everything below | `style="…"` |
| ID | `1-0-0` | `#header` |
| Class, attribute, pseudo-class | `0-1-0` | `.card`, `[href]`, `:hover` |
| Element, pseudo-element | `0-0-1` | `p`, `::before` |
| `*`, combinators, `:where()` | `0-0-0` | `*`, `>`, `:where(.a)` |
```css
#nav .item a /* 1-1-1 */
.menu .item a /* 0-2-1 — loses to the line above */
:where(#nav) a /* 0-0-1 — :where() zeroes out what is inside it */
```
> 💡 Reach for `:where()` in library or reset styles so consumers can override them without an `!important` arms race.
## 📚 Resources
- [MDN — CSS selectors](https://developer.mozilla.org/en-US/docs/Web/CSS/CSS_selectors)
- [MDN — specificity](https://developer.mozilla.org/en-US/docs/Web/CSS/Specificity)
- [Selectors Level 4 specification](https://www.w3.org/TR/selectors-4/)
+414 -382
View File
@@ -1,463 +1,495 @@
Docker Compose is a powerful tool for defining and running multi-container Docker applications. It uses a YAML file (`compose.yaml`) to configure application services, networks, volumes, and more. This file allows developers to declaratively describe infrastructure and dependencies, making it easier to manage complex environments.
**Docker Compose** describes a multi-container application in one YAML file and runs it with a single command. The file declares *services* (containers), plus the *networks*, *volumes*, *secrets* and *configs* they use.
Whether you're building a local dev stack or deploying to production, Compose simplifies orchestration and keeps your configuration readable and version-controlled.
> ℹ️ The `version:` key at the top is obsolete — the Compose Spec ignores it. Start the file with `services:`.
## 📁 File Name
```yaml
compose.yaml
## 📁 File Names & Precedence
| File | Role |
|-------------------------|--------------------------------------------------------------|
| `compose.yaml` | The preferred name (`compose.yml` also works) |
| `docker-compose.yaml` | Legacy name, still supported |
| `compose.override.yaml` | Merged on top of the base file automatically |
| `-f a.yaml -f b.yaml` | Explicit list; later files override earlier ones |
| `.env` | Variables for interpolation, read from the project directory |
```bash
# check what Compose actually resolved, after merges and interpolation
docker compose config
# use a specific set of files
docker compose -f compose.yaml -f compose.prod.yaml up -d
```
### ✅ YAML Formatting Rules
### YAML rules that bite
- Use **2 spaces** for indentation (not tabs)
- Keys and values are **case-sensitive**
- Lists use `-` for each item
- Strings with special characters should be quoted
- Environment variables can be defined inline or via `.env` files
- Two spaces per indent level, never tabs
- Keys and values are case-sensitive
- Lists use `-`; quote any string containing `:`, `#`, `{`, `}` or a leading `*`
- `yes`/`no`/`on`/`off` are booleans — quote them if you mean the words
## 🧱 Basic Structure
```yaml
services:
service_name:
image: image_name:tag
build:
context: .
dockerfile: Dockerfile
ports:
- "host_port:container_port"
volumes:
- ./host_path:/container_path
environment:
- VAR_NAME=value
depends_on:
- other_service
networks:
- custom_network
networks:
custom_network:
driver: bridge
volumes:
custom_volume:
```
## ⚙️ Services
Each service defines a container.
### Common Service Options
## 🧱 Minimal Structure
```yaml
services:
web:
image: nginx:latest
build:
context: ./app
dockerfile: Dockerfile
command: ["nginx", "-g", "daemon off;"]
container_name: custom_name
image: nginx:1.27
ports:
- "8080:80"
depends_on:
- api
api:
build: .
environment:
DATABASE_URL: postgres://db:5432/app
networks:
- backend
networks:
backend:
volumes:
db-data:
```
## ⚙️ Service Options
```yaml
services:
app:
# where the image comes from
image: myapp:1.2
build: .
pull_policy: always
# identity and lifecycle
container_name: myapp
hostname: app
restart: unless-stopped
init: true
stop_grace_period: 30s
# what it runs
entrypoint: ["/entrypoint.sh"]
command: ["node", "server.js"]
working_dir: /app
user: "1000:1000"
# configuration
environment:
NODE_ENV: production
API_KEY: ${API_KEY}
env_file:
- .env
- .env.production
# connectivity
ports:
- "8080:80"
expose:
- "80"
environment:
- DEBUG=true
env_file:
- .env
volumes:
- ./data:/data
restart: always
depends_on:
- db
- "9000"
networks:
- frontend
extra_hosts:
- "host.docker.internal:host-gateway"
dns:
- 1.1.1.1
# storage
volumes:
- db-data:/var/lib/data
- ./src:/app/src:ro
tmpfs:
- /tmp
# ordering and health
depends_on:
db:
condition: service_healthy
healthcheck:
test: ["CMD", "curl", "-f", "http://localhost"]
test: ["CMD", "curl", "-f", "http://localhost/health"]
interval: 30s
timeout: 10s
retries: 5
timeout: 5s
retries: 3
start_period: 20s
# host access and limits
cap_add:
- SYS_PTRACE
devices:
- /dev/dri:/dev/dri
ulimits:
nofile: 65535
deploy:
resources:
limits:
cpus: "1.5"
memory: 512M
# bookkeeping
labels:
com.example.team: platform
logging:
driver: json-file
options:
max-size: "10m"
max-file: "3"
```
## 🏗️ Build Options
## 🏗 Build Options
```yaml
build:
context: ./dir
dockerfile: Dockerfile
args:
build_arg: value
target: build-stage
services:
app:
build:
context: .
dockerfile: docker/Dockerfile
target: production
args:
NODE_VERSION: "22"
cache_from:
- myapp:cache
secrets:
- npmrc
platforms:
- linux/amd64
- linux/arm64
image: myapp:1.2 # the name given to the built image
```
- **`build:`** Tells Compose how to build the image.
- `context:` is the directory containing the Dockerfile and source code.
- `dockerfile:` lets you specify a custom Dockerfile name or path.
## 📦 Volumes
```yaml
services:
db:
volumes:
# named volume — managed by Docker, survives 'down'
- db-data:/var/lib/postgresql/data
# bind mount — a host path, read-only
- ./config:/etc/app:ro
# anonymous volume — keeps node_modules out of the bind mount above
- /app/node_modules
# long syntax
- type: bind
source: ./src
target: /app/src
read_only: true
volumes:
data_volume:
driver: local
db-data:
shared:
external: true # created outside Compose
nfs-data:
driver_opts:
type: none
device: /path/on/host
o: bind
type: nfs
o: addr=10.0.0.10,rw
device: ":/exports/data"
```
### Mounting Volumes
```yaml
volumes:
- data_volume:/app/data
- ./local:/container/path
```
- **`volumes:`** Mounts host directories or named volumes into the container.
- `./src:/app/src` mounts the local `src` folder into the container at `/app/src`.
## 🌐 Networks
```yaml
services:
web:
networks:
- frontend
api:
networks:
frontend:
aliases:
- api.internal
backend:
networks:
frontend:
driver: bridge
backend:
driver: overlay
internal: true # no outbound access
existing:
external: true
name: some-other-network
```
- **`networks:`** Connects the service to one or more custom networks. Enables service discovery and isolation.
> 💡 Compose creates a default network per project and every service joins it, so containers already reach each other by service name. Declare networks when you want to *separate* things.
### Assigning Networks to Services
## 🔀 Ports
```yaml
services:
app:
networks:
- frontend
- backend
```
## 🌐 Ports
```yaml
web:
ports:
- "3000:3000"
```
- **`ports:`** Maps container ports to host ports. Format is `"host:container"`. Useful for exposing services to your local machine.
## 🔐 Secrets (Docker Swarm only)
```yaml
secrets:
db_password:
file: ./db_password.txt
services:
db:
secrets:
- db_password
```
## 🔑 Configs (Docker Swarm only)
```yaml
configs:
my_config:
file: ./config.txt
services:
app:
configs:
- source: my_config
target: /etc/config.txt
```
## 🧪 Healthcheck
```yaml
healthcheck:
test: ["CMD", "curl", "-f", "http://localhost"]
interval: 30s
timeout: 10s
retries: 3
start_period: 5s
```
- **`healthcheck:`** Defines how Docker checks if the container is healthy.
- `test:` is the command to run
- `interval:` how often to run the check
- `timeout:` how long to wait for a response
- `retries:` how many failures before marking unhealthy
## 🔄 Restart Policies
```yaml
restart: no # Never restart
restart: always # Always restart
restart: on-failure # Restart on failure
restart: unless-stopped
- "8080:80" # host:container
- "127.0.0.1:8080:80" # bind to one interface only
- "8080-8090:80-90" # a range
- "80" # random host port
- target: 80 # long syntax
published: "8080"
protocol: tcp
mode: host
```
## 🧬 Environment Variables
```yaml
environment:
- VAR1=value1
- VAR2=value2
env_file:
- .env
services:
app:
environment:
# map form (preferred)
LOG_LEVEL: debug
# take the value from the shell or .env
API_KEY: ${API_KEY}
# with a default, and a hard requirement
PORT: ${PORT:-3000}
DB_URL: ${DB_URL:?DB_URL must be set}
env_file:
- path: .env.production
required: false
```
- **`environment:`** Sets environment variables inside the container. Useful for configuration.
- **`env_file:`** Loads environment variables from a file. Keeps secrets and config separate from the Compose file.
## Command
```yaml
command: npm start
```
- **`command:`** Overrides the default command defined in the Dockerfile. Useful for customizing container behavior.
## Dependencies
```yaml
depends_on:
- db
```
- **`depends_on:`** Specifies service startup order. In Compose, this does not wait for the service to be "ready"—just started.
## 🧹 Clean Up
```bash
docker compose down # Stop and remove containers, networks, volumes
docker compose down -v # Also remove named volumes
# variables come from the shell, .env, and --env-file, in that order of precedence
API_KEY=abc docker compose up -d
docker compose --env-file .env.staging up -d
```
## 🚀 Commands
## 🩺 Healthchecks & Startup Order
| Command | Description |
|--------|-------------|
| `docker compose up` | Start services |
| `docker compose up -d` | Start in detached mode |
| `docker compose down` | Stop and remove services |
| `docker compose build` | Build images |
| `docker compose ps` | List containers |
| `docker compose logs` | View logs |
| `docker compose exec <service> <cmd>` | Run command in container |
| `docker compose config` | Validate and view config |
## 🧠 Mounting GPU / iGPU
Docker Compose supports GPU access via the `device_requests` field (Compose v3.8+ and Docker 19.03+).
### ✅ NVIDIA GPU Example
`depends_on` on its own only waits for the container to *start*. Wait for it to be **healthy** instead:
```yaml
services:
gpu-app:
image: nvidia/cuda:11.0-base
db:
image: postgres:16
healthcheck:
test: ["CMD-SHELL", "pg_isready -U postgres"]
interval: 10s
timeout: 5s
retries: 5
start_period: 30s
api:
image: myapi
depends_on:
db:
condition: service_healthy # or service_started, service_completed_successfully
```
## 🔄 Restart Policies
| Policy | Behaviour |
|------------------|---------------------------------------------------------------|
| `no` | Never restart (the default) |
| `on-failure` | Restart only on a non-zero exit — `on-failure:5` to cap tries |
| `always` | Always restart, including after a daemon restart |
| `unless-stopped` | Like `always`, but stays stopped if you stopped it yourself |
## 🧩 Profiles
Profiles keep optional services out of the way until you ask for them.
```yaml
services:
web:
image: nginx # no profile: always started
debug:
image: busybox
command: top
profiles: [debug]
seed:
image: myapp
command: npm run seed
profiles: [tools]
```
```bash
# start the default services plus one profile
docker compose --profile debug up -d
# several at once
docker compose --profile debug --profile tools up -d
# COMPOSE_PROFILES works too
COMPOSE_PROFILES=debug,tools docker compose up -d
```
## 👀 Watch Mode
`docker compose watch` syncs or rebuilds automatically as you edit — a dev loop without bind-mount surprises.
```yaml
services:
web:
build: .
develop:
watch:
- action: sync
path: ./src
target: /app/src
- action: rebuild
path: package.json
- action: sync+restart
path: ./config
target: /etc/app
```
```bash
docker compose watch
```
## ♻️ Reuse: include, extends & anchors
```yaml
# pull in another compose file as if it were written here
include:
- path: ./monitoring/compose.yaml
services:
# inherit another service's definition
worker:
extends:
file: common.yaml
service: base-app
command: ["node", "worker.js"]
# YAML anchors for repeated blocks
x-logging: &default-logging
driver: json-file
options:
max-size: "10m"
services:
api:
logging: *default-logging
web:
logging: *default-logging
```
## 🔐 Secrets & Configs
File-based secrets work in plain Compose: each one is mounted read-only at `/run/secrets/<name>`. External secrets and configs require Swarm.
```yaml
services:
db:
image: postgres:16
environment:
POSTGRES_PASSWORD_FILE: /run/secrets/db_password
secrets:
- db_password
configs:
- source: pg_conf
target: /etc/postgresql/postgresql.conf
secrets:
db_password:
file: ./db_password.txt
api_token:
external: true # Swarm only
configs:
pg_conf:
file: ./postgresql.conf
```
## 🖥 GPU & Device Access
```yaml
services:
# NVIDIA — needs the NVIDIA Container Toolkit on the host
cuda-app:
image: nvidia/cuda:12.4.1-base-ubuntu22.04
deploy:
resources:
reservations:
devices:
- driver: nvidia
count: all
count: all # or: device_ids: ["0"]
capabilities: [gpu]
```
### ✅ Intel iGPU (via VAAPI or OpenCL)
```yaml
services:
# Intel iGPU — VAAPI/OpenCL through the render device
igpu-app:
image: intel/openvino
devices:
- /dev/dri:/dev/dri
group_add:
- video
```
> 🔧 Make sure your host has the necessary drivers and runtime (e.g., NVIDIA Container Toolkit or Intel Media SDK).
## 🔄 Compose vs Swarm YAML Differences
| Feature | Docker Compose (`compose.yaml`) | Docker Swarm (`stack.yml`) |
|--------------------|----------------------------------------|-----------------------------|
| `restart` | ✅ Supported | ❌ Not supported |
| `depends_on` | ✅ Supported | ❌ Not supported |
| `deploy` | ❌ Ignored | ✅ Required for replicas |
| `build` | ✅ Supported | ❌ Not supported |
| `volumes` (bind) | ✅ Supported | ✅ Supported |
| `configs`/`secrets`| ❌ Ignored | ✅ Supported |
| `healthcheck` | ✅ Supported | ✅ Supported |
> 🧠 **Tip:** Use `compose.yaml` for local development and `stack.yml` for Swarm deployments.
## 🧬 Profiles (Compose v3.9+)
Profiles allow conditional inclusion of services based on the active profile. This is useful for separating dev/test/staging environments.
### ✅ Defining Profiles
```yaml
services:
web:
image: nginx
profiles:
- default
debug:
image: busybox
command: top
profiles:
- debug
```
### ✅ Activating Profiles
## 🚀 Everyday Commands
```bash
docker compose --profile debug up
# start everything in the background
docker compose up -d
# rebuild the images first
docker compose up -d --build
# start one service and what it depends on
docker compose up -d <service>
# recreate containers even if nothing changed
docker compose up -d --force-recreate
# stop and remove containers and networks
docker compose down
# also remove the named volumes and local images
docker compose down -v --rmi local
# what is running, including health
docker compose ps
# follow the logs
docker compose logs -f
docker compose logs -f --tail 100 <service>
# a shell inside a running service
docker compose exec <service> sh
# a one-off container for a task
docker compose run --rm <service> npm test
# rebuild, pull, restart
docker compose build --no-cache
docker compose pull
docker compose restart <service>
# scale a stateless service
docker compose up -d --scale worker=3
# validate and print the effective configuration
docker compose config
docker compose config --services
# processes and resource usage
docker compose top
docker compose stats
```
### ✅ Notes
## ⚔️ Compose vs Swarm Stacks
- Services without a `profiles` key are always included.
- Multiple profiles can be activated simultaneously.
- Useful for feature toggles, optional services, or environment-specific setups.
Both read a Compose file, but `docker stack deploy` honours a different subset of it.
## ⚔️ YAML Differences: Docker Compose vs Docker Swarm Mode
| Key | `docker compose` | `docker stack deploy` |
|----------------------------------|-----------------------------|-------------------------------------------------|
| `build:` | ✅ builds locally | ❌ ignored — push the image to a registry first |
| `restart:` | ✅ | ❌ use `deploy.restart_policy` |
| `depends_on:` | ✅ | ❌ ignored — rely on health checks and retries |
| `profiles:` | ✅ | ❌ |
| `develop.watch:` | ✅ | ❌ |
| `deploy.replicas/placement` | ❌ ignored | ✅ this is how you scale |
| `configs:` / external `secrets:` | ❌ external ones need Swarm | ✅ |
| `healthcheck:` | ✅ | ✅ also drives rescheduling |
| `volumes:` / `networks:` | ✅ | ✅ (overlay networks in Swarm) |
Docker Compose and Docker Swarm both use YAML files to define services, but they serve different purposes and support different features. Compose is optimized for local development and testing, while Swarm is designed for production-grade orchestration across clusters.
### 🧭 Purpose
| Mode | Use Case |
|-------------|-----------------------------------|
| Compose | Local development, testing |
| Swarm | Cluster deployment, scaling |
### 🧩 Key Differences in YAML Structure
| Feature | Compose (`compose.yaml`) | Swarm (`stack.yml`) |
|----------------------|-------------------------------|----------------------|
| `build:` | ✅ Supported | ❌ Ignored |
| `restart:` | ✅ Supported | ❌ Ignored |
| `depends_on:` | ✅ Supported | ❌ Ignored |
| `deploy:` | ❌ Ignored | ✅ Required for scaling, placement |
| `configs:` | ❌ Ignored | ✅ Supported |
| `secrets:` | ❌ Ignored | ✅ Supported |
| `healthcheck:` | ✅ Supported | ✅ Supported |
| `volumes:` (bind) | ✅ Supported | ✅ Supported |
| `networks:` | ✅ Supported | ✅ Supported |
| `profiles:` | ✅ Supported (v3.9+) | ❌ Not supported |
### 🔧 Compose-Only Features
These features are useful for local development but are ignored in Swarm:
#### `build:`
```yaml
services:
app:
build:
context: .
dockerfile: Dockerfile
```
- Compose builds images locally.
- Swarm requires pre-built images pushed to a registry.
#### `restart:`
```yaml
restart: unless-stopped
```
- Compose uses this to auto-restart containers.
- Swarm uses `deploy.restart_policy` instead.
#### `depends_on:`
```yaml
depends_on:
- db
```
- Compose starts services in order.
- Swarm ignores this; use healthchecks and wait-for-it scripts.
### 🛡️ Swarm-Only Features
These features are exclusive to Swarm and ignored by Compose:
#### `deploy:`
```yaml
services:
app:
deploy:
replicas: 3
placement:
constraints:
- node.role == manager
restart_policy:
condition: on-failure
```
- Controls scaling, placement, and restart behavior in a cluster.
#### `configs:` and `secrets:`
```yaml
configs:
app_config:
file: ./config.yml
secrets:
db_password:
file: ./password.txt
```
- Used to securely distribute configuration and secrets across nodes.
#### `placement:` (inside `deploy`)
```yaml
placement:
constraints:
- node.labels.env == production
```
- Assigns services to specific nodes based on labels.
### 🧪 Healthchecks (Supported in Both)
```yaml
healthcheck:
test: ["CMD", "curl", "-f", "http://localhost"]
interval: 30s
timeout: 10s
retries: 3
```
- Works in both Compose and Swarm.
- In Swarm, health status can influence service rescheduling.
### 📦 Volume Differences
| Type | Compose | Swarm |
|-------------|---------|-------|
| Bind mount | ✅ | ✅ |
| Named volume| ✅ | ✅ |
| External volume | ✅ | ✅ |
| Volume driver options | ✅ | ✅ |
Swarm requires external volumes to be pre-created on all nodes.
### 🧠 Summary
| Feature Category | Compose | Swarm |
|----------------------|---------|-------|
| Local builds | ✅ | ❌ |
| Cluster scaling | ❌ | ✅ |
| Secrets/configs | ❌ | ✅ |
| Profiles | ✅ | ❌ |
| Restart policies | ✅ | ✅ (via `deploy`) |
| Service dependencies | ✅ | ❌ |
> 🧭 **Tip:** Use `compose.yaml` for development and `stack.yml` for Swarm. You can split your configuration into multiple files or use tools like `kompose` to convert Kubernetes manifests.
> 🧭 Keep `compose.yaml` for local development and a separate `stack.yaml` for Swarm, or put the Swarm-only bits in an override file.
## 📚 Resources
- [Compose File Reference](https://docs.docker.com/compose/compose-file/)
- [Docker CLI Reference](https://docs.docker.com/engine/reference/commandline/compose/)
- [NVIDIA GPU Support](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/install-guide.html)
- [Intel GPU Support](https://github.com/intel/media-driver)
- [Compose file reference](https://docs.docker.com/reference/compose-file/)
- [`docker compose` CLI reference](https://docs.docker.com/reference/cli/docker/compose/)
- [Compose watch](https://docs.docker.com/compose/how-tos/file-watch/)
- [Environment variables and interpolation](https://docs.docker.com/compose/how-tos/environment-variables/)
- [NVIDIA Container Toolkit](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/install-guide.html)
+413 -356
View File
@@ -1,482 +1,539 @@
Docker provides the ability to package and run an application in a loosely isolated environment called a container. The isolation and security allows you to run many containers simultaneously on a given host. Containers are lightweight and contain everything needed to run the application, so you do not need to rely on what is currently installed on the host. You can easily share containers while you work, and be sure that everyone you share with gets the same container that works in the same way.
**Docker** packages an application and everything it needs into a *container* — a lightweight, isolated process that runs the same way on any host. Images are the immutable blueprints; containers are running instances of them.
## Installation & Resources
- **Image** — a read-only template built from a `Dockerfile`
- **Container** — a running (or stopped) instance of an image
- **Volume** — storage that outlives the container that wrote it
- **Network** — a virtual network where containers reach each other by name
- **Registry** — where images are pushed and pulled (Docker Hub, GHCR, ECR, …)
- Docker Desktop (Mac, Linux, Windows): https://docs.docker.com/desktop
- Example projects using Docker: https://github.com/docker/awesome-compose
- Official documentation: https://docs.docker.com
> 💡 Every subcommand has its own help: `docker run --help`, `docker compose up --help`.
## General Commands
## ⚡ Most-Used Commands
| Command | What it does |
|----------------------------------|--------------------------------------------|
| `docker ps -a` | List all containers, running or not |
| `docker images` | List local images |
| `docker run -it --rm <image> sh` | Throwaway shell in a fresh container |
| `docker exec -it <container> sh` | Shell inside a *running* container |
| `docker logs -f <container>` | Follow a container's output |
| `docker build -t <name>:<tag> .` | Build an image from the local `Dockerfile` |
| `docker compose up -d --build` | Rebuild and start a whole stack |
| `docker system df` | See what is eating your disk |
## 🧭 General
Start the Docker daemon:
```bash
# start the Docker daemon (usually done by the service manager)
dockerd
```
Get help with Docker (works on all subcommands, e.g. `docker run --help`):
```bash
# top-level help; works on every subcommand
docker --help
```
Display system-wide information:
```bash
# system-wide information: storage driver, resources, warnings
docker info
```
Show Docker version (client and server):
```bash
# client and server versions
docker version
# list contexts (local socket, remote hosts, ...)
docker context ls
# target another daemon with the same commands
docker context use <ctx>
# live stream of daemon events
docker events
```
## Images
## 📦 Images
Docker images are a lightweight, standalone, executable package of software that includes everything needed to run an application: code, runtime, system tools, system libraries, and settings.
An image is a stack of read-only layers: code, runtime, libraries and settings baked together.
Build an image from a Dockerfile in the current directory:
```bash
docker build -t <image_name> .
```
# build from the Dockerfile in this directory
docker build -t <image> .
Build an image without using the cache:
```bash
docker build -t <image_name> . --no-cache
```
# build and tag, e.g. myapp:1.2
docker build -t <image>:<tag> .
Build with a specific tag (e.g. `myapp:v1.2`):
```bash
docker build -t <image_name>:<tag> .
```
# rebuild every layer from scratch
docker build -t <image> . --no-cache
List local images:
```bash
# use a Dockerfile somewhere else
docker build -f <path>/Dockerfile -t <image> .
# pass a build argument
docker build --build-arg KEY=value -t <image> .
# stop at a stage of a multi-stage build
docker build --target <stage> -t <image> .
# build for another architecture
docker build --platform linux/amd64 -t <image> .
# list local images
docker images
```
Tag an image for pushing to a registry:
```bash
docker tag <image> <username>/<image>:<tag>
```
# include intermediate layers
docker images -a
Delete an image:
```bash
docker rmi <image_name>
```
# add a name for a registry
docker tag <image> <user>/<image>:<tag>
Remove dangling (unused) images:
```bash
# full metadata as JSON
docker inspect <image>
# layers, sizes and the commands behind them
docker history <image>
# delete an image
docker rmi <image>
# force-delete an image containers still reference
docker rmi -f <image>
# remove dangling images
docker image prune
```
Remove all unused images, not just dangling:
```bash
# remove every image no container uses
docker image prune -a
# export an image to a tarball
docker save -o <file>.tar <image>
# import an image from a tarball
docker load -i <file>.tar
```
Show the layers of an image:
```bash
docker history <image_name>
```
## Docker Hub
Docker Hub is a service for finding and sharing container images. Learn more at https://hub.docker.com
Log in to Docker Hub:
## 🐳 Registries & Docker Hub
[Docker Hub](https://hub.docker.com) is the default public registry; the same commands work against any other one.
```bash
# log in to Docker Hub
docker login -u <username>
```
Log out:
```bash
# log in to a private registry (GHCR, ECR, ...)
docker login <registry-url>
# drop the stored credentials
docker logout
# search Docker Hub from the terminal
docker search <term>
# pull the :latest tag
docker pull <image>
# pull a specific tag
docker pull <image>:<tag>
# pull for another architecture
docker pull --platform linux/arm64 <image>
# publish an image
docker push <user>/<image>:<tag>
# architectures available for a tag
docker manifest inspect <image>:<tag>
```
Search Docker Hub for an image:
## 🚢 Running Containers
```bash
docker search <image_name>
# create and start a container
docker run <image>
# give it a stable name
docker run --name <container> <image>
# detached: run in the background
docker run -d <image>
# interactive shell (sh on minimal images)
docker run -it <image> bash
# delete the container as soon as it exits
docker run --rm <image>
# publish a port, e.g. -p 8080:80
docker run -p <host>:<container> <image>
# publish every EXPOSEd port on random ports
docker run -P <image>
# set an environment variable
docker run -e KEY=value <image>
# load environment variables from a file
docker run --env-file ./.env <image>
# mount a named volume
docker run -v <volume>:/data <image>
# bind-mount the current directory
docker run -v $(pwd):/app <image>
# bind-mount the current directory read-only
docker run -v $(pwd):/app:ro <image>
# set the working directory
docker run -w /app <image> <command>
# run as your own UID/GID, not root
docker run -u $(id -u):$(id -g) <image>
# attach to a user-defined network
docker run --network <network> <image>
# restart policy, see the flags table below
docker run --restart unless-stopped <image>
# cap the resources it may use
docker run --memory 512m --cpus 1.5 <image>
# override the image entrypoint
docker run --entrypoint <cmd> <image>
# create without starting
docker create --name <container> <image>
```
Pull an image from Docker Hub:
```bash
docker pull <image_name>
```
Pull a specific tagged version:
```bash
docker pull <image_name>:<tag>
```
Publish an image to Docker Hub:
```bash
docker push <username>/<image_name>
```
## Containers
A container is a runtime instance of a Docker image. A container will always run the same, regardless of the infrastructure. Containers isolate software from its environment and ensure it works uniformly across development, staging, and production.
### Running containers
Create and run a container from an image:
```bash
docker run <image_name>
```
Run with a custom name:
```bash
docker run --name <container_name> <image_name>
```
Run in the background (detached):
```bash
docker run -d <image_name>
```
Run interactively with a shell:
```bash
docker run -it <image_name> bash
```
Auto-remove the container when it exits:
```bash
docker run --rm <image_name>
```
Publish a container's port to the host:
```bash
docker run -p <host_port>:<container_port> <image_name>
```
Set an environment variable:
```bash
docker run -e KEY=value <image_name>
```
Load environment variables from a file:
```bash
docker run --env-file ./env.list <image_name>
```
Mount a named volume:
```bash
docker run -v <volume_name>:/path/in/container <image_name>
```
Bind-mount the current directory:
```bash
docker run -v $(pwd):/app <image_name>
```
Set a restart policy (options: `no`, `on-failure`, `always`, `unless-stopped`):
```bash
docker run --restart unless-stopped <image_name>
```
Attach to a specific network:
```bash
docker run --network <network_name> <image_name>
```
### Managing containers
List currently running containers:
## 🎛 Managing Containers
```bash
# running containers
docker ps
```
List all containers (running and stopped):
```bash
# every container, running or exited
docker ps -a
```
Start a stopped container:
```bash
docker start <container_name>
```
# IDs only — handy for scripting
docker ps -q
Stop a running container:
```bash
docker stop <container_name>
```
# filter by status, name, label, ancestor, ...
docker ps --filter "status=exited"
Restart a container:
```bash
docker restart <container_name>
```
# start a stopped container
docker start <container>
Remove a stopped container:
```bash
docker rm <container_name>
```
# graceful stop (SIGTERM, then SIGKILL)
docker stop <container>
Force-remove a running container:
```bash
docker rm -f <container_name>
```
# stop and start again
docker restart <container>
Remove all stopped containers:
```bash
# immediate SIGKILL
docker kill <container>
# freeze all processes in the container
docker pause <container>
# resume them
docker unpause <container>
# remove a stopped container
docker rm <container>
# stop and remove in one go
docker rm -f <container>
# rename a container
docker rename <old> <new>
# change resource limits or restart policy live
docker update --restart=always <container>
# remove every stopped container
docker container prune
# block until it exits, then print its exit code
docker wait <container>
```
Rename a container:
```bash
docker rename <old_name> <new_name>
```
## 🔍 Inspecting & Debugging
### Inspecting and interacting
Open a shell inside a running container:
```bash
docker exec -it <container_name> bash
```
# shell inside a running container
docker exec -it <container> bash
Use `sh` for minimal images (e.g. alpine) that don't ship bash:
```bash
docker exec -it <container_name> sh
```
# for alpine/distroless-style images
docker exec -it <container> sh
Fetch logs:
```bash
docker logs <container_name>
```
# get in as root to install debug tools
docker exec -it -u root <container> sh
Follow logs (like `tail -f`):
```bash
docker logs -f <container_name>
```
# run a one-off command
docker exec <container> <command>
Show only the last 100 lines:
```bash
docker logs --tail 100 <container_name>
```
# print the container's output
docker logs <container>
Show low-level container details (JSON):
```bash
docker inspect <container_name>
```
# follow it, like tail -f
docker logs -f <container>
Live resource usage stats for all containers:
```bash
# last 100 lines, with timestamps
docker logs --tail 100 -t <container>
# only the last 10 minutes
docker logs --since 10m <container>
# low-level details as JSON
docker inspect <container>
# one field via a Go template
docker inspect -f '{{.State.Status}}' <container>
# the container's IP address on the default bridge
docker inspect -f '{{.NetworkSettings.IPAddress}}' <container>
# live CPU/memory/IO for all containers
docker stats
# processes running inside
docker top <container>
# published port mappings
docker port <container>
# filesystem changes since it started
docker diff <container>
# copy a file out of a container
docker cp <container>:/path/file ./
# copy a file into a container
docker cp ./file <container>:/path/
# snapshot a container as a new image
docker commit <container> <image>:<tag>
# attach to the main process (Ctrl-P Ctrl-Q detaches)
docker attach <container>
```
Show running processes inside a container:
## 💾 Volumes
Volumes keep data outside the container's writable layer, so it survives `docker rm` and image upgrades.
```bash
docker top <container_name>
```
# create a named volume
docker volume create <volume>
Copy a file out of a container:
```bash
docker cp <container_name>:/path/in/container ./
```
Copy a file into a container:
```bash
docker cp ./local_file <container_name>:/path/
```
Show filesystem changes since container start:
```bash
docker diff <container_name>
```
## Volumes
Volumes persist data outside the container's writable layer, so data survives container removal.
Create a named volume:
```bash
docker volume create <volume_name>
```
List volumes:
```bash
# list volumes
docker volume ls
```
Show details about a volume:
```bash
docker volume inspect <volume_name>
```
# driver, mount point, labels
docker volume inspect <volume>
Remove a volume:
```bash
docker volume rm <volume_name>
```
# delete a volume (and its data)
docker volume rm <volume>
Remove all unused volumes:
```bash
# delete every unused volume
docker volume prune
# named volume, managed by Docker
docker run -v <volume>:/data <image>
# bind mount from the host
docker run -v $(pwd):/app <image>
# explicit bind-mount syntax
docker run --mount type=bind,src=$(pwd),dst=/app <image>
# the same named volume, spelled out with --mount
docker run --mount type=volume,src=<volume>,dst=/data <image>
# in-memory scratch space
docker run --tmpfs /tmp <image>
```
Attach a named volume to a container (managed by Docker):
```bash
docker run -v <volume_name>:/data <image_name>
```
Bind-mount a host path into a container:
```bash
docker run -v $(pwd):/app <image_name>
```
Equivalent bind mount using the `--mount` syntax:
```bash
docker run --mount type=bind,src=$(pwd),dst=/app <image_name>
```
## Networks
Containers on the same network can reach each other by container name.
List networks:
> 💡 Back up a volume: `docker run --rm -v <volume>:/data -v $(pwd):/backup alpine tar czf /backup/backup.tar.gz -C /data .`
## 🌐 Networks
Containers on the same user-defined network resolve each other by container name.
```bash
# list networks
docker network ls
```
Create a network:
```bash
docker network create <network_name>
```
# create a bridge network
docker network create <network>
Show network details:
```bash
docker network inspect <network_name>
```
# create a network with a fixed subnet
docker network create --driver bridge --subnet 172.30.0.0/16 <network>
Attach a running container to a network:
```bash
docker network connect <network_name> <container_name>
```
# subnet and connected containers
docker network inspect <network>
Detach a container from a network:
```bash
docker network disconnect <network_name> <container_name>
```
# attach a running container
docker network connect <network> <container>
Remove a network:
```bash
docker network rm <network_name>
```
# detach it again
docker network disconnect <network> <container>
Remove all unused networks:
```bash
# delete a network
docker network rm <network>
# delete every unused network
docker network prune
# share the host network stack (Linux)
docker run --network host <image>
# no networking at all
docker run --network none <image>
```
## Docker Compose
## 🧩 Docker Compose
Compose runs multi-container apps from a single `compose.yaml` (or `docker-compose.yml`) file.
Compose describes a multi-container application in a single `compose.yaml` (or `docker-compose.yml`).
Start all services (foreground):
```bash
# start every service in the foreground
docker compose up
```
Start all services (detached):
```bash
# start detached
docker compose up -d
```
Rebuild images before starting:
```bash
docker compose up --build
```
# rebuild images first
docker compose up -d --build
Stop and remove containers and networks:
```bash
# start one service and its dependencies
docker compose up -d <service>
# stop and remove containers and networks
docker compose down
```
Also remove named volumes:
```bash
# stop, remove, and delete the named volumes too
docker compose down -v
```
List services in the project:
```bash
# status of the project's services
docker compose ps
```
Follow logs for all services:
```bash
# follow the logs of all services
docker compose logs -f
```
Follow logs for one service:
```bash
# follow the logs of a single service
docker compose logs -f <service>
```
Build (or rebuild) services:
```bash
# (re)build the service images
docker compose build
```
Pull service images:
```bash
# pull the service images
docker compose pull
```
Open a shell in a running service:
```bash
docker compose exec <service> bash
```
# shell inside a running service
docker compose exec <service> sh
Run a one-off command in a new container:
```bash
docker compose run --rm <service> <command>
```
# one-off command in a new container
docker compose run --rm <service> <cmd>
Restart a single service:
```bash
# restart one service
docker compose restart <service>
```
Validate and view the resolved compose file:
```bash
# stop the containers, keep them around
docker compose stop
# start them again
docker compose start
# validate and print the resolved configuration
docker compose config
# processes running in each service
docker compose top
# use a specific compose file
docker compose -f <file> up -d
# include services behind a profile
docker compose --profile <name> up -d
# rebuild/sync automatically on file changes
docker compose watch
```
## System & Cleanup
## 🧹 System & Cleanup
Disk usage adds up fast. These commands reclaim space.
Disk usage adds up fast — these are the commands that give the space back.
Show disk usage (images, containers, volumes, build cache):
```bash
# what images, containers, volumes and cache cost
docker system df
```
Remove stopped containers, dangling images, and unused networks:
```bash
# the same, itemised
docker system df -v
# stopped containers, dangling images, unused networks
docker system prune
```
Also remove all unused images (not just dangling):
```bash
# the same, plus every image no container uses
docker system prune -a
```
Also remove unused volumes (destructive — double-check before running):
```bash
# the same, plus unused volumes ⚠️ destructive
docker system prune -a --volumes
```
Clean the build cache:
```bash
# clear the build cache
docker builder prune
# images unused for more than a week
docker image prune -a --filter "until=168h"
```
## Quick Reference: Common Flags
> ⚠️ `--volumes` deletes data no running container is using. Check `docker volume ls` first.
| Flag | Meaning |
|------|---------|
| `-d` | Detached (run in background) |
| `-it` | Interactive + TTY (for shells) |
| `--rm` | Remove container when it exits |
| `-p host:container` | Publish a port |
| `-v src:dst` | Mount volume or bind mount |
| `-e KEY=value` | Environment variable |
| `--name` | Assign a container name |
| `--network` | Attach to a network |
| `--restart` | Restart policy (`no`, `on-failure`, `always`, `unless-stopped`) |
## 💡 Handy One-Liners
```bash
# stop every running container
docker stop $(docker ps -q)
# remove every container
docker rm -f $(docker ps -aq)
# remove dangling images
docker rmi $(docker images -qf dangling=true)
# shell into the most recent container
docker exec -it $(docker ps -ql) sh
# disposable dev environment
docker run --rm -it -v $(pwd):/app -w /app node:22 sh
# follow logs by partial name
docker logs -f $(docker ps -qf name=<partial>)
# list every mount of a container
docker inspect \
-f '{{range .Mounts}}{{.Source}} -> {{.Destination}}{{println}}{{end}}' <container>
```
## 🛠 Common Flags Reference
| Flag | Meaning |
|-------------------------------|--------------------------------------------------|
| `-d`, `--detach` | Run in the background |
| `-it` | Interactive session with a TTY (shells) |
| `--rm` | Remove the container when it exits |
| `-p <host>:<container>` | Publish a port to the host |
| `-v <src>:<dst>[:ro]` | Named volume or bind mount, optionally read-only |
| `-e KEY=value` / `--env-file` | Environment variables |
| `--name` | Assign a stable container name |
| `--network` | Attach to a network |
| `--restart` | `no`, `on-failure`, `always`, `unless-stopped` |
| `-u <uid>:<gid>` | Run as a specific user |
| `-w <dir>` | Working directory inside the container |
| `--memory` / `--cpus` | Resource limits |
| `--platform` | Target architecture, e.g. `linux/arm64` |
## 📚 Resources
- [Official documentation](https://docs.docker.com)
- [Docker Desktop (Mac, Linux, Windows)](https://docs.docker.com/desktop)
- [CLI reference](https://docs.docker.com/reference/cli/docker/)
- [Dockerfile reference](https://docs.docker.com/reference/dockerfile/)
- [Compose file reference](https://docs.docker.com/reference/compose-file/)
- [Awesome Compose — example projects](https://github.com/docker/awesome-compose)
+383 -304
View File
@@ -1,309 +1,356 @@
**Docker Swarm Mode** is Docker's native clustering and orchestration solution. It lets you manage a group of Docker nodes as a single virtual system, enabling high availability, load balancing, and simplified deployment of containerized applications.
**Docker Swarm mode** is Docker's native clustering and orchestration solution. It turns a group of Docker hosts into a single virtual system with high availability, load balancing and a declarative deployment model — using the CLI you already know.
Key features:
- Built-in orchestration with a declarative service model
- Rolling updates and rollback
- Self-healing (failed replicas are rescheduled automatically)
- Secure node-to-node communication via mutual TLS
- Built-in secrets and config distribution
- Rolling updates and one-command rollback
- Self-healing: failed replicas are rescheduled automatically
- Secure node-to-node communication over mutual TLS, rotated automatically
- Secrets and configs distributed straight from the cluster store
> ℹ️ Swarm does **not** auto-scale based on load. Replica counts are set manually with `docker service scale` or `--replicas`.
> ℹ️ Swarm does **not** auto-scale on load. Replica counts are set manually with `--replicas` or `docker service scale`.
## 🧱 Concepts
| Term | What it is |
|-------------|-------------------------------------------------------------------|
| **Node** | A Docker engine that joined the swarm — a *manager* or a *worker* |
| **Manager** | Keeps cluster state via Raft, schedules tasks, serves the API |
| **Worker** | Runs tasks only; has no view of the cluster state |
| **Service** | The declaration of what should run (image, replicas, ports, …) |
| **Task** | One container slot of a service, scheduled onto a node |
| **Stack** | A group of services deployed together from a Compose file |
## 📌 Swarm Initialization
Initialize a new swarm on the current node (becomes the first manager):
```bash
# initialize a swarm; this node becomes the first manager
docker swarm init
```
If the host has multiple network interfaces, specify which one to advertise on:
```bash
# pick the interface to advertise on multi-homed hosts
docker swarm init --advertise-addr <manager-ip>
```
Show the command another node needs to join as a worker:
# address pool for the overlay networks
docker swarm init --default-addr-pool 10.20.0.0/16
```bash
# print the command a worker needs to join
docker swarm join-token worker
```
Show the command for joining as a manager:
```bash
# print the command a manager needs to join
docker swarm join-token manager
```
Join an existing swarm (run on the joining node, using the output of the commands above):
```bash
# run on the joining node
docker swarm join --token <token> <manager-ip>:2377
```
Rotate join tokens (invalidates the old ones):
```bash
# rotate a token, invalidating the old one
docker swarm join-token --rotate worker
```
Leave the swarm (on a worker):
```bash
# leave the swarm (on a worker)
docker swarm leave
```
Force leave (on a manager — last manager must use `--force`):
```bash
# leave on a manager, or on the last node
docker swarm leave --force
# keep fewer terminated tasks per service
docker swarm update --task-history-limit 5
# lifetime of the node certificates
docker swarm update --cert-expiry 720h
# rotate the swarm certificate authority
docker swarm ca --rotate
```
### 🔌 Ports that must be open between nodes
| Port | Protocol | Used for |
|------------------------|-----------|--------------------------------------|
| `2377` | TCP | Cluster management (managers only) |
| `7946` | TCP + UDP | Node discovery and gossip |
| `4789` | UDP | Overlay network data plane (VXLAN) |
| IP protocol `50` (ESP) | — | Only with encrypted overlay networks |
## 👥 Node Management
List nodes in the swarm:
```bash
# list the nodes of the swarm
docker node ls
# filter by role, id, name, label, membership
docker node ls --filter role=manager
# readable summary of one node
docker node inspect <node> --pretty
# a single field via a Go template
docker node inspect -f '{{.Status.State}}' <node>
# worker → manager
docker node promote <node>
# manager → worker
docker node demote <node>
# move tasks off and stop scheduling new ones
docker node update --availability drain <node>
# keep running tasks, schedule nothing new
docker node update --availability pause <node>
# bring a drained or paused node back
docker node update --availability active <node>
# label a node for placement constraints
docker node update --label-add env=production <node>
# remove a label again
docker node update --label-rm env <node>
# tasks running on a node
docker node ps <node>
# remove a node that already left
docker node rm <node>
# remove an unreachable node
docker node rm --force <node>
```
Inspect a node:
```bash
docker node inspect <node-name> --pretty
```
Promote a worker to manager:
```bash
docker node promote <node-name>
```
Demote a manager to worker:
```bash
docker node demote <node-name>
```
Drain a node (reschedule its tasks elsewhere, prevent new scheduling):
```bash
docker node update --availability drain <node-name>
```
Pause a node (keep running tasks, prevent new ones):
```bash
docker node update --availability pause <node-name>
```
Activate a drained or paused node:
```bash
docker node update --availability active <node-name>
```
Add a label to a node (for use with placement constraints):
```bash
docker node update --label-add env=production <node-name>
```
Remove a node from the swarm (after it has left or is unreachable):
```bash
docker node rm <node-name>
```
## 🧠 Manager Nodes
Manager nodes orchestrate tasks, maintain cluster state via Raft consensus, and handle API requests. You can run multiple managers for HA, but only one is the **leader** at any time.
> ⚠️ **Use an odd number of managers** (3, 5, or 7). Raft tolerates `(N-1)/2` failures — 3 managers tolerate 1 failure, 5 tolerate 2. More than 7 hurts performance.
Show Swarm and Raft state for the local node:
> 💡 Drain a node before maintenance (`--availability drain`), then set it back to `active`. Swarm reschedules the tasks for you.
## 🧠 Manager Nodes & Raft
Managers orchestrate tasks, maintain the cluster state through Raft consensus and serve the API. Several managers give you high availability, but only one is the **leader** at a time.
> ⚠️ **Use an odd number of managers** (3, 5 or 7). Raft tolerates `(N-1)/2` failures — 3 managers survive 1 loss, 5 survive 2. Beyond 7 the consensus overhead outweighs the benefit.
```bash
# swarm and Raft state of the local node
docker info
```
Inspect the current node (includes manager status if applicable):
```bash
# details of the current node, manager status included
docker node inspect self --pretty
```
Enable autolock (encrypts Raft logs; managers need a key to restart):
```bash
# encrypt the Raft logs at rest
docker swarm update --autolock=true
```
Unlock a manager after restart:
```bash
# unlock a manager after a restart
docker swarm unlock
```
Show or rotate the unlock key:
```bash
# show the current unlock key
docker swarm unlock-key
# rotate it
docker swarm unlock-key --rotate
```
## 📦 Service Management
Create a service:
```bash
docker service create --name <service-name> <image>
```
# create a service
docker service create --name <service> <image>
Create a service with a fixed number of replicas:
# create a service with a fixed replica count
docker service create --name <service> --replicas <n> <image>
```bash
docker service create --name <service-name> --replicas <n> <image>
```
# create a global service: one task on every node
docker service create --name <service> --mode global <image>
Create a global service (one task per node):
```bash
docker service create --name <service-name> --mode global <image>
```
List services:
```bash
# list services
docker service ls
# filter the list
docker service ls --filter name=<service>
# readable service definition
docker service inspect <service> --pretty
# tasks and the nodes they run on
docker service ps <service>
# change the replica count
docker service scale <service>=<n>
# scale several services at once
docker service scale <svc-a>=3 <svc-b>=5
# roll out a new image
docker service update --image <new-image> <service>
# redistribute tasks without changing anything
docker service update --force <service>
# go back to the previous definition
docker service rollback <service>
# follow the logs of every task
docker service logs -f <service>
# recent output only
docker service logs --tail 100 --since 10m <service>
# remove a service
docker service rm <service>
```
Inspect a service:
### Changing a running service
```bash
docker service inspect <service-name> --pretty
# add or replace an environment variable
docker service update --env-add KEY=value <service>
# remove one
docker service update --env-rm KEY <service>
# publish another port
docker service update --publish-add 8080:80 <service>
# stop publishing it
docker service update --publish-rm 8080 <service>
# attach a volume
docker service update --mount-add type=volume,src=data,dst=/data <service>
# detach it again, by target path
docker service update --mount-rm /data <service>
# add a placement constraint
docker service update --constraint-add 'node.labels.env==prod' <service>
# at most one task per node
docker service update --replicas-max-per-node 1 <service>
# change the resource limits
docker service update --limit-memory 512m --reserve-memory 256m <service>
```
Scale a service:
```bash
docker service scale <service-name>=<n>
```
Update a service (e.g. change image):
```bash
docker service update --image <new-image> <service-name>
```
Roll back the last update:
```bash
docker service rollback <service-name>
```
Follow logs from all tasks of a service:
```bash
docker service logs -f <service-name>
```
Remove a service:
```bash
docker service rm <service-name>
```
## 🔁 Replicas
Replicas define how many instances of a service should run across the swarm. Swarm distributes them across available nodes and reschedules them if they fail.
Set replicas at create time:
## 🔁 Replicas & Modes
Replicas define how many instances of a service run across the swarm. Swarm spreads them over the available nodes and reschedules them when a task or a node fails.
| Mode | Flag | Behaviour |
|----------------|-------------------------------|---------------------------------------------------|
| Replicated | `--mode replicated` (default) | Run exactly `--replicas` tasks, anywhere they fit |
| Global | `--mode global` | Exactly one task per eligible node |
| Replicated job | `--mode replicated-job` | Run `--replicas` tasks **to completion** |
| Global job | `--mode global-job` | Run one task to completion on every node |
```bash
# five tasks
docker service create --replicas 5 --name myapp myimage
```
Change replica count later:
```bash
# scale the same service up to ten
docker service scale myapp=10
# one per node
docker service create --mode global --name agent myimage
# batch job: 20 tasks in total, 4 running at a time
docker service create --mode replicated-job --replicas 20 \
--max-concurrent 4 --name migrate myimage
```
## 🐝 Task & Container Management
## 🐝 Tasks & Containers
In Swarm, a **task** is a single container slot managed by Swarm. A task maps to a running container on some node.
List tasks of a service (shows which node each runs on):
A **task** is a single container slot managed by Swarm; it maps to one container on one node and is never moved — a failed task is replaced by a new one.
```bash
docker service ps <service-name>
```
# tasks of a service, with their node
docker service ps <service>
List tasks running on a specific node:
# full error messages — start debugging here
docker service ps --no-trunc <service>
```bash
docker node ps <node-name>
```
# hide the terminated tasks
docker service ps -f "desired-state=running" <service>
List containers on the current node:
# tasks on one node
docker node ps <node>
```bash
# containers on the current node
docker container ls
```
Inspect a container:
```bash
# low-level details of one container
docker container inspect <container-id>
```
| Task state | Meaning |
|--------------------------|-------------------------------------------------------------|
| `NEW` / `PENDING` | Accepted, waiting for a node that satisfies the constraints |
| `ASSIGNED` / `PREPARING` | Sent to a node, image being pulled |
| `STARTING` / `RUNNING` | The container is starting or up |
| `COMPLETE` | A job task finished successfully |
| `FAILED` | The container exited with an error |
| `SHUTDOWN` | Stopped on purpose (update, drain, scale down) |
| `REJECTED` | The node refused the task (missing image, bad mount) |
| `ORPHANED` | The node has been unreachable too long |
## 🎯 Placement
```bash
docker service create --constraint 'node.role==worker' --name web nginx
docker service create --constraint 'node.labels.env==production' --name api myimage
docker service create --placement-pref 'spread=node.labels.zone' --name web nginx
docker service create --replicas-max-per-node 1 --name web nginx
```
| Expression | Matches |
|-------------------------------------------|---------------------------------------------------|
| `node.role` | `manager` or `worker` |
| `node.hostname` | The node's hostname |
| `node.id` | The node's ID |
| `node.labels.<key>` | A label set with `docker node update --label-add` |
| `engine.labels.<key>` | A label set in the engine's `daemon.json` |
| `node.platform.os` / `node.platform.arch` | `linux`, `windows` / `amd64`, `arm64` |
## 🌐 Networking
### What is an overlay network?
An overlay network is a virtual network that spans multiple Docker hosts. It lets containers running on different nodes communicate as if they were on the same Layer 2 network, with built-in DNS-based service discovery and encrypted traffic between nodes.
Create an overlay network:
An **overlay network** is a virtual network spanning multiple Docker hosts. Containers on different nodes talk as if they were on the same L2 segment, with DNS-based service discovery built in.
```bash
docker network create --driver overlay <network-name>
```
# create an overlay network
docker network create --driver overlay <network>
Create an encrypted overlay network (encrypts data plane between nodes):
# encrypt the data plane too
docker network create --driver overlay --opt encrypted <network>
```bash
docker network create --driver overlay --opt encrypted <network-name>
```
# let standalone containers join
docker network create --driver overlay --attachable <network>
List networks:
```bash
# list networks
docker network ls
# attached services and containers
docker network inspect <network>
# attach at creation time
docker service create --name <service> --network <network> <image>
# attach an existing service
docker service update --network-add <network> <service>
# detach it again
docker service update --network-rm <network> <service>
```
Inspect a network (shows attached services and containers):
### Publishing ports
```bash
docker network inspect <network-name>
# routing mesh (ingress)
docker service create --publish 8080:80 --name web nginx
# host port, no mesh
docker service create --publish mode=host,target=80,published=8080 --name web nginx
```
Attach a service to a network at create time:
```bash
docker service create --name <service-name> --network <network-name> <image>
```
Attach an existing service to an additional network:
```bash
docker service update --network-add <network-name> <service-name>
```
| Mode | Behaviour |
|---------------------|----------------------------------------------------------------------------|
| `ingress` (default) | Every node accepts the port and load-balances to the tasks |
| `host` | The port is published only on nodes running a task — no mesh, no extra hop |
## 🔄 Rolling Updates & Rollback
Configure update behavior at service create or update time:
```bash
docker service create \
--name web \
@@ -313,164 +360,192 @@ docker service create \
--update-failure-action rollback \
--update-monitor 30s \
nginx:1.25
```
Trigger an update by changing the image:
```bash
# trigger the rolling update
docker service update --image nginx:1.26 web
```
Roll back to the previous spec:
```bash
# return to the previous definition
docker service rollback web
```
Update flags worth knowing:
| Flag | Description |
|---------------------------|------------------------------------------------------------------------------------------------------------------------------|
| `--update-parallelism` | How many tasks are updated at once (`0` = all) |
| `--update-delay` | Pause between batches |
| `--update-failure-action` | `pause` (default), `continue` or `rollback` |
| `--update-monitor` | How long a task is watched before it counts as healthy |
| `--update-order` | `stop-first` (default) or `start-first` |
| `--rollback-parallelism` | Same knobs for the rollback path (`--rollback-delay`, `--rollback-monitor`, `--rollback-failure-action`, `--rollback-order`) |
| Flag | Description |
|------|-------------|
| `--update-parallelism` | How many tasks to update at once |
| `--update-delay` | Delay between batches |
| `--update-failure-action` | `pause`, `continue`, or `rollback` on failure |
| `--update-monitor` | Time to watch each task for failure |
| `--update-order` | `stop-first` (default) or `start-first` |
## 🩺 Health Checks
Swarm reschedules a task as soon as its health check fails, so an update never marks a broken image as healthy.
```bash
docker service create \
--name api \
--health-cmd 'curl -f http://localhost:8080/health || exit 1' \
--health-interval 10s \
--health-timeout 3s \
--health-retries 3 \
--health-start-period 30s \
myimage
```
## 🔐 Secrets
Secrets are encrypted at rest in the Raft log and mounted into containers as in-memory files under `/run/secrets/`.
Create a secret from a file:
Secrets are encrypted in the Raft log and mounted into the container as in-memory files under `/run/secrets/`.
```bash
# create from a file
docker secret create db_password ./db_password.txt
```
Create a secret from stdin:
```bash
# create from stdin
echo "s3cr3t" | docker secret create db_password -
```
List secrets:
```bash
# list secrets
docker secret ls
```
Inspect a secret (does **not** reveal the value):
```bash
# metadata only — never the value
docker secret inspect db_password
```
Attach a secret to a service:
```bash
# attach a secret; it appears at /run/secrets/db_password
docker service create --name db --secret db_password postgres:16
# ...or mount it at a custom path with a fixed mode
docker service create --name db \
--secret source=db_password,target=/run/secrets/pg_pw,mode=0400 postgres:16
# only possible when no service uses it
docker secret rm db_password
```
Remove a secret (only if no service uses it):
Rotating a secret — create the new one, swap it, then drop the old one:
```bash
echo "n3w-s3cr3t" | docker secret create db_password_v2 -
docker service update \
--secret-rm db_password \
--secret-add source=db_password_v2,target=db_password \
db
docker secret rm db_password
```
## 🔑 Configs
Configs are like secrets but **not encrypted at rest** — meant for non-sensitive config files (nginx.conf, etc.).
Create a config:
Configs work like secrets but are **not encrypted at rest** — use them for non-sensitive files such as `nginx.conf`.
```bash
# create a config
docker config create my_nginx_conf ./nginx.conf
```
List configs:
```bash
# list configs
docker config ls
```
Attach a config to a service at a specific path:
# metadata and the stored content
docker config inspect my_nginx_conf
```bash
# mount a config at a specific path
docker service create \
--name web \
--config source=my_nginx_conf,target=/etc/nginx/nginx.conf \
nginx
```
Remove a config:
# swap in a new version of the config
docker service update \
--config-rm my_nginx_conf \
--config-add source=my_nginx_conf_v2,target=/etc/nginx/nginx.conf \
web
```bash
# only when no service uses it
docker config rm my_nginx_conf
```
## 📚 Stacks
A **stack** is a group of related services deployed together from a Compose file. This is the standard production workflow for Swarm.
Deploy (or update) a stack:
A **stack** is a group of related services deployed together from a Compose file — the standard production workflow for Swarm.
```bash
# deploy or update a stack
docker stack deploy -c stack.yaml mystack
```
List stacks:
# pass your registry credentials along
docker stack deploy -c stack.yaml --with-registry-auth mystack
```bash
# remove services no longer in the file
docker stack deploy -c stack.yaml --prune mystack
# list stacks
docker stack ls
```
List services in a stack:
```bash
# services of a stack
docker stack services mystack
```
List tasks in a stack:
```bash
# tasks of a stack
docker stack ps mystack
```
Remove a stack:
# tasks of a stack, with full error messages
docker stack ps --no-trunc mystack
```bash
# print the merged, resolved file
docker stack config -c stack.yaml
# remove the whole stack
docker stack rm mystack
```
Stacks honor the `deploy:` block in Compose files (replicas, placement, restart policy, update config). `build:`, top-level `restart:`, and `depends_on:` are ignored.
Stacks honour the `deploy:` block of the Compose file (replicas, placement, resources, restart policy, update config). `build:`, top-level `restart:` and `depends_on:` are ignored.
## 🧯 Troubleshooting
```bash
# the error column tells you why a task died
docker service ps --no-trunc <service>
# application output of every task
docker service logs --tail 200 <service>
# is a node Down or Unreachable?
docker node ls
# what was actually deployed
docker service inspect --pretty <service>
# live stream of orchestration events
docker events --filter type=service
```
| Symptom | Usual cause |
|--------------------------------------|--------------------------------------------------------------------------------------|
| Task stuck in `PENDING` | No node satisfies the constraints, ports, or resource reservations |
| `no suitable node` | Placement constraint or `--replicas-max-per-node` cannot be met |
| Task loops `FAILED` → `STARTING` | The container exits immediately — read `docker service logs` |
| `REJECTED` with an image error | The image is missing on that node; use `--with-registry-auth` for private registries |
| Service unreachable from other nodes | Ports `7946`/`4789` blocked, or the services are not on the same overlay network |
## 🛠 Common Flags Reference
### `docker service create` / `update`
### `docker service create` / `docker service update`
| Flag | Description |
|------|-------------|
| `--replicas` | Number of service replicas |
| `--mode` | `replicated` (default) or `global` |
| `--publish` | Port mapping (`<host>:<container>`) |
| `--mount` | Volume mounting |
| `--network` | Attach to a network |
| `--constraint` | Node placement rule (e.g. `node.role==worker`) |
| `--placement-pref` | Spread tasks (e.g. `spread=node.labels.zone`) |
| `--limit-cpu` / `--limit-memory` | Hard resource limits |
| `--reserve-cpu` / `--reserve-memory` | Reserved resources |
| `--secret` / `--config` | Attach a secret or config |
| `--env` / `--env-file` | Environment variables |
| `--restart-condition` | `none`, `on-failure`, or `any` |
### Update-specific flags
| Flag | Description |
|------|-------------|
| `--update-parallelism` | Tasks updated in parallel |
| `--update-delay` | Delay between update batches |
| `--update-failure-action` | `pause`, `continue`, `rollback` |
| `--update-order` | `stop-first` or `start-first` |
| Flag | Description |
|--------------------------------------|------------------------------------------------------------------|
| `--replicas` | Number of tasks for a replicated service |
| `--mode` | `replicated` (default), `global`, `replicated-job`, `global-job` |
| `--publish` | Port mapping (`<published>:<target>`, or `mode=host,...`) |
| `--mount` | Attach a volume or bind mount |
| `--network` | Attach to an overlay network |
| `--constraint` | Placement rule, e.g. `node.role==worker` |
| `--placement-pref` | Spread tasks, e.g. `spread=node.labels.zone` |
| `--replicas-max-per-node` | Cap the tasks of this service per node |
| `--limit-cpu` / `--limit-memory` | Hard resource limits |
| `--reserve-cpu` / `--reserve-memory` | Resources reserved for scheduling |
| `--secret` / `--config` | Attach a secret or a config |
| `--env` / `--env-file` | Environment variables |
| `--restart-condition` | `none`, `on-failure` or `any` (default) |
| `--health-cmd` and friends | Override the image health check |
| `--with-registry-auth` | Forward registry credentials to the nodes |
## 📄 Example: Production-style Web Service
@@ -482,11 +557,14 @@ docker service create \
--network webnet \
--constraint 'node.role==worker' \
--constraint 'node.labels.env==production' \
--placement-pref 'spread=node.labels.zone' \
--update-parallelism 1 \
--update-delay 10s \
--update-failure-action rollback \
--update-order start-first \
--limit-memory 512m \
--reserve-memory 256m \
--restart-condition any \
nginx:1.25
```
@@ -496,4 +574,5 @@ docker service create \
- [Swarm services](https://docs.docker.com/engine/swarm/services/)
- [Manage secrets](https://docs.docker.com/engine/swarm/secrets/)
- [Manage configs](https://docs.docker.com/engine/swarm/configs/)
- [Stack file reference (Compose Spec `deploy:`)](https://docs.docker.com/compose/compose-file/deploy/)
- [Stack file reference (Compose Spec `deploy:`)](https://docs.docker.com/reference/compose-file/deploy/)
- [`docker service create` CLI reference](https://docs.docker.com/reference/cli/docker/service/create/)
+235 -216
View File
@@ -1,258 +1,277 @@
A **Dockerfile** is a script composed of instructions to build a Docker image. Each instruction creates a layer in the image.
A **Dockerfile** is the recipe for an image: each instruction runs in order and most of them add a layer. Layers are cached, so the order you write them in decides how fast your rebuilds are.
## 📦 Basic Structure
```Dockerfile
# Comment
INSTRUCTION arguments
```
## 🚀 Core Instructions
### `FROM`
Specifies the base image.
```Dockerfile
FROM ubuntu:20.04
```dockerfile
# syntax=docker/dockerfile:1
FROM node:22-alpine
```
### `LABEL`
Adds metadata to the image.
```Dockerfile
LABEL maintainer="you@example.com"
LABEL version="1.0" description="My App"
```
### `ENV`
Sets environment variables.
```Dockerfile
ENV NODE_ENV=production
ENV PATH="/app/bin:$PATH"
```
### `RUN`
Executes commands in a shell during build.
```Dockerfile
RUN apt-get update && apt-get install -y curl
RUN npm install
```
Use `RUN ["executable", "param1", "param2"]` for JSON array form.
### `COPY`
Copies files from host to image.
```Dockerfile
COPY . /app
COPY config.json /app/config.json
```
### `ADD`
Similar to `COPY`, but supports remote URLs and auto-extracts archives.
```Dockerfile
ADD https://example.com/file.tar.gz /app/
ADD archive.zip /app/
```
### `CMD`
Sets default command to run when container starts.
```Dockerfile
CMD ["node", "server.js"] # Preferred exec form
CMD node server.js # Shell form
```
Only one `CMD` allowed; later ones override earlier.
### `ENTRYPOINT`
Configures a container to run as an executable.
```Dockerfile
ENTRYPOINT ["python", "app.py"]
```
Use with `CMD` to pass default arguments.
### `WORKDIR`
Sets the working directory for subsequent instructions.
```Dockerfile
WORKDIR /app
COPY package*.json ./
RUN npm ci
COPY . .
CMD ["node", "server.js"]
```
### `EXPOSE`
Documents the port the container listens on.
> 💡 Start the file with `# syntax=docker/dockerfile:1` to get the current BuildKit frontend — that is what enables heredocs, cache mounts and build secrets.
```Dockerfile
EXPOSE 80
EXPOSE 443
## 📋 Instruction Reference
| Instruction | Purpose |
|---------------|---------------------------------------------------------------|
| `FROM` | The base image, and the start of a build stage |
| `ARG` | A build-time variable (`--build-arg`) |
| `ENV` | An environment variable, kept in the final image |
| `WORKDIR` | The working directory for the instructions that follow |
| `COPY` | Copy files from the context (or another stage) into the image |
| `ADD` | Like `COPY`, but also fetches URLs and unpacks archives |
| `RUN` | Execute a command at build time and commit the result |
| `CMD` | Default command, easily overridden at `docker run` |
| `ENTRYPOINT` | The executable the container always runs |
| `EXPOSE` | Document the port the app listens on |
| `USER` | The user the following instructions and the container run as |
| `VOLUME` | Declare a path that should be a mount point |
| `HEALTHCHECK` | How Docker decides the container is healthy |
| `LABEL` | Image metadata |
| `SHELL` | Change the shell used by the shell form of `RUN`/`CMD` |
| `STOPSIGNAL` | The signal sent to stop the container |
| `ONBUILD` | An instruction that runs when this image is used as a base |
## 🏗 Base Image & Metadata
```dockerfile
# pin the base image — 'latest' makes builds irreproducible
FROM node:22-alpine
# name a stage so later stages can copy from it
FROM golang:1.23 AS build
# an ARG before the first FROM can parameterise the base image itself
ARG NODE_VERSION=22
FROM node:${NODE_VERSION}-alpine
# metadata; the OCI keys are the conventional ones
LABEL org.opencontainers.image.source="https://github.com/acme/app"
LABEL org.opencontainers.image.description="Acme API"
```
Note: This does not publish the port.
## 📂 Files & Working Directory
### `VOLUME`
Creates a mount point for persistent or shared data.
```dockerfile
# always set an absolute working directory
WORKDIR /app
```Dockerfile
# copy from the build context
COPY package*.json ./
COPY . .
# copy and set ownership in one step, no extra layer for chown
COPY --chown=node:node . .
# copy with explicit permissions
COPY --chmod=755 entrypoint.sh /usr/local/bin/
# copy from another stage, or straight from another image
COPY --from=build /src/app /usr/local/bin/app
COPY --from=nginx:alpine /etc/nginx/nginx.conf /etc/nginx/nginx.conf
# ADD unpacks archives and fetches URLs — prefer COPY unless you need that
ADD archive.tar.gz /opt/
ADD https://example.com/file.tar.gz /tmp/
# declare a mount point for data that should not live in the image
VOLUME ["/data"]
```
### `USER`
Sets the user to run subsequent instructions.
## ⚙️ Build Steps
```Dockerfile
```dockerfile
# shell form: runs through /bin/sh -c
RUN apt-get update && apt-get install -y --no-install-recommends curl \
&& rm -rf /var/lib/apt/lists/*
# exec form: no shell, so no globbing or variable expansion
RUN ["npm", "ci", "--omit=dev"]
# heredoc: a readable multi-line script (BuildKit)
RUN <<EOF
set -eux
apk add --no-cache curl
adduser -D appuser
EOF
# cache mount: keep the package cache between builds
RUN --mount=type=cache,target=/root/.npm npm ci
# secret mount: use a credential without baking it into a layer
RUN --mount=type=secret,id=npmrc,target=/root/.npmrc npm ci
# bind mount: read from the context without copying it in
RUN --mount=type=bind,source=package.json,target=package.json npm ci
```
## 🚀 Runtime Configuration
```dockerfile
# environment variables are visible to the running container
ENV NODE_ENV=production
ENV PATH="/app/bin:$PATH"
# document the port — this does not publish it, -p does
EXPOSE 3000
# drop root as early as you can
RUN adduser -D appuser
USER appuser
# tell Docker how to check the app is alive
HEALTHCHECK --interval=30s --timeout=3s --start-period=20s --retries=3 \
CMD wget -qO- http://localhost:3000/health || exit 1
# the signal that stops the process cleanly
STOPSIGNAL SIGTERM
# the command the container runs
CMD ["node", "server.js"]
```
### `ARG`
Defines build-time variables.
## ⚔️ CMD vs ENTRYPOINT
```Dockerfile
ARG VERSION=1.0
RUN echo $VERSION
| | `CMD` | `ENTRYPOINT` |
|--------------------------------|-------------------------------------------------|------------------------------------------|
| Purpose | Default arguments, or the whole default command | The executable the image is built around |
| `docker run <image> other-cmd` | Replaced entirely | Still runs; the argument is appended |
| Override flag | — | `--entrypoint` |
| Typical use | `CMD ["node", "server.js"]` | `ENTRYPOINT ["python", "app.py"]` |
```dockerfile
# together: ENTRYPOINT is the binary, CMD holds the default arguments
ENTRYPOINT ["python", "app.py"]
CMD ["--port", "8000"]
# docker run image → python app.py --port 8000
# docker run image --port 9000 → python app.py --port 9000
```
Use `--build-arg VERSION=2.0` during `docker build`.
> ⚠️ Prefer the **exec form** (`["cmd", "arg"]`). The shell form wraps your process in `/bin/sh -c`, which swallows `SIGTERM` and turns graceful shutdown into a ten-second kill.
### `ONBUILD`
Triggers instructions when the image is used as a base.
## 🧱 Multi-Stage Builds
```Dockerfile
ONBUILD COPY . /app
Several `FROM` instructions in one file: build with a fat toolchain, ship only the artifact.
```dockerfile
# stage 1 — build
FROM node:22-alpine AS build
WORKDIR /app
COPY package*.json ./
RUN npm ci
COPY . .
RUN npm run build
# stage 2 — the image that actually ships
FROM nginx:alpine
COPY --from=build /app/dist /usr/share/nginx/html
EXPOSE 80
CMD ["nginx", "-g", "daemon off;"]
```
```dockerfile
# a compiled binary on a near-empty base
FROM golang:1.23 AS build
WORKDIR /src
COPY . .
RUN CGO_ENABLED=0 go build -o /out/app ./cmd/app
FROM gcr.io/distroless/static-debian12
COPY --from=build /out/app /app
USER nonroot:nonroot
ENTRYPOINT ["/app"]
```
```bash
# build only up to a named stage — handy for a test or lint stage
docker build --target build -t app:build .
```
## 📁 .dockerignore
Everything in the build context is sent to the daemon and busts the cache. Exclude what the build does not need:
```plaintext
.git
node_modules
dist
*.log
.env
Dockerfile
.dockerignore
```
## 🧠 Best Practices
- Use minimal base images (e.g., `alpine`) for smaller size.
- Combine `RUN` commands to reduce layers.
- Use `.dockerignore` to exclude unnecessary files.
- Prefer `COPY` over `ADD` unless you need archive extraction.
- Use `ENTRYPOINT` for fixed commands and `CMD` for arguments.
- Avoid hardcoding secrets or credentials.
- **Order by how often things change**: base image, then dependency manifests, then `npm ci`/`pip install`, then the source. A source edit should not reinstall dependencies.
- **Pin versions** — `node:22.11-alpine`, not `node:latest`.
- **One `RUN` per logical step**, chaining installs and cleanup in the same layer (`rm -rf /var/lib/apt/lists/*`), or the deleted files still weigh on the image.
- **Never bake secrets in.** A deleted file stays in the layer below; use `--mount=type=secret` or build arguments that never reach the final stage.
- **Run as a non-root user** and keep the final stage as small as you can (`alpine`, `distroless`, `scratch`).
- **Use `COPY`, not `ADD`,** unless you specifically want URL fetching or archive extraction.
- **Combine with `--platform`** for multi-arch: `docker buildx build --platform linux/amd64,linux/arm64`.
## 🧪 Example Dockerfile
```Dockerfile
FROM node:22-alpine
LABEL maintainer="guillaume@example.com"
WORKDIR /app
COPY package*.json ./
RUN npm install
COPY . .
EXPOSE 3000
ENV NODE_ENV=production
CMD ["node", "index.js"]
```
## 🛠️ Build & Run
## 🛠 Building & Running
```bash
# Build image
# build from the Dockerfile in this directory
docker build -t my-app .
# Run container
# build without the cache
docker build --no-cache -t my-app .
# a Dockerfile somewhere else
docker build -f docker/Dockerfile.prod -t my-app .
# pass a build argument
docker build --build-arg NODE_VERSION=20 -t my-app .
# pass a secret (never becomes a layer)
docker build --secret id=npmrc,src=$HOME/.npmrc -t my-app .
# build for another architecture
docker buildx build --platform linux/arm64 -t my-app .
# inspect the layers of the result
docker history my-app
# run it
docker run -p 3000:3000 my-app
```
## 📁 .dockerignore Example
## 📄 Full Example
```plaintext
node_modules
*.log
Dockerfile
.git
```
## 🏗️ Multistage Builds
**Multistage builds** allow you to use multiple `FROM` statements in one Dockerfile to optimize image size and separate build dependencies from runtime.
### 🎯 Why Use Multistage Builds?
- Reduce final image size by excluding build tools and intermediate files.
- Improve security by minimizing attack surface.
- Keep Dockerfiles clean and maintainable.
### 🧱 Basic Syntax
```Dockerfile
# Stage 1: Build
FROM node:22-alpine AS builder
```dockerfile
# syntax=docker/dockerfile:1
FROM node:22-alpine AS deps
WORKDIR /app
COPY package*.json ./
RUN npm install
RUN --mount=type=cache,target=/root/.npm npm ci
FROM node:22-alpine AS build
WORKDIR /app
COPY --from=deps /app/node_modules ./node_modules
COPY . .
RUN npm run build
# Stage 2: Production
FROM nginx:alpine
COPY --from=builder /app/dist /usr/share/nginx/html
```
### 🏷️ Named Stages
You can name each stage using `AS <name>` and reference it later with `--from=<name>`.
```Dockerfile
FROM golang:1.21 AS build
WORKDIR /src
COPY . .
RUN go build -o myapp
FROM alpine:latest
COPY --from=build /src/myapp /usr/local/bin/myapp
ENTRYPOINT ["myapp"]
```
### 📦 Copying Artifacts
Use `COPY --from=<stage>` to copy files from one stage to another.
```Dockerfile
COPY --from=builder /app/output /app/output
```
You can copy:
- Files
- Directories
- Binaries
- Configs
### 🧼 Clean Final Image
Multistage builds help you avoid bloated images:
```Dockerfile
# Without multistage: includes compilers, source code, etc.
# With multistage: only includes runtime essentials
```
### 🧪 Real-World Example: React App
```Dockerfile
# Build stage
FROM node:22 AS build
FROM node:22-alpine
LABEL org.opencontainers.image.source="https://github.com/acme/app"
ENV NODE_ENV=production
WORKDIR /app
COPY package*.json ./
RUN npm install
COPY . .
RUN npm run build
COPY --from=deps --chown=node:node /app/node_modules ./node_modules
COPY --from=build --chown=node:node /app/dist ./dist
USER node
EXPOSE 3000
HEALTHCHECK --interval=30s CMD wget -qO- http://localhost:3000/health || exit 1
CMD ["node", "dist/server.js"]
```
# Serve stage
FROM nginx:alpine
COPY --from=build /app/build /usr/share/nginx/html
EXPOSE 80
CMD ["nginx", "-g", "daemon off;"]
```
## 📚 Resources
- [Dockerfile reference](https://docs.docker.com/reference/dockerfile/)
- [Building best practices](https://docs.docker.com/build/building/best-practices/)
- [BuildKit mounts (cache, secret, bind)](https://docs.docker.com/build/cache/optimize/)
- [Multi-stage builds](https://docs.docker.com/build/building/multi-stage/)
+512 -183
View File
@@ -1,247 +1,576 @@
## Configuration
**Git** is a distributed version control system: every clone is a full repository with its own history, branches and tags. This is a quick reference for the commands that come up daily — placeholders are written as `<file>`, `<branch>`, `<commit>` and `<url>`.
Set the global config
```shell
git config --global user.name "[name]"
git config --global user.email "[email]"
> 💡 Every command below has its own help (`git rebase --help`), and the destructive ones (`clean`, `push`, `rm`, `mv`) take `-n` / `--dry-run` to preview what would happen.
## ⚙️ Configuration
```bash
# identity used on every commit
git config --global user.name "<name>"
git config --global user.email "<email>"
# name of the branch created by git init
git config --global init.defaultBranch main
# editor for messages, rebases, ...
git config --global core.editor "code --wait"
# rebase instead of merging on pull
git config --global pull.rebase true
# plain 'git push' works on new branches
git config --global push.autoSetupRemote true
# every setting and the file it came from
git config --list --show-origin
# override for this repository only
git config --local user.email "<work-email>"
```
## Get started
Handy aliases:
Create a git repository
```shell
```bash
git config --global alias.st status
git config --global alias.co checkout
git config --global alias.lg "log --oneline --graph --decorate --all"
git config --global alias.last "log -1 --stat"
```
## 🚀 Get Started
```bash
# create a repository in the current directory
git init
# clone a remote repository
git clone <url>
# clone into a specific directory
git clone <url> <directory>
# clone and check out one branch
git clone --branch <branch> <url>
# shallow clone, latest commit only
git clone --depth 1 <url>
# clone including submodules
git clone --recurse-submodules <url>
# what is staged, changed, untracked
git status
# the same, in short format
git status -sb
```
Pull a git repository
```shell
git pull [url]
```
## ➕ Staging
Clone an existing git repository
```shell
git clone [url]
```
```bash
# stage a specific file
git add <file>
## Staging
Stage specific files
```shell
git add [file]
```
Stage all changes
```shell
# stage everything under the current directory
git add .
# stage everything in the repository, deletions included
git add -A
# interactively stage individual hunks
git add -p <file>
# stage modifications and deletions, not new files
git add -u
# unstage, keep the changes in the working tree
git restore --staged <file>
# stop tracking a file, keep it on disk
git rm --cached <file>
# rename and stage in one step
git mv <old> <new>
# which .gitignore rule is hiding this file?
git check-ignore -v <file>
```
Interactively stage parts of a file (hunks)
```shell
git add -p [file]
```
## ✅ Commit
Unstage a file but keep changes
```shell
git restore --staged [file]
```
```bash
# commit what is staged
git commit -m "<message>"
## Commit
# stage tracked changes and commit
git commit -am "<message>"
Commit all tracked changes
```shell
git commit -am "[commit message]"
```
# rewrite the last commit (message + content)
git commit --amend
Add new modifications to the last commit
```shell
# amend without changing the message
git commit --amend --no-edit
# fixup commit, squashed later by --autosquash
git commit --fixup <commit>
# add a Signed-off-by trailer
git commit -s -m "<message>"
# empty commit, e.g. to trigger CI
git commit --allow-empty -m "<message>"
```
## Branches
> ⚠️ `--amend` rewrites history. Only amend commits that have not been pushed to a shared branch.
Lists all local branches in your repository (use -a for local and remote branches)
```shell
## 🌿 Branches
```bash
# list local branches
git branch
# local and remote-tracking branches
git branch -a
# last commit and upstream of each branch
git branch -vv
# switch to an existing branch
git switch <branch>
# create a branch and switch to it
git switch -c <branch>
# jump back to the previous branch
git switch -
# older equivalent of switch -c
git checkout -b <branch>
# branch off a specific commit or tag
git switch -c <branch> <commit>
# rename a branch
git branch -m <old> <new>
# delete a merged branch
git branch -d <branch>
# force-delete an unmerged branch
git branch -D <branch>
# branches already merged into HEAD (safe to delete)
git branch --merged
```
Switch to an existing branch
```shell
git switch [branch name]
## 🔀 Merging & Rebasing
```bash
# merge a branch into the current one
git merge <branch>
# always create a merge commit
git merge --no-ff <branch>
# bring in the changes as one staged change set
git merge --squash <branch>
# bail out of a conflicted merge
git merge --abort
# replay the current branch on top of another
git rebase <branch>
# interactively squash, reword, drop, reorder commits
git rebase -i HEAD~5
# apply the --fixup commits automatically
git rebase -i --autosquash <base>
# resume after resolving conflicts
git rebase --continue
# drop the conflicting commit and continue
git rebase --skip
# return to the state before the rebase
git rebase --abort
# resolve conflicts with the configured merge tool
git mergetool
```
Create a new branch
```shell
git checkout -b [branch name]
Resolving a conflict:
```bash
# 1. see which files conflict
git status
# 2. edit them and remove the <<<<<<< ======= >>>>>>> markers
# 3. mark each resolved file
git add <file>
# 4. resume (during a merge: git merge --continue)
git rebase --continue
```
## Stashing
> ⚠️ Rebasing rewrites commits. Never rebase a branch other people are already working on.
Save uncommitted changes for later
```shell
git stash
```
## 🌍 Remotes & Syncing
List all stashes
```shell
git stash list
```
Apply most recent stash and remove it from the list
```shell
git stash pop
```
Apply a specific stash without removing it
```shell
git stash apply stash@{2}
```
## Inspecting
Show changes between working directory and staging
```shell
git diff
```
Show changes between staging and last commit
```shell
git diff --staged
```
Show commit history for a specific file
```shell
git log --follow [file]
```
Show who changed each line in a file
```shell
git blame [file]
```
## Remote
Add a remote repository
```shell
git remote add origin [url]
```
List remote repositories
```shell
```bash
# list remotes and their URLs
git remote -v
```
Push a branch and set upstream tracking
```shell
git push -u origin [branch-name]
```
# add a remote
git remote add origin <url>
Delete a remote branch
```shell
git push origin --delete [branch-name]
```
# point a remote somewhere else
git remote set-url origin <url>
## Tags
# rename a remote
git remote rename <old> <new>
Create a tagged release
```shell
git tag -a v1.0.0 -m "Release v1.0.0"
```
# forget a remote
git remote remove <name>
Push tags to remote
```shell
# download objects and refs, change nothing
git fetch origin
# fetch everything, drop deleted remote branches
git fetch --all --prune
# fetch + merge (or rebase) the upstream branch
git pull
# replay local commits on top of the upstream
git pull --rebase
# push the current branch
git push
# push and set the upstream tracking branch
git push -u origin <branch>
# force-push, but refuse to clobber new commits
git push --force-with-lease
# delete a remote branch
git push origin --delete <branch>
# push all tags
git push origin --tags
```
## I've made a mistake
> ⚠️ Prefer `--force-with-lease` over `--force`: it aborts if someone else pushed in the meantime.
Change last commit message
```shell
git commit --amend
## 📦 Stashing
```bash
# shelve tracked changes and clean the working tree
git stash
# include untracked files
git stash -u
# stash specific paths with a label
git stash push -m "<message>" <file>
# list every stash
git stash list
# show a stash as a patch
git stash show -p stash@{0}
# re-apply the newest stash and drop it
git stash pop
# re-apply a specific stash, keep it in the list
git stash apply stash@{2}
# create a branch from a stash and apply it
git stash branch <branch>
# delete one stash
git stash drop stash@{0}
# delete all stashes
git stash clear
```
Undo most recent commit and keep changes
```shell
git reset HEAD~1
```
## 🔍 Inspecting
Undo the `N` most recent commit and keep changes
```shell
git reset HEAD~N
```
```bash
# the whole history as a compact graph
git log --oneline --graph --decorate --all
Undo most recent commit and get rid of changes
```shell
git reset HEAD~1 --hard
```
# history of a file, with diffs
git log -p <file>
Reset branch to remote state
```shell
git fetch origin
git reset --hard origin/[branch-name]
```
# history of a file, following renames
git log --follow <file>
Revert a commit by creating a new undo commit (safe for shared branches)
```shell
git revert [commit-hash]
```
# filter by date and author
git log --since="2 weeks ago" --author=<name>
Discard all uncommitted changes in working directory
```shell
git restore .
```
# search commit messages
git log --grep="<pattern>"
Recover a deleted branch or lost commit
```shell
# commits that added or removed a string
git log -S"<string>"
# commits in b that are not in a
git log <branch-a>..<branch-b>
# commit count per author
git shortlog -sn
# a single commit with its diff
git show <commit>
# working tree vs. index
git diff
# index vs. last commit
git diff --staged
# between two commits
git diff HEAD~1 HEAD
# changes since the branches diverged
git diff <branch-a>...<branch-b>
# summary of changed files
git diff --stat
# who last touched every line
git blame <file>
# blame a range of lines only
git blame -L 10,40 <file>
# search the tracked files
git grep "<pattern>"
# every position HEAD has had — your safety net
git reflog
git checkout -b [branch-name] [commit-hash]
```
## Cherry-pick
## 🍒 Cherry-pick
Apply a specific commit from another branch
```shell
git cherry-pick [commit-hash]
```bash
# apply one commit onto the current branch
git cherry-pick <commit>
# apply a range of commits
git cherry-pick <commit-a>^..<commit-b>
# apply without committing
git cherry-pick -n <commit>
# after resolving conflicts
git cherry-pick --continue
# undo the whole cherry-pick
git cherry-pick --abort
```
## Cleaning
## 🏷 Tags
Remove untracked files (dry run first)
```shell
```bash
# list tags
git tag
# list matching tags
git tag -l "v1.*"
# annotated tag on HEAD
git tag -a v1.0.0 -m "Release v1.0.0"
# tag an older commit
git tag -a v1.0.0 <commit>
# show a tag and its commit
git show v1.0.0
# push one tag
git push origin v1.0.0
# push all tags
git push origin --tags
# delete a local tag
git tag -d v1.0.0
# delete a remote tag
git push origin --delete v1.0.0
# closest tag to the current commit
git describe --tags
```
## 🧯 I've Made a Mistake
```bash
# fix the last commit message
git commit --amend
# undo the last commit, keep the changes unstaged
git reset HEAD~1
# undo the last n commits, keep the changes
git reset HEAD~<n>
# undo the last commit and throw the changes away
git reset --hard HEAD~1
# discard changes to one file
git restore <file>
# discard every uncommitted change
git restore .
# restore a file as it was at a commit
git restore --source=<commit> <file>
# undo a commit with a new commit (safe when shared)
git revert <commit>
# stage the revert without committing
git revert -n <commit>
# make the local branch match the remote (fetch first)
git reset --hard origin/<branch>
# find the lost commit...
git reflog
# bring a lost commit back on a new branch
git switch -c <branch> <commit>
```
`git reset` modes at a glance:
| Mode | Moves `HEAD` | Index (staging) | Working tree | Use it to |
|-----------|--------------|-----------------|--------------|----------------------------------------------|
| `--soft` | ✅ | untouched | untouched | Recommit differently, keep everything staged |
| `--mixed` | ✅ | reset | untouched | Unstage but keep the edits (default) |
| `--hard` | ✅ | reset | **reset** | Throw the changes away entirely ⚠️ |
> 💡 Nothing committed is really lost for ~90 days: `git reflog` lists every commit `HEAD` pointed at, even on deleted branches.
## 🧹 Cleaning
```bash
# dry run: what would be removed
git clean -n
```
Remove untracked files and directories
```shell
# remove untracked files and directories
git clean -fd
# also remove ignored files (build output, node_modules)
git clean -fdx
# compress and tidy the object database
git gc
# drop unreachable objects
git prune
# check the repository for corruption
git fsck
```
## Miscellaneous
## 🐛 Debugging
Renaming the local master branch to main
```shell
git branch -m master main
```
Checking log graph
```shell
git log --graph
```
Checking log graph (merges only)
```shell
git log --graph --merges
```
Tracking down a bad commit using binary search
```shell
```bash
# begin a binary search for a bad commit
git bisect start
git bisect good 13c988d4f15e06bcdd0b0af290086a3079cdadb0
git bisect bad ca82a6dff817ec66f44342007202690a93763949
# a commit that is broken (often HEAD)
git bisect bad <commit>
# a commit that was fine
git bisect good <commit>
# let a test script decide automatically
git bisect run <command>
# end the search and return to the original HEAD
git bisect reset
# follow lines moved from other files
git blame -C <file>
```
Pulling new changes into current branch from mainline
```shell
git checkout [branch-name]
git fetch origin [master-branch-name]
git rebase origin/[master-branch-name]
## 🧩 Submodules & Worktrees
```bash
# add a submodule
git submodule add <url> <path>
# check out every submodule
git submodule update --init --recursive
# update submodules to their latest commit
git submodule update --remote
# commit each submodule sits on
git submodule status
# check out a second branch side by side
git worktree add ../<dir> <branch>
# list linked working trees
git worktree list
# remove one again
git worktree remove ../<dir>
```
## 🧠 Miscellaneous
```bash
# rename the local default branch
git branch -m master main
# graph of merge commits only
git log --graph --merges
# export a snapshot without .git
git archive -o release.zip HEAD
# contributions per author
git shortlog -sn --no-merges
# full SHA of the current commit
git rev-parse HEAD
# name of the current branch
git rev-parse --abbrev-ref HEAD
# repository size on disk
git count-objects -vH
# enable background repacking (Git 2.30+)
git maintenance start
```
Update a feature branch with the latest mainline:
```bash
git switch <branch>
git fetch origin <main-branch>
git rebase origin/<main-branch> # or: git merge origin/<main-branch>
```
## 📚 Resources
- [Official documentation](https://git-scm.com/docs)
- [Pro Git (free book)](https://git-scm.com/book)
- [Interactive branching tutorial](https://learngitbranching.js.org)
- [Dangit, Git!?!](https://dangitgit.com) — recovering from the classic mistakes
@@ -1,163 +1,186 @@
# Conventional Commits Cheatsheet
**Conventional Commits** is a convention for commit messages that both humans and tools can read: the type tells you what kind of change it is, and release tooling turns that into version numbers and changelogs automatically.
## Structure
A conventional commit message follows this structure:
```
<type>[optional scope]: <description>
```text
<type>[optional scope][!]: <description>
[optional body]
[optional footer(s)]
```
## Elements
### Type (Required)
The type describes the kind of change being made. Common types include:
- **feat**: A new feature for the user
- **fix**: A bug fix for the user
- **docs**: Documentation changes
- **style**: Code style changes (formatting, missing semicolons, etc.)
- **refactor**: Code changes that neither fix a bug nor add a feature
- **test**: Adding or updating tests
- **chore**: Maintenance tasks, dependency updates, build changes
- **perf**: Performance improvements
- **ci**: Changes to CI/CD configuration
- **build**: Changes to build system or external dependencies
- **revert**: Reverting a previous commit
### Scope (Optional)
The scope provides additional context about what part of the codebase is affected:
```text
feat(auth): add OAuth2 login
```
## 🏷 Types
| Type | Use it for | Release effect |
|------------|---------------------------------------------------------|----------------|
| `feat` | A new feature for the user | Minor |
| `fix` | A bug fix for the user | Patch |
| `docs` | Documentation only | None |
| `style` | Formatting, whitespace, semicolons — no logic change | None |
| `refactor` | Code change that neither fixes a bug nor adds a feature | None |
| `perf` | A change that improves performance | Patch |
| `test` | Adding or correcting tests | None |
| `build` | Build system or dependencies | None |
| `ci` | CI configuration and scripts | None |
| `chore` | Maintenance that does not touch src or tests | None |
| `revert` | Reverting an earlier commit | Depends |
> 💡 Any type with a `!` or a `BREAKING CHANGE:` footer triggers a **major** release, `feat` included.
## 🎯 Scope
The scope is an optional noun in parentheses naming the part of the codebase affected. Keep the list short and consistent — package names, modules, or layers.
```text
feat(auth): add OAuth2 integration
fix(api): resolve timeout issues
docs(readme): update installation instructions
fix(api): resolve timeout on slow upstreams
docs(readme): update the installation steps
refactor(parser)!: drop support for the legacy format
```
### Description (Required)
A brief description of the change:
## ✍️ Description
- Use imperative mood ("add" not "added" or "adds")
- Keep it concise (50 characters or less recommended)
- Don't capitalize the first letter
- Don't end with a period
| Rule | ✅ Good | ❌ Avoid |
|--------------------------------|-----------------------------|----------------------------------|
| Imperative mood | `add user export` | `added user export` |
| Lower case, no trailing period | `fix flaky login test` | `Fix flaky login test.` |
| Say what changed, not where | `fix off-by-one in paging` | `fix bug in file` |
| Around 50 characters | `feat(api): add rate limit` | a full sentence with sub-clauses |
### Body (Optional)
Provides more detailed explanation of the change:
## 📝 Body & Footers
- Separate from description with a blank line
- Explain the motivation and contrast with previous behavior
- Use imperative mood
The body explains **why**, not how — the diff already shows how. Separate it from the description with a blank line and wrap at ~72 characters.
### Footer (Optional)
Contains metadata about the commit:
Footers come last, one per line:
- **Breaking changes**: Start with `BREAKING CHANGE:`
- **Issue references**: `Closes #123`, `Fixes #456`
- **Co-authors**: `Co-authored-by: Name <email>`
| Footer | Meaning |
|--------------------------------|-----------------------------------------------|
| `BREAKING CHANGE: <what>` | Incompatible change, forces a major release |
| `Closes #123` / `Fixes #456` | Closes the issue when merged (GitHub, GitLab) |
| `Refs #789` | Related, but does not close it |
| `Co-authored-by: Name <email>` | Credits a second author |
| `Reviewed-by: Name <email>` | Records the reviewer |
## Examples
## 💥 Breaking Changes
### Simple commit
Two ways to mark one — the `!` is visible in `git log --oneline`, the footer explains the migration. Use both.
```text
feat(api)!: require an email address when shipping
BREAKING CHANGE: POST /orders now rejects requests without a customer
email. Add the field before upgrading; see docs/migrations/2026-08.md.
```
## 📄 Examples
```text
feat: add user authentication
```
### With scope
```
fix(parser): handle edge case in JSON parsing
```text
fix(parser): handle a trailing comma in JSON input
```
### With body
```
```text
feat: add email notifications
Users can now receive email notifications for important events.
This includes account changes, security alerts, and system updates.
Users can now receive email notifications for account changes,
security alerts and system updates. Delivery is queued so a slow
SMTP server never blocks the request.
```
### With footer
```
```text
fix: prevent racing of requests
Introduce a request id and a reference to latest request. Dismiss
incoming responses other than from latest request.
Introduce a request id and a reference to the latest request.
Dismiss incoming responses other than from the latest request.
Closes #123
```
### Breaking change
```
feat!: send an email to the customer when a product is shipped
```text
feat(cart): add the ability to remove items
BREAKING CHANGE: The shipping service now requires an email address
```
### Full example
```
feat(shopping cart): add ability to remove items
Users can now remove items from their shopping cart by clicking
the remove button next to each item. This improves the user
experience by allowing corrections without starting over.
Users can remove items from the cart instead of starting over,
which was the most common complaint in support tickets.
Closes #456
Co-authored-by: Jane Doe <jane@example.com>
```
## Common Tools
## 🔢 Versioning
### Commitizen
Interactive tool for creating conventional commits:
Release tooling maps the history since the last tag onto a semantic version:
```shell
npm install -g commitizen
npm install -g cz-conventional-changelog
| Commits since the last release | New version from `1.4.2` |
|--------------------------------|--------------------------|
| Only `docs`, `chore`, `style`… | no release |
| At least one `fix` or `perf` | `1.4.3` |
| At least one `feat` | `1.5.0` |
| Any `!` or `BREAKING CHANGE:` | `2.0.0` |
## 🛠 Tooling
### Commitizen — a prompt instead of a blank editor
```bash
npm install -g commitizen cz-conventional-changelog
echo '{ "path": "cz-conventional-changelog" }' > ~/.czrc
```
Usage:
```shell
# then commit with
git cz
```
### Commitlint
Lints commit messages to ensure they follow conventional format:
### commitlint — reject messages that do not follow the convention
```shell
npm install --save-dev @commitlint/config-conventional @commitlint/cli
```bash
npm install --save-dev @commitlint/cli @commitlint/config-conventional
```
Configuration in `.commitlintrc.json`:
```json
{
"extends": ["@commitlint/config-conventional"]
}
```
### Husky
Git hooks to enforce commit message format:
### husky — run commitlint from a git hook
```shell
```bash
npm install --save-dev husky
npx husky add .husky/commit-msg 'npx --no -- commitlint --edit ${1}'
npx husky init
# husky v9+: write the hook file yourself
echo 'npx --no -- commitlint --edit "$1"' > .husky/commit-msg
```
### Semantic Release
Automatically generates releases based on conventional commits:
### Releases & changelogs
```shell
```bash
# decide the version, tag, and publish from the commit history
npm install --save-dev semantic-release
# or just generate the changelog
npx conventional-changelog-cli -p angular -i CHANGELOG.md -s
# a fast, language-agnostic alternative
git cliff --tag v1.5.0 --output CHANGELOG.md
```
### Conventional Changelog
Generates changelogs from conventional commits:
## ✅ Habits That Keep It Useful
```shell
npm install -g conventional-changelog-cli
conventional-changelog -p angular -i CHANGELOG.md -s
```
- One logical change per commit — if the description needs an "and", split it.
- Write the message for the person bisecting in six months, not for the linter.
- Keep the scope vocabulary small and documented in `CONTRIBUTING.md`.
- Never invent types the tooling does not know; `config` or `wip` silently drop out of the changelog.
- Squash-merging? The **PR title** becomes the commit message, so lint that too.
## 📚 Resources
- [Conventional Commits specification](https://www.conventionalcommits.org/)
- [commitlint](https://commitlint.js.org/)
- [semantic-release](https://semantic-release.gitbook.io/)
- [Semantic Versioning](https://semver.org/)
+181 -65
View File
@@ -1,89 +1,205 @@
# Processing JSON using jq
**jq** is a command-line JSON processor: it slices, filters, maps and transforms structured data the way `sed` and `awk` do for text. A jq program is a *filter* — it takes an input, and produces an output.
jq is useful to slice, filter, map and transform structured json data.
```bash
# pretty-print a file
jq . data.json
## Installing jq
# read from a pipe
curl -s https://api.example.com/users | jq '.[] | .name'
```
### On Mac OS
## 📦 Installation
`brew install jq`
```bash
brew install jq # macOS
apt install jq # Debian, Ubuntu
dnf install jq # Fedora, RHEL
choco install jq # Windows
```
### On AWS Linux
## 🚩 Command-Line Flags
Not available as yum install on our current AMI. It should be on the latest AMI though: https://aws.amazon.com/amazon-linux-ami/2015.09-release-notes/
| Flag | Description |
|--------------------------|------------------------------------------------------------------|
| `-r`, `--raw-output` | Print strings without quotes — what you want for shell variables |
| `-c`, `--compact-output` | One compact line per result instead of pretty-printed JSON |
| `-n`, `--null-input` | Do not read input; build JSON from scratch |
| `-s`, `--slurp` | Read the whole input into one array |
| `-e`, `--exit-status` | Exit non-zero when the last output is `false` or `null` |
| `-j`, `--join-output` | Raw output with no newline between results |
| `--tab` / `--indent n` | Indent with tabs, or with `n` spaces |
| `-S`, `--sort-keys` | Sort object keys in the output |
| `--arg name value` | Pass a string into the program as `$name` |
| `--argjson name json` | Pass parsed JSON in as `$name` |
| `--slurpfile name f` | Read a whole file into `$name` as an array |
| `--raw-input`, `-R` | Treat each input line as a string instead of JSON |
Installing from the source proved to be tricky.
## 🧭 Core Syntax
## Useful arguments
| Filter | Description |
|--------------|---------------------------------------------------------------|
| `.` | The identity filter — the input, unchanged |
| `.foo` | The value of field `foo` |
| `.foo.bar` | Nested field access |
| `.foo?` | Same, but stays quiet when the input is not an object |
| `."odd key"` | A field whose name needs quoting |
| `.[]` | Every element of an array (or every value of an object) |
| `.[0]` | One element by index (`-1` is the last) |
| `.[2:4]` | A slice — see the table below |
| `\|` | Pipe: feed the result of the left filter into the right one |
| `,` | Run both filters on the same input, emit both results |
| `()` | Grouping |
| `//` | Alternative: use the right side if the left is `false`/`null` |
| `?` | Suppress errors from the preceding filter |
When running jq, the following arguments may become handy:
## 📇 Objects
| Argument | Description |
| ------ | :--------- |
| `--version`| Output the jq version and exit with zero. |
| `--sort-keys` | Output the fields of each object with the keys in sorted order.|
| Task | Filter |
|-----------------------------|---------------------------------------------------|
| List the keys | `jq 'keys'` (`keys_unsorted` to keep order) |
| Does a key exist? | `jq 'has("foo")'` |
| Pick a few fields | `jq '{name, id}'` |
| Rename while picking | `jq '{title: .name, ref: .id}'` |
| Delete a key | `jq 'del(.foo)'` |
| Add 1 to every value | `jq 'map_values(.+1)'` |
| Merge two objects | `jq '.a * .b'` (deep) or `jq '.a + .b'` (shallow) |
| Object → array of pairs | `jq 'to_entries'` |
| Array of pairs → object | `jq 'from_entries'` |
| Transform every entry | `jq 'with_entries(.value += 1)'` |
| Build an object from values | `jq -n '{time: now, host: $ENV.HOSTNAME}'` |
## Basic concepts
## 📚 Arrays
The syntax for jq is pretty coherent:
### Slicing and filtering
| Syntax | Description |
| ------ | :--------- |
| , | Filters separated by a comma will produce multiple independent outputs|
| ? | Will ignores error if the type is unexpected |
| [] | Array construction |
| {} | Object construction |
| + | Concatenate or Add |
| - | Difference of sets or Substract |
| length | Size of selected element |
| &#124; | Pipes are used to chain commands in a similar fashion than bash|
| Task | Filter |
|------------------------------|--------------------------------------------------------------------------------------------------------------------|
| Every element | `jq '.[]'` |
| First / last | `jq '.[0]'` / `jq '.[-1]'` |
| Range, first three, last two | `jq '.[2:4]'`, `jq '.[:3]'`, `jq '.[-2:]'` |
| Numbers above a threshold | `jq 'map(select(. >= 2))'` |
| Objects matching a field | `jq '.[] \| select(.id == "second")'` |
| Several conditions | `jq '.[] \| select(.age > 18 and .city == "Oslo")'` |
| By type | `jq '.[] \| numbers'` — also `strings`, `booleans`, `nulls`, `arrays`, `objects`, `iterables`, `scalars`, `values` |
| Test a string field | `jq '.[] \| select(.name \| test("^a"; "i"))'` |
### Mapping and transforming
## Dealing with json objects
| Task | Filter |
|----------------------------|---------------------------------------------|
| Apply to every element | `jq 'map(.+1)'` |
| Pull one field out of each | `jq 'map(.name)'` or `jq '.[].name'` |
| Drop elements by index | `jq 'del(.[1, 2])'` |
| Concatenate nested arrays | `jq 'add'` |
| Flatten | `jq 'flatten'` (`flatten(1)` for one level) |
| Sort | `jq 'sort'` / `jq 'sort_by(.foo)'` |
| Group | `jq 'group_by(.foo)'` |
| Deduplicate | `jq 'unique'` / `jq 'unique_by(.foo)'` |
| Reverse | `jq 'reverse'` |
| Smallest / largest | `jq 'min'`, `jq 'max_by(.price)'` |
| Count | `jq 'length'` |
| Sum | `jq 'map(.amount) \| add'` |
| Any / all match | `jq 'any(.active)'`, `jq 'all(.age > 18)'` |
| A range of numbers | `jq '[range(2;4)]'` |
| The type of each item | `jq 'map(type)'` |
| Description | Command |
| ------ | :--------- |
| Display all keys | `jq 'keys'` |
| Adds + 1 to all items | `jq 'map_values(.+1)'` |
| Delete a key| `jq 'del(.foo)'` |
| Convert an object to array | `to_entries &#124; map([.key, .value])` |
## 🔤 Strings & Formatting
## Dealing with fields
| Task | Filter |
|---------------------------|------------------------------------------------|
| Interpolate | `jq -r '"\(.name) is \(.age)"'` |
| Change case | `jq 'ascii_downcase'` / `ascii_upcase` |
| Split and join | `jq 'split(",")'` / `jq 'join(", ")'` |
| Trim a prefix or suffix | `jq 'ltrimstr("v")'` / `rtrimstr(".json")` |
| Match a regex | `jq 'test("^http")'` |
| Replace | `jq 'sub("^v"; "")'` / `gsub("\\s+"; "-")` |
| Capture named groups | `jq 'capture("(?<host>[^:]+):(?<port>\\d+)")'` |
| CSV / TSV output | `jq -r '.[] \| [.id, .name] \| @csv'` |
| Escape for a URL or shell | `jq -r '@uri "?q=\(.term)"'`, `@sh` |
| Base64 | `jq -r '@base64'` / `jq -r '@base64d'` |
| Object → JSON string | `jq 'tojson'` (and `fromjson` back) |
| Description | Command |
| ------ | :--------- |
| Concatenate two fields| `fieldNew=.field1+' '+.field2` |
## 🔀 Conditionals & Errors
```bash
# if / then / else — 'end' is required
jq 'if .age >= 18 then "adult" else "minor" end'
## Dealing with json arrays
# several branches
jq 'if .n > 100 then "big" elif .n > 10 then "medium" else "small" end'
### Slicing and Filtering
# fall back when a value is missing or null
jq '.nickname // .name'
| Description | Command |
| ------ | :--------- |
| All | `jq .[]` |
| First | `jq '.[0]'` |
| Range | `jq '.[2:4]'` |
| First 3 | `jq '.[:3]'` |
| Last 2 | `jq '.[-2:]'` |
| Before Last | `jq '.[-2]'`|
| Select array of int by value | `jq 'map(select(. >= 2))'` |
| Select array of objects by value| `jq '.[] &#124; select(.id == "second")'` |
| Select by type | `jq '.[] &#124; numbers'` with type been arrays, objects, iterables, booleans, numbers, normals, finites, strings, nulls, values, scalars |
# keep going when a filter would fail
jq '.items[]? // empty'
### Mapping and Transforming
# turn a failure into a value of your own
jq 'try (.a.b.c) catch "not found"'
| Description | Command |
| ------ | :--------- |
| Add + 1 to all items | `jq 'map(.+1)'` |
| Delete 2 items| `jq 'del(.[1, 2])'` |
| Concatenate arrays | `jq 'add'` |
| Flatten an array | `jq 'flatten'` |
| Create a range of numbers | `jq '[range(2;4)]'` |
| Display the type of each item| `jq 'map(type)'` |
| Sort an array of basic type| `jq 'sort'` |
| Sort an array of objects | `jq 'sort_by(.foo)'` |
| Group by a key - opposite to flatten | `jq 'group_by(.foo)'` |
| Minimun value of an array| `jq 'min'` .See also min, max, min_by(path_exp), max_by(path_exp) |
| Remove duplicates| `jq 'unique'` or `jq 'unique_by(.foo)'` or `jq 'unique_by(length)'` |
| Reverse an array | `jq 'reverse'` |
# stop with a message
jq 'if .id == null then error("id is required") else . end'
```
## 🧮 Variables, Functions & Reduction
```bash
# bind a value to a variable
jq '.items[] as $item | $item.name'
# pass values in from the shell
jq --arg env prod '.deploys[] | select(.env == $env)'
jq --argjson min 10 '.[] | select(.count > $min)'
# define a reusable function
jq 'def is_active: .status == "active"; map(select(is_active))'
# accumulate a value
jq 'reduce .[] as $x (0; . + $x.amount)'
# keep the intermediate results too
jq '[foreach .[] as $x (0; . + $x; .)]'
# read the environment
jq -n 'env.HOME'
```
## 🛠 Everyday Recipes
```bash
# pretty-print, sorted, and back to a file
jq -S . data.json > sorted.json
# one field, unquoted, for use in a shell variable
version=$(jq -r .version package.json)
# every key of every object, deduplicated
jq -r '[.[] | keys[]] | unique[]' data.json
# turn an array of objects into a CSV with a header row
jq -r '(.[0] | keys_unsorted), (.[] | [.[]]) | @csv' data.json
# merge two JSON files
jq -s '.[0] * .[1]' a.json b.json
# read newline-delimited JSON (one object per line)
jq -c '.event' events.ndjson
# turn plain text lines into a JSON array
printf 'a\nb\n' | jq -R -s 'split("\n") | map(select(length > 0))'
# find the path to every occurrence of a key
jq -c 'paths(scalars) as $p | {path: $p, value: getpath($p)}' data.json
# rewrite every string in a document
jq 'walk(if type == "string" then ascii_downcase else . end)' data.json
# use the exit status in a script
jq -e '.errors | length == 0' report.json && echo "clean"
```
## 📚 Resources
- [Manual](https://jqlang.github.io/jq/manual/)
- [Try jq in the browser (jqplay)](https://jqplay.org)
- [Tutorial](https://jqlang.github.io/jq/tutorial/)
+107 -38
View File
@@ -1,52 +1,121 @@
## Syntax
**JSONPath** is a query language for JSON, the way XPath is for XML. An expression walks the document from the root and returns every value that matches.
Depending on the client used JSONPath expressions do start with `$.` indicating the root element.
Some clients omit the leading `$.`.
> ℹ️ Expressions normally start with `$`, the root element. Some clients let you drop the leading `$.`. Implementations differ in the details — [RFC 9535](https://www.rfc-editor.org/rfc/rfc9535.html) standardised the syntax in 2024, but older libraries predate it.
| Syntax | Description |
| ------ | :--------- |
| `$.store.book[0].title` | |
| `store.book[0].title` | With implicit `$.` |
| `$['store']['book'][0]['title']` | Alternative notation similar to scripting languages |
## 🧭 Syntax
## Tree Traversal
| Syntax | Description |
|----------------------------------|----------------------------------------------------------|
| `$` | The root element |
| `@` | The element currently being filtered |
| `$.store.book[0].title` | Dot notation |
| `store.book[0].title` | With the `$.` implied |
| `$['store']['book'][0]['title']` | Bracket notation — needed for keys with spaces or dashes |
| `..` | Recursive descent, at any depth |
| `*` | Wildcard: every element or member |
| `[]` | Subscript: index, slice, union or filter |
| Syntax | Description |
| ------ | :--------- |
| `$.parentNode.childNode.field` | XPath: `/parentNode/childNode/@field` (content of "field" of all "childNode"s of "parentNode") |
| `$..anyChildNode` | XPath: `//anyChildNode` (all children at any depth named "anyChildNode") |
| `$.parentNode.*` | XPath: `/parentNode/*` (all children below) |
## 🌳 Tree Traversal
## Array Access
| Syntax | Description |
|--------------------------------|--------------------------------------------------------------------------|
| `$.parentNode.childNode.field` | XPath `/parentNode/childNode/@field` — the field of every matching child |
| `$..anyChildNode` | XPath `//anyChildNode` — every node with that name, at any depth |
| `$.parentNode.*` | XPath `/parentNode/*` — every child of the node |
| `$..*` | Every value in the document |
| `$..book[*].author` | The author of every book, however deeply nested |
| Syntax | Description |
| ------ | :--------- |
| `$.myList[0]` | first element |
| `$.myList[-1]` | last element |
| `$.myList[2:3]` | range |
| `$.myList[0,4,5]` | selection |
## 🔢 Array Access
## Filtering
| Syntax | Description |
|-------------------|-----------------------------------------|
| `$.myList[0]` | First element |
| `$.myList[-1]` | Last element |
| `$.myList[*]` | Every element |
| `$.myList[2:4]` | Slice — index 2 up to, not including, 4 |
| `$.myList[:3]` | The first three |
| `$.myList[-2:]` | The last two |
| `$.myList[::2]` | Every second element |
| `$.myList[0,4,5]` | A union of specific indexes |
| Syntax | Description |
| ------ | :--------- |
| `$.customer[?(@.car)]` | Only "customer"s that have attribute "car" |
| `$.customer[?(@.car == 'Ford Fiesta')]` | Only "customer"s with "Ford Fiesta"s |
| `$.customer[?(@.age > 18)]` | Only adults |
## 🔎 Filtering
## Complex Conditions
Filters are written `[?(...)]` — `[?...]` in RFC 9535 — and keep the elements for which the expression is true.
| Syntax | Description |
| ------ | :--------- |
| `$.customer[?(@.age > 18 \|\| @.car == 'Ford Fiesta')]` | logical or |
| `$.customer[?(@.age < 18 && @.hobby == 'Biking' )]` | logical and |
| Syntax | Description |
|-----------------------------------------|----------------------------------------------------|
| `$.customer[?(@.car)]` | Customers that have a `car` field at all |
| `$.customer[?(@.car == 'Ford Fiesta')]` | Customers with a Ford Fiesta |
| `$.customer[?(@.age > 18)]` | Adults only |
| `$.customer[?(@.age != 18)]` | Everyone but the 18-year-olds |
| `$.book[?(@.price <= 10)]` | Also `<`, `>=`, `==` |
| `$.book[?(@.title =~ /^The/)]` | Regular expression match (implementation-specific) |
## Output Mapping
### Combining conditions
| Syntax | Description |
| ------ | :--------- |
| `$.[].{Name:name, Age:age, Hobbies:details.hobbies}` | Mapping fields/nested fields to new set
| Syntax | Description |
|---------------------------------------------------------|-------------|
| `$.customer[?(@.age > 18 \|\| @.car == 'Ford Fiesta')]` | Logical or |
| `$.customer[?(@.age < 18 && @.hobby == 'Biking')]` | Logical and |
| `$.customer[?(!@.car)]` | Logical not |
## Credits
## 🧮 Functions
Original author: https://gist.github.com/mackoj/5786f8b95da0a82e8e003f444c4295bf
Available in RFC 9535 and in most modern implementations:
| Function | Example | Returns |
|------------|---------------------------------------|----------------------------------------|
| `length()` | `$.book[?(length(@.tags) > 2)]` | Length of a string, array or object |
| `count()` | `$.book[?(count(@.authors.*) > 1)]` | Number of nodes a query matched |
| `match()` | `$.book[?(match(@.isbn, "\\d{13}"))]` | True when the **whole** string matches |
| `search()` | `$.book[?(search(@.title, "Moon"))]` | True when the pattern occurs anywhere |
| `value()` | `$.book[?(value(@.price) > 10)]` | The value of a single-node query |
Older Jayway-style implementations instead offer `.length()`, `.min()`, `.max()`, `.avg()` and `.sum()` at the end of a path — `$.book[*].price.sum()`.
## 📄 Worked Example
```json
{
"store": {
"book": [
{ "title": "Sayings of the Century", "author": "Nigel Rees", "price": 8.95 },
{ "title": "Moby Dick", "author": "Herman Melville", "price": 8.99 },
{ "title": "The Lord of the Rings", "author": "J. R. R. Tolkien", "price": 22.99 }
],
"bicycle": { "color": "red", "price": 19.95 }
}
}
```
| Expression | Result |
|-------------------------------------|---------------------------------------|
| `$.store.book[*].author` | All three authors |
| `$..price` | `8.95`, `8.99`, `22.99`, `19.95` |
| `$.store.book[?(@.price < 10)]` | The two cheap books |
| `$.store.book[-1].title` | `"The Lord of the Rings"` |
| `$..book[?(@.author =~ /Tolkien/)]` | The Tolkien entry |
| `$.store.*` | The book array and the bicycle object |
## 🗺 Output Mapping
| Syntax | Description |
|------------------------------------------------------|-------------------------------------------------------------------------------------------|
| `$.[].{Name:name, Age:age, Hobbies:details.hobbies}` | Map fields and nested fields into a new object (JMESPath-style, not supported everywhere) |
## 🧰 Where You'll Meet It
- `kubectl get pods -o jsonpath='{.items[*].metadata.name}'`
- Jayway JSONPath in Java, REST Assured assertions
- Karate, Postman and Newman test scripts
- Nginx, Grafana and Prometheus JSON scrapers
> 💡 For heavier transformation on the command line, `jq` is usually the better tool — JSONPath selects, `jq` selects *and* reshapes.
## 📚 Resources
- [RFC 9535 — JSONPath](https://www.rfc-editor.org/rfc/rfc9535.html)
- [Try expressions online (jsonpath.com)](https://jsonpath.com)
- [Comparison of implementations](https://cburgmer.github.io/json-path-comparison/)
Original author: https://gist.github.com/mackoj/5786f8b95da0a82e8e003f444c4295bf
+264 -143
View File
@@ -1,183 +1,304 @@
# Kubernetes
**kubectl** is the CLI for the Kubernetes API. Almost every command follows the same shape: `kubectl <verb> <resource> <name> [flags]` — `get pods`, `describe node web-1`, `delete deployment api`.
* PDF: https://sematext.com/kubernetes-cheat-sheet/
* WEBSITE: https://kubernetes.io/
* DOCUMENTATION: https://kubernetes.io/docs/home
> 💡 Set up completion and an alias once and everything below gets shorter: `source <(kubectl completion bash)`, `alias k=kubectl`, `complete -o default -F __start_kubectl k`.
## Client Configuration
## ⚙️ Configuration & Context
* Setup autocomplete in bash; bash-completion package should be installed first
```
```bash
# show the merged kubeconfig
kubectl config view
# read a single value out of it
kubectl config view -o jsonpath='{.clusters[*].name}'
# which cluster am I talking to?
kubectl config current-context
# list every context
kubectl config get-contexts
# switch clusters
kubectl config use-context <context>
# change the default namespace for this context
kubectl config set-context --current --namespace=<namespace>
# use a different kubeconfig for one command
KUBECONFIG=~/.kube/staging.yaml kubectl get nodes
# shell completion (bash; use 'zsh' or 'fish' as needed)
source <(kubectl completion bash)
```
* View Kubernetes config
```
kubectl config view
```
## 🔍 Viewing & Finding Resources
* View specific config items by json path
```
kubectl config view -o jsonpath='{.users[?(@.name == "k8s")].user.password}'
```
```bash
# the everyday listing
kubectl get pods
kubectl get pods -o wide
kubectl get pods --all-namespaces
* Set credentials for `foo.kuberntes.com`
```
kubectl config set-credentials kubeuser/foo.kubernetes.com --username=kubeuser --password=kubepassword
```
# every kind of resource in a namespace
kubectl get all -n <namespace>
* Set active namespace
```
kubectl config set-context --current --namespace=namespace_name
```
# full definition, as stored by the API server
kubectl get pod <pod> -o yaml
## Viewing, Finding resources
# just the fields you care about
kubectl get pods -o custom-columns='NAME:.metadata.name,NODE:.spec.nodeName'
* List all services in the namespace
```
kubectl get services
```
# pull one value out with JSONPath
kubectl get pods -o jsonpath='{.items[*].metadata.name}'
* List all pods in all namespaces in wide format
```
kubectl get pods -o wide --all-namespaces
```
# filter by label, or by field
kubectl get pods -l app=nginx,tier=frontend
kubectl get pods --field-selector status.phase=Running
* List all pods in json (or yaml) format
```
kubectl get pods -o json
```
* Describe resource details (node, pod, svc)
```
kubectl describe nodes my-node
```
* List services sorted by name
```
# sort the output
kubectl get services --sort-by=.metadata.name
```
* List pods sorted by restart count
```
kubectl get pods --sort-by='.status.containerStatuses[0].restartCount'
# watch changes as they happen
kubectl get pods -w
# events, newest last — the first place to look when something is stuck
kubectl get events --sort-by=.lastTimestamp
# the long-form story of one object
kubectl describe pod <pod>
# what resources exist, and what are their short names?
kubectl api-resources
# what fields does this kind have?
kubectl explain deployment.spec.template.spec.containers
```
* Rolling update pods for frontend-v1
```
kubectl rolling-update frontend-v1 -f frontend-v2.json
## 🚀 Creating, Applying & Deleting
```bash
# apply a manifest, creating or updating as needed
kubectl apply -f ./manifest.yaml
# apply everything in a directory, recursively
kubectl apply -R -f ./k8s/
# apply a kustomize overlay
kubectl apply -k ./overlays/production
# generate a manifest instead of creating anything
kubectl create deployment web --image=nginx --dry-run=client -o yaml > web.yaml
# quick one-off objects
kubectl create deployment web --image=nginx:1.27
kubectl run debug --image=busybox -it --rm -- sh
kubectl expose deployment web --port=80 --target-port=8080 --type=ClusterIP
# see what an apply would change before doing it
kubectl diff -f ./manifest.yaml
# edit an object in your $EDITOR
kubectl edit deployment web
# change one field without an editor
kubectl patch deployment web -p '{"spec":{"replicas":4}}'
# delete
kubectl delete -f ./manifest.yaml
kubectl delete pod <pod> --grace-period=0 --force
```
* Scale a replicaset named 'foo' to 3
```
kubectl scale --replicas=3 rs/foo
## 📈 Scaling & Rollouts
```bash
# scale a deployment, or scale from the manifest
kubectl scale deployment web --replicas=3
kubectl scale --replicas=3 -f web.yaml
# only scale if it currently has 3 replicas
kubectl scale --current-replicas=3 --replicas=5 deployment/web
# horizontal autoscaling
kubectl autoscale deployment web --min=2 --max=10 --cpu-percent=70
# roll out a new image
kubectl set image deployment/web nginx=nginx:1.27
# watch the rollout
kubectl rollout status deployment/web
# rollout history, and going back
kubectl rollout history deployment/web
kubectl rollout undo deployment/web
kubectl rollout undo deployment/web --to-revision=2
# restart every pod of a deployment (picks up new config or secrets)
kubectl rollout restart deployment/web
# pause and resume a rollout
kubectl rollout pause deployment/web
kubectl rollout resume deployment/web
```
* Scale a resource specified in "foo.yaml" to 3
```
kubectl scale --replicas=3 -f foo.yaml
## 📜 Logs & Debugging
```bash
# logs of a pod
kubectl logs <pod>
# follow them
kubectl logs -f <pod>
# a specific container in a multi-container pod
kubectl logs -f <pod> -c <container>
# what the previous, crashed container printed
kubectl logs <pod> --previous
# recent output only
kubectl logs <pod> --since=10m --tail=100
# logs from every pod behind a label
kubectl logs -l app=web --all-containers --max-log-requests=10
# a shell inside a running container
kubectl exec -it <pod> -- sh
kubectl exec -it <pod> -c <container> -- bash
# a one-off command
kubectl exec <pod> -- env
# copy files in and out
kubectl cp <pod>:/var/log/app.log ./app.log
kubectl cp ./config.yaml <pod>:/etc/app/config.yaml
# reach a service or pod from your laptop
kubectl port-forward pod/<pod> 8080:80
kubectl port-forward svc/<service> 8080:80
# attach a debug container to a running pod (distroless-friendly)
kubectl debug -it <pod> --image=busybox --target=<container>
# a throwaway pod on a specific node
kubectl debug node/<node> -it --image=busybox
# resource usage (needs metrics-server)
kubectl top node
kubectl top pod
kubectl top pod <pod> --containers
```
* Execute a command in every pod / replica
```
for i in 0 1; do kubectl exec foo-$i -- sh -c 'echo $(hostname) > /usr/share/nginx/html/index.html'; done
## 🔐 Secrets & ConfigMaps
```bash
# create a secret from literals or files
kubectl create secret generic db-creds \
--from-literal=username=jane \
--from-literal=password=s3cr3t
kubectl create secret generic tls-key --from-file=./tls.key
# a registry pull secret
kubectl create secret docker-registry regcred \
--docker-server=ghcr.io --docker-username=<user> --docker-password=<token>
# configmaps work the same way
kubectl create configmap app-config --from-file=./config.yaml
kubectl create configmap app-config --from-literal=LOG_LEVEL=debug
# read a secret back (values are base64)
kubectl get secret db-creds -o jsonpath='{.data.password}' | base64 -d
# every key at once
kubectl get secret db-creds -o go-template='{{range $k,$v := .data}}{{$k}}={{$v | base64decode}}{{"\n"}}{{end}}'
```
## Manage Resources
## 🏷 Labels, Annotations & Namespaces
* Get documentation for pod or service
```
kubectl explain pods,svc
```bash
# add or change a label (--overwrite to replace an existing one)
kubectl label pod <pod> env=prod --overwrite
# remove a label
kubectl label pod <pod> env-
# annotations work the same way
kubectl annotate deployment web kubernetes.io/change-cause="bump to 1.27"
# show the labels in the listing
kubectl get pods --show-labels
# namespaces
kubectl get namespaces
kubectl create namespace staging
kubectl delete namespace staging
```
* Create resource(s) like pods, services or daemonsets
```
kubectl create -f ./my-manifest.yaml
## 🖥 Nodes & Cluster
```bash
# nodes and their state
kubectl get nodes -o wide
kubectl describe node <node>
# stop scheduling new pods onto a node
kubectl cordon <node>
# evict the pods and prepare for maintenance
kubectl drain <node> --ignore-daemonsets --delete-emptydir-data
# put it back into service
kubectl uncordon <node>
# keep pods off a node unless they tolerate the taint
kubectl taint nodes <node> key=value:NoSchedule
# where is the control plane?
kubectl cluster-info
# what API versions does the cluster serve?
kubectl api-versions
# am I allowed to do this?
kubectl auth can-i create deployments --namespace production
```
* Apply a configuration to a resource
```
kubectl apply -f ./my-manifest.yaml
```
## 🧰 Handy One-Liners
* Start a single instance of Nginx
```
kubectl run nginx --image=nginx
```
```bash
# every pod that is not Running
kubectl get pods -A --field-selector=status.phase!=Running
* Create a secret with several keys
```
cat <<EOF | kubectl create -f -
# the ten pods that restarted most
kubectl get pods -A --sort-by='.status.containerStatuses[0].restartCount' | tail -10
# clean up evicted pods
kubectl get pods -A --field-selector=status.phase=Failed -o name | xargs -r kubectl delete
# which node is each pod on
kubectl get pods -o custom-columns='POD:.metadata.name,NODE:.spec.nodeName'
# images running in the cluster
kubectl get pods -A -o jsonpath='{.items[*].spec.containers[*].image}' | tr ' ' '\n' | sort -u
# run a command in several pods
for pod in $(kubectl get pods -l app=web -o name); do kubectl exec "$pod" -- hostname; done
# apply a manifest straight from a heredoc
kubectl apply -f - <<EOF
apiVersion: v1
kind: Secret
kind: ConfigMap
metadata:
name: mysecret
type: Opaque
name: demo
data:
password: $(echo "s33msi4" | base64)
username: $(echo "jane"| base64)
LOG_LEVEL: debug
EOF
```
* Delete a resource
```
kubectl delete -f ./my-manifest.yaml
```
## 📚 Resources
## Monitoring & Logging
- [kubectl reference](https://kubernetes.io/docs/reference/kubectl/)
- [kubectl quick reference](https://kubernetes.io/docs/reference/kubectl/quick-reference/)
- [Documentation home](https://kubernetes.io/docs/home/)
- [JSONPath support in kubectl](https://kubernetes.io/docs/reference/kubectl/jsonpath/)
* Deploy Heapster from Github repository
```
kubectl create -f deploy/kube-config/standalone/
```
* Show metrics for nodes
```
kubectl top node
```
* Show metrics for pods
```
kubectl top pod
```
* Show metrics for a given pod and its containers
```
kubectl top pod pod_name --containers
```
* Dump pod logs (stdout)
```
kubectl logs pod_name
```
* Stream pod container logs (stdout, multi-container case)
```
kubectl logs -f pod_name -c my-container
```
## Interacting with running pods
* Run command in pod
```
kubectl exec pod_name -- command_name
```
* Run command in pod with multiple containers
```
kubectl exec pod_name -c container_name -- command_name
```
* Get terminal of pod
```
kubectl exec -it pod_name /bin/sh
```
* Get terminal of a container running in pod with multiple containers
```
kubectl exec -it pod_name -c container_name /bin/sh
```
### Credits
https://raw.githubusercontent.com/LeCoupa/awesome-cheatsheets/refs/heads/master/tools/kubernetes.md
Original cheat sheet: https://github.com/LeCoupa/awesome-cheatsheets
+141 -100
View File
@@ -1,109 +1,150 @@
Overview of nano's shortcuts      
=====================================
**GNU nano** is a small, modeless terminal editor: you start typing straight away, and every command is a control or meta chord shown along the bottom of the screen.
# The editor's keystrokes and their functions         
> ℹ️ nano's own help writes `^X` for `Ctrl+X` and `M-X` for `Alt+X` (Meta). This sheet spells them out. On macOS, `Alt` is usually `Esc` pressed first, or `Option` with "Use Option as Meta" enabled.
## File handling**
## 🚀 Starting nano
| Shortcut | Description |
|:--|:--|
| Ctrl+S | Save current file |
| Ctrl+O | Offer to write file ("Save as") |
| Ctrl+R | Insert a file into current one |
| Ctrl+X | Close buffer, exit from nano |
## **Editing**
| Shortcut | Description |
|:--|:--|
| Ctrl+K | Cut current line into cutbuffer |
| Alt+6 | Copy current line into cutbuffer |
| Ctrl+U | Paste contents of cutbuffer |
| Alt+T | Cut until end of buffer |
| Ctrl+\] | Complete current word |
| Alt+3 | Comment/uncomment line/region |
| Alt+U | Undo last action |
| Alt+E | Redo last undone action |
## **Search and replace**
| Shortcut | Description |
|:--|:--|
| Ctrl+Q | Start backward search |
| Ctrl+W | Start forward search |
| Alt+Q | Find next occurrence backward |
| Alt+W | Find next occurrence forward |
| Alt+R | Start a replacing session |
## Deletion
| Shortcut | Description |
|:--|:--|
| Ctrl+H | Delete character before cursor |      
| Ctrl+D | Delete character under cursor |
| Alt+Bsp | Delete word to the left |
| Ctrl+Del | Delete word to the right |
| Alt+Del | Delete current line |
## Operations
```bash
# open a file (created on save if it does not exist)
nano <file>
| Shortcut | Description |
|:--|:--|
| Ctrl+T | Execute some command |
| Ctrl+J | Justify paragraph or region |
| Alt+J | Justify entire buffer |
| Alt+B | Run a syntax check |
| Alt+F | Run a formatter/fixer/arranger |
| Alt+: | Start/stop recording of macro |
| Alt+; | Replay macro |
# jump straight to a line, or a line and column
nano +42 <file>
nano +42,8 <file>
## Moving around
# show line numbers
nano -l <file>
| Shortcut | Description |
|:--|:--|
| Ctrl+B | One character backward |
| Ctrl+F | One character forward |
| Ctrl+**←** | One word backward |
| Ctrl+**→** | One word forward |
| Ctrl+A | To start of line |
| Ctrl+E | To end of line |
| Ctrl+P | One line up |
| Ctrl+N | One line down |
| Ctrl+**↑** | To previous block |
| Ctrl+**↓** | To next block |
| Ctrl+Y | One page up |
| Ctrl+V | One page down |
| Alt+\\ | To top of buffer |
| Alt+/ | To end of buffer |
## Special movement
# enable mouse support
nano -m <file>
| Shortcut | Description |
|:--|:--|
| Alt+G | Go to specified line |
| Alt+\] | Go to complementary bracket |
| Alt+**↑** | Scroll viewport up |
| Alt+**↓** | Scroll viewport down |
| Alt+< | Switch to preceding buffer |
| Alt+> | Switch to succeeding buffer |
## Information
# keep the indentation of the previous line
nano -i <file>
| Shortcut | Description |
|:--|:--|
| Ctrl+C | Report cursor position |
| Alt+D | Report line/word/character count |
| Ctrl+G | Display help text |
## Various
# convert typed tabs to spaces, 4 wide
nano -ET4 <file>
| Shortcut | Description |
|:--|:--|
| Alt+A | Turn the mark on/off |
| Tab | Indent marked region |
| Shift+Tab | Unindent marked region |
| Alt+V | Enter next keystroke verbatim |
| Alt+N | Turn line numbers on/off |
| Alt+P | Turn visible whitespace on/off |
| Alt+X | Hide or unhide the help lines |
| Ctrl+L | Refresh the screen |
# do not wrap long lines
nano -w <file>
# keep a backup of the original as file~
nano -B <file>
# read-only view mode
nano -v <file>
# open at the position you left last time
nano -P <file>
```
## 💾 File Handling
| Shortcut | Action |
|----------|-----------------------------------|
| `Ctrl+S` | Save the current file |
| `Ctrl+O` | Write to a named file ("save as") |
| `Ctrl+R` | Insert another file into this one |
| `Ctrl+X` | Close the buffer and leave nano |
| `Alt+<` | Switch to the previous buffer |
| `Alt+>` | Switch to the next buffer |
## ✏️ Editing
| Shortcut | Action |
|---------------------|----------------------------------------------|
| `Ctrl+K` | Cut the current line into the cutbuffer |
| `Alt+6` | Copy the current line into the cutbuffer |
| `Ctrl+U` | Paste the cutbuffer |
| `Alt+T` | Cut from the cursor to the end of the buffer |
| `Alt+U` | Undo the last action |
| `Alt+E` | Redo the last undone action |
| `Alt+3` | Comment or uncomment the line or selection |
| `Ctrl+]` | Complete the current word |
| `Alt+A` | Set the mark — move to select |
| `Tab` / `Shift+Tab` | Indent / unindent the selection |
## 🔎 Search & Replace
| Shortcut | Action |
|----------|-----------------------------|
| `Ctrl+W` | Search forward ("where is") |
| `Ctrl+Q` | Search backward |
| `Alt+W` | Repeat the search forward |
| `Alt+Q` | Repeat the search backward |
| `Alt+R` | Search and replace |
> 💡 At the search prompt, `Alt+C` toggles case sensitivity, `Alt+R` switches to regular expressions and `Alt+B` limits the search to the selection.
## ⌫ Deletion
| Shortcut | Action |
|------------|---------------------------------------|
| `Ctrl+H` | Delete the character to the left |
| `Ctrl+D` | Delete the character under the cursor |
| `Alt+Bksp` | Delete the word to the left |
| `Ctrl+Del` | Delete the word to the right |
| `Alt+Del` | Delete the whole line |
## 🧭 Moving Around
| Shortcut | Action |
|---------------------|-------------------------------------------|
| `Ctrl+B` / `Ctrl+F` | One character left / right |
| `Ctrl+←` / `Ctrl+→` | One word left / right |
| `Ctrl+A` / `Ctrl+E` | Start / end of the line |
| `Ctrl+P` / `Ctrl+N` | One line up / down |
| `Ctrl+↑` / `Ctrl+↓` | Previous / next block |
| `Ctrl+Y` / `Ctrl+V` | One page up / down |
| `Alt+\` / `Alt+/` | Top / bottom of the buffer |
| `Alt+G` | Go to a line number |
| `Alt+]` | Jump to the matching bracket |
| `Alt+↑` / `Alt+↓` | Scroll the view without moving the cursor |
## 🔧 Operations
| Shortcut | Action |
|----------|----------------------------------------------|
| `Ctrl+T` | Run a command, or pipe the buffer through it |
| `Ctrl+J` | Justify the paragraph or selection |
| `Alt+J` | Justify the whole buffer |
| `Alt+B` | Run a syntax check (linter) |
| `Alt+F` | Run a formatter |
| `Alt+:` | Start or stop recording a macro |
| `Alt+;` | Replay the macro |
## ℹ️ Information & Display
| Shortcut | Action |
|----------|------------------------------------|
| `Ctrl+G` | Open the help text |
| `Ctrl+C` | Report the cursor position |
| `Alt+D` | Count lines, words and characters |
| `Alt+N` | Toggle line numbers |
| `Alt+P` | Toggle visible whitespace |
| `Alt+X` | Hide or show the shortcut bar |
| `Alt+V` | Insert the next keystroke verbatim |
| `Ctrl+L` | Redraw the screen |
## ⚙️ Configuration
Persistent settings live in `~/.nanorc` (system-wide: `/etc/nanorc`).
```bash
set linenumbers
set mouse
set tabsize 4
set tabstospaces
set autoindent
set constantshow
set softwrap
set positionlog
# syntax highlighting definitions shipped with nano
include /usr/share/nano/*.nanorc
```
## 📚 Resources
- [Official documentation](https://www.nano-editor.org/docs.php)
- [Manual page](https://man7.org/linux/man-pages/man1/nano.1.html)
- Built-in help: `Ctrl+G`
+377 -177
View File
@@ -1,218 +1,418 @@
**nginx** is a web server, reverse proxy and load balancer. Configuration is a tree of *contexts* — `main` → `events` / `http` → `server` → `location` — and a directive is only valid in the contexts that define it.
## 🛠 Service & CLI
```bash
##############################################################################
# NGINX
# DOCUMENTATION: https://nginx.org/en/docs/
##############################################################################
# check the configuration before you break production
sudo nginx -t
sudo nginx -t # Check syntax
sudo systemctl status nginx # nginx current status
sudo systemctl reload nginx # Reload nginx
sudo systemctl restart nginx # Restart nginx
sudo ln -s /etc/nginx/sites-available/example.com /etc/nginx/sites-enabled/ # Link website
sudo tail -f /var/log/nginx/access.log # Tail logs to inspect requests
# print the entire resolved configuration, includes and all
sudo nginx -T
# *****************************************************************************
# General Settings
# *****************************************************************************
# reload without dropping connections
sudo nginx -s reload
sudo systemctl reload nginx
# Ports
# start, stop, restart, status
sudo systemctl start nginx
sudo systemctl stop nginx
sudo systemctl restart nginx
sudo systemctl status nginx
server {
# Use HTTP protocol
listen 80;
# Use HTTPS protocol
listen 443 ssl;
# Listen on port 80 using IPv6
listen [::]:80;
# Listen on port 80 using **only** IPv6
listen [::]:80 ipv6only=on;
# version, and the modules it was built with
nginx -V
# enable a site (Debian/Ubuntu layout)
sudo ln -s /etc/nginx/sites-available/example.com /etc/nginx/sites-enabled/
# watch the requests come in
sudo tail -f /var/log/nginx/access.log
sudo tail -f /var/log/nginx/error.log
```
> 💡 `nginx -t` then `nginx -s reload` — never `restart` on a live server if you can avoid it. A reload keeps existing connections alive; a restart drops them.
## 📁 Where Things Live
| Path | What it holds |
|-------------------------------|------------------------------------------------------|
| `/etc/nginx/nginx.conf` | The main file — worker settings and the `http` block |
| `/etc/nginx/conf.d/*.conf` | Included by default on RHEL-style installs |
| `/etc/nginx/sites-available/` | Site configs on Debian/Ubuntu |
| `/etc/nginx/sites-enabled/` | Symlinks to the sites that are actually live |
| `/var/log/nginx/access.log` | Every request |
| `/var/log/nginx/error.log` | Errors, and anything the workers complain about |
| `/var/www/html` | The conventional document root |
## 🧱 Config Skeleton
```nginx
user www-data;
worker_processes auto;
events {
worker_connections 1024;
}
# Domain name (server_name)
http {
include /etc/nginx/mime.types;
default_type application/octet-stream;
sendfile on;
keepalive_timeout 65;
server {
listen 80;
server_name example.com;
location / {
root /var/www/html;
index index.html;
}
}
}
```
## 🎧 listen & server_name
```nginx
server {
# Listen to example.com
# plain HTTP
listen 80;
# HTTPS, with HTTP/2
listen 443 ssl;
http2 on;
# IPv6 as well, or IPv6 only
listen [::]:80;
listen [::]:80 ipv6only=on;
# the fallback server for requests that match no other server_name
listen 80 default_server;
# one name, several names, wildcards, or a regex
server_name example.com;
# Listen to multiple domains
server_name example.com www.example.com;
# Listen to all sub-domains
server_name *.example.com;
# Listen to all top-level domains
server_name example.*;
# Listen to unspecified hostnames (listens to IP address itself)
server_name ~^(?<sub>.+)\.example\.com$;
# requests that arrive with no Host header
server_name "";
}
```
# *****************************************************************************
# Serving Files
# *****************************************************************************
# Static assets (traditional web server)
server {
listen 80;
server_name example.com;
root /path/to/website;
# root /www/data/ for example
# If there's no 'root' inside, it will look for /www/data/index.html
location / {
}
# If there's no 'root' inside, it will look for /www/data/images/index.html
location /images/ {
}
# Since there's 'root' inside, it will look for /www/media/videos/index.html
location /videos/ {
root /www/media;
}
}
# *****************************************************************************
# Redirects
# *****************************************************************************
# 301 Permanent
server {
# Redirect www.example.com to example.com
listen 80;
server_name www.example.com;
return 301 http://example.com$request_uri;
}
server {
# Redirect http to https
listen 80;
server_name example.com;
return 301 https://example.com$request_uri;
}
# 302 Temporary
server {
listen 80;
server_name yourdomain.com;
return 302 http://otherdomain.com;
}
# *****************************************************************************
# Reverse proxy
# *****************************************************************************
# Useful for Node.js, Streamlit, Jupyter, etc
# Basic
## 📄 Serving Files
```nginx
server {
listen 80;
server_name example.com;
root /var/www/example.com;
index index.html index.htm;
# try the file, then the directory, then fall back — the SPA pattern
location / {
proxy_pass http://0.0.0.0:3000;
# where 0.0.0.0:3000 is your Node.js Server bound on 0.0.0.0 listing on port 3000
try_files $uri $uri/ /index.html;
}
# cache fingerprinted assets hard
location ~* \.(js|css|png|jpe?g|gif|svg|woff2?)$ {
expires 1y;
add_header Cache-Control "public, immutable";
access_log off;
}
# a directory listing, when you actually want one
location /downloads/ {
autoindex on;
}
# a single file
location = /robots.txt {
root /var/www/example.com;
log_not_found off;
}
}
```
# Basic + (upstream)
> ⚠️ **`root` vs `alias`**: `root` appends the whole URI to the path, `alias` replaces the matched prefix. With `location /static/ { root /var/www; }` a request for `/static/a.png` reads `/var/www/static/a.png`; with `alias /var/www/assets/;` it reads `/var/www/assets/a.png`. Always end an `alias` path with `/`.
upstream node_js {
server 0.0.0.0:3000;
# where 0.0.0.0:3000 is your Node.js Server bound on 0.0.0.0 listing on port 3000
## 📍 Location Matching
nginx does not pick locations top to bottom — it picks by modifier, in this order:
| Modifier | Example | Meaning | Priority |
|----------|--------------------------|-----------------------------------------------------|----------------------|
| `=` | `location = /health` | Exact match | 1 — wins immediately |
| `^~` | `location ^~ /static/` | Prefix match that stops regex matching | 2 |
| `~` | `location ~ \.php$` | Case-sensitive regex, first match in file order | 3 |
| `~*` | `location ~* \.(jpe?g)$` | Case-insensitive regex | 3 |
| *(none)* | `location /images/` | Prefix match — the longest one wins if no regex did | 4 |
```nginx
location = /health { return 200 "ok\n"; } # checked first, cheapest
location ^~ /assets/ { root /var/www; } # never falls through to the regex below
location ~* \.(png|jpg)$ { expires 30d; }
location / { try_files $uri $uri/ =404; } # the catch-all
```
## 🔀 Redirects & Rewrites
```nginx
# permanent redirect to the canonical host
server {
listen 80;
server_name www.example.com;
return 301 https://example.com$request_uri;
}
# force HTTPS
server {
listen 80;
server_name example.com;
location / {
proxy_pass http://node_js;
}
}
# Upgraded Connection (useful for applications with support for WebSockets)
upstream node_js {
server 0.0.0.0:3000;
}
server {
listen 80;
server_name example.com;
location / {
proxy_pass http://node_js;
proxy_redirect off;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_set_header Host $host;
}
}
# *****************************************************************************
# HTTPS Protocol
# *****************************************************************************
# The majority of SSL options depend on what your application does or needs
server {
listen 443 ssl http2;
server_name example.com;
ssl on;
ssl_certificate /path/to/cert.pem;
ssl_certificate_key /path/to/privkey.pem;
ssl_stapling on;
ssl_stapling_verify on;
ssl_trusted_certificate /path/to/fullchain.pem;
ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
ssl_session_timeout 1d;
ssl_session_cache shared:SSL:50m;
add_header Strict-Transport-Security max-age=15768000;
}
# Permanent redirect to HTTPS secured domain
server {
listen 80;
server_name yourdomain.com;
return 301 https://$host$request_uri;
}
# You can easily secure you website/app using Let's Encrypt.
# Go https://certbot.eff.org/lets-encrypt/ubuntuxenial-nginx.html for more information
# temporary redirect of a single path
location /old-page {
return 302 /new-page;
}
# *****************************************************************************
# Load Balancing
# *****************************************************************************
# rewrite with a captured segment
location /blog/ {
rewrite ^/blog/(.*)$ /articles/$1 permanent;
}
# Useful for large applications running in multiple instances. Below example is for reverse proxy
upstream node_js {
server 0.0.0.0:3000;
server 0.0.0.0:4000;
server 127.155.142.421;
# serve a maintenance page for everyone but your own IP
location / {
if ($remote_addr != 203.0.113.7) {
return 503;
}
}
```
> 💡 Prefer `return` over `rewrite` — it is faster and clearer. Reach for `rewrite` only when you need to transform the path.
## 🔁 Reverse Proxy
```nginx
server {
listen 80;
server_name app.example.com;
location / {
proxy_pass http://127.0.0.1:3000;
# pass the client's details through to the app
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
# timeouts: raise these for slow backends
proxy_connect_timeout 60s;
proxy_send_timeout 60s;
proxy_read_timeout 60s;
# buffering off for streaming responses (SSE, logs)
proxy_buffering on;
}
# WebSockets need the upgrade headers
location /ws/ {
proxy_pass http://127.0.0.1:3000;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_read_timeout 3600s;
}
}
```
> ⚠️ The trailing slash on `proxy_pass` changes everything. `proxy_pass http://backend;` forwards `/api/users` as `/api/users`; `proxy_pass http://backend/;` strips the matched location prefix first.
## ⚖️ Load Balancing
```nginx
upstream backend {
# least_conn; # send to the server with the fewest connections
# ip_hash; # sticky sessions by client IP
# hash $request_uri; # sticky by URI
server 10.0.0.1:3000 weight=3;
server 10.0.0.2:3000;
server 10.0.0.3:3000 max_fails=3 fail_timeout=30s;
server 10.0.0.4:3000 backup;
keepalive 32; # reuse upstream connections
}
server {
listen 80;
server_name example.com;
location / {
proxy_pass http://node_js;
proxy_pass http://backend;
proxy_http_version 1.1;
proxy_set_header Connection ""; # required for keepalive to work
}
}
```
```
| Method | Behaviour |
|-------------------------|-------------------------------------------------|
| *(default)* | Round robin, honouring `weight` |
| `least_conn` | Fewest active connections wins |
| `ip_hash` | Same client IP always reaches the same server |
| `hash <key>` | Distribute by any variable, e.g. `$request_uri` |
| `random two least_conn` | Pick two at random, then the less busy one |
## 🔐 HTTPS
```nginx
server {
listen 443 ssl;
http2 on;
server_name example.com;
ssl_certificate /etc/letsencrypt/live/example.com/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/example.com/privkey.pem;
ssl_protocols TLSv1.2 TLSv1.3;
ssl_prefer_server_ciphers off;
ssl_session_cache shared:SSL:10m;
ssl_session_timeout 1d;
ssl_stapling on;
ssl_stapling_verify on;
# tell browsers to stay on HTTPS
add_header Strict-Transport-Security "max-age=63072000" always;
}
```
```bash
# get and renew certificates automatically
sudo certbot --nginx -d example.com -d www.example.com
sudo certbot renew --dry-run
```
## 🚦 Limits & Access Control
```nginx
http {
# 10 MB of state ≈ 160k addresses; 10 requests/second per IP
limit_req_zone $binary_remote_addr zone=req_limit:10m rate=10r/s;
limit_conn_zone $binary_remote_addr zone=conn_limit:10m;
server {
# allow short bursts, queue the rest without delaying legitimate users
location /api/ {
limit_req zone=req_limit burst=20 nodelay;
limit_conn conn_limit 10;
}
# the size of the largest upload you accept
client_max_body_size 25m;
# HTTP basic auth (htpasswd -c /etc/nginx/.htpasswd user)
location /admin/ {
auth_basic "Restricted";
auth_basic_user_file /etc/nginx/.htpasswd;
}
# allow a subnet, deny the rest
location /internal/ {
allow 10.0.0.0/8;
deny all;
}
# do not serve dotfiles
location ~ /\.(?!well-known) {
deny all;
}
}
}
```
## 🗜 Performance
```nginx
http {
# compress text responses
gzip on;
gzip_vary on;
gzip_min_length 1024;
gzip_types text/plain text/css application/json application/javascript
text/xml application/xml image/svg+xml;
# kernel-level file sending
sendfile on;
tcp_nopush on;
tcp_nodelay on;
# keep connections around for reuse
keepalive_timeout 65;
keepalive_requests 1000;
# cache open file handles
open_file_cache max=10000 inactive=30s;
open_file_cache_valid 60s;
# a proxy cache for upstream responses
proxy_cache_path /var/cache/nginx keys_zone=app_cache:10m max_size=1g inactive=60m;
server {
location / {
proxy_cache app_cache;
proxy_cache_valid 200 10m;
add_header X-Cache-Status $upstream_cache_status;
}
}
}
```
## 🧪 Useful Variables
| Variable | Holds |
|------------------------------|--------------------------------------------------|
| `$host` | The Host header, or the server name that matched |
| `$uri` | The normalised path, without the query string |
| `$request_uri` | The original path **with** the query string |
| `$args` / `$arg_name` | The whole query string / one named parameter |
| `$scheme` | `http` or `https` |
| `$remote_addr` | The client's IP address |
| `$proxy_add_x_forwarded_for` | The existing chain plus `$remote_addr` |
| `$http_<name>` | Any request header, e.g. `$http_user_agent` |
| `$request_method` | `GET`, `POST`, … |
| `$status` | The response status |
| `$upstream_addr` | Which backend actually served the request |
| `$upstream_response_time` | How long the backend took |
| `$request_time` | How long nginx took, start to finish |
## 🩺 Troubleshooting
| Symptom | Usual cause |
|----------------------------------|----------------------------------------------------------------------------------|
| `502 Bad Gateway` | The backend is down, or refused the connection — check the app and `proxy_pass` |
| `504 Gateway Timeout` | The backend is too slow — raise `proxy_read_timeout`, then fix the app |
| `413 Request Entity Too Large` | Raise `client_max_body_size` |
| `403 Forbidden` on a static file | The worker user cannot read the path, or a directory lacks `x` |
| The wrong site is served | No `server_name` matched, so the `default_server` answered |
| Changes have no effect | The config was never reloaded, or the file is not symlinked into `sites-enabled` |
```bash
# see the config nginx is actually running, includes resolved
sudo nginx -T | less
# which worker user needs read access
ps -o user= -C nginx | sort -u
# turn up the error log while debugging
# error_log /var/log/nginx/error.log debug;
```
## 📚 Resources
- [Official documentation](https://nginx.org/en/docs/)
- [Directive index](https://nginx.org/en/docs/dirindex.html)
- [Variable index](https://nginx.org/en/docs/varindex.html)
- [Mozilla SSL configuration generator](https://ssl-config.mozilla.org/)
- [Certbot — Let's Encrypt](https://certbot.eff.org/)
@@ -1,220 +1,434 @@
## Variables
**PowerShell** is a shell and scripting language built on .NET. Unlike POSIX shells it passes **objects** down the pipeline rather than text, so `Get-Process | Sort-Object CPU` sorts real numbers instead of parsing columns.
| Syntax | Description |
|-------|-------------|
| `$var = "string"` | Assign variable |
| `[Type]$var="typedVar"` | Assign strong typed variable |
| `[ValidateRange(1,9)][int]$x=1` | Strong typed attribute‑controlled variable |
| `$a,$b,$c = 0` or `$a,$b = 'a','b'` | Assign multiple variables |
| `$a,$b = $b,$a` | Flip variables |
| *Scopes* | global, local, private, script |
| `$global:var = "var"` | Assign global scoped variable |
> ℹ️ **PowerShell 7+** (`pwsh`) is cross-platform and current; **Windows PowerShell 5.1** (`powershell.exe`) ships with Windows and is in maintenance. Check with `$PSVersionTable`.
## Arrays
## 🔎 Finding Your Way
| Syntax | Description |
|--------|-------------|
| `"a","b","c"` | Array of strings |
| `@()` | Empty array |
| `1,(2,3),4` | Array within array |
| `,"hi"` | Array of one element |
| `$arr[5]` | Sixth element |
| `$arr[2..20]` | Elements 3–21 |
| `$arr[-1]` | Last element |
| `$arr[-3..-1]` | Last three elements |
| `$arr[1,4+6..9]` | Elements at positions 1,4,6–9 |
| `@(Get-Process)` | Force result to array |
| `$arr[($arr.length-1)..0]` | Reverse array |
| `$arr[1] += 200` | Add to existing item |
| `$b = $arr[0,1 + 3..6]` | New array from elements |
| `$z = $arrA + $arrB` | Combine arrays |
```powershell
# what commands exist for a noun?
Get-Command *service*
Get-Command -Module Microsoft.PowerShell.Management
## Hash Tables (Associative Arrays)
# help, with the part everyone actually wants
Get-Help Get-ChildItem -Examples
Get-Help Get-ChildItem -Full
Update-Help
| Syntax | Description |
|--------|-------------|
| `$hash = @{}` | Empty hash table |
| `@{foo=1; bar='value2'}` | Initialize hash |
| `[ordered]@{a=1; b=2; c=3}` | Ordered dictionary |
| `$hash.key1 = 1` | Assign value |
| `$hash.key1` | Get value |
| `$hash["key1"]` | Get value |
| `$hash.GetEnumerator \| sort Key` | Sort by key |
| `[pscustomobject]@{x=1;z="z"}` | Custom object |
# what properties and methods does this object have?
Get-Process | Get-Member
## Strings
# what did that command actually return?
Get-Process | Select-Object -First 1 | Format-List *
```
| Syntax | Description |
|--------|-------------|
| `"$var expand"` | Expand variables |
| `'$var no expand'` | No expansion |
| `@" ... "@` | Here‑string (double quotes allow expressions) |
## 📦 Variables
## Comments & Escape Characters
| Syntax | Description |
|---------------------------------|-----------------------------------------------|
| `$var = "string"` | Assign a variable |
| `[int]$var = 5` | Strongly typed variable |
| `[ValidateRange(1,9)][int]$x=1` | Typed and constrained |
| `$a, $b = 'a', 'b'` | Assign several at once |
| `$a, $b = $b, $a` | Swap two values |
| `$global:var = "v"` | Scope: `global`, `script`, `local`, `private` |
| `${my var}` | A name containing spaces or punctuation |
| `$env:PATH` | An environment variable |
| `Remove-Variable x` | Delete a variable |
| Syntax | Description |
|--------|-------------|
| `# Comment` | Comment |
| `<# comment #>` | Multiline comment |
| `"A '\"test`\""` | Escape `'` |
| `` `t `` | Tab |
| `` `n `` | New line |
| `` ` `` | Line continuation |
## 📚 Arrays
## Basics of Text & Files
| Syntax | Description |
|---------------------------------------------------------|---------------------------------------------------|
| `"a","b","c"` | Array of strings |
| `@()` | Empty array |
| `,"hi"` | Array of exactly one element |
| `1,(2,3),4` | Nested array |
| `$arr[5]` | Sixth element (0-based) |
| `$arr[2..20]` | A range of elements |
| `$arr[-1]` | The last element |
| `$arr[-3..-1]` | The last three |
| `$arr[1,4+6..9]` | Positions 1, 4 and 6–9 |
| `$arr[($arr.length-1)..0]` | Reversed |
| `@(Get-Process)` | Force a single result into an array |
| `$z = $arrA + $arrB` | Concatenate |
| `$list = [System.Collections.Generic.List[int]]::new()` | A growable list — much faster than `+=` in a loop |
| Command | Description |
|---------|-------------|
| `Get-Location` | Current directory |
| `Set-Location` | Change directory |
| `Get-Content` | Read file |
| `Add-Content` | Append |
| `Set-Content` | Overwrite |
| `Out-File` | Formatted output |
| `Out-Null` | Discard output |
| `Out-String` | Convert to string |
| `Copy-Item` | Copy |
| `Remove-Item` | Delete |
| `Move-Item` | Move |
| `Rename-Item` | Rename |
| `Set-Item` | Set file contents |
| `Clear-Item` | Clear file |
| `New-Item` | New file/dir |
## 🗂 Hash Tables
## Flow Control
| Syntax | Description |
|--------------------------------------------|----------------------------|
| `$hash = @{}` | Empty hash table |
| `@{foo=1; bar='two'}` | Initialise with values |
| `[ordered]@{a=1; b=2}` | Keeps insertion order |
| `$hash.key1` / `$hash["key1"]` | Read a value |
| `$hash.key1 = 1` | Write a value |
| `$hash.Remove("key1")` | Delete a key |
| `$hash.ContainsKey("key1")` | Test for a key |
| `$hash.GetEnumerator() \| Sort-Object Key` | Iterate in key order |
| `[pscustomobject]@{x=1; z="z"}` | Turn a hash into an object |
| Syntax | Description |
|--------|-------------|
| `If($x -eq 5){ } elseif($x -gt 5){ } else{ }` | If |
| `$x=1; while($x -lt 10){$x;$x++}` | While |
| `for($i=0; $i -lt 10; $i++){ $i }` | For |
| `foreach($file in dir C:\){$file.Name}` | Foreach |
| `1..10 \| foreach{$_}` | Foreach pipeline |
| `Switch -options (...) { Pattern { } Default { } }` | Switch |
## 🔤 Strings
## Operators
| Syntax | Description |
|----------------------------------|-------------------------------------------------------|
| `"$var expands"` | Double quotes interpolate |
| `'$var does not'` | Single quotes are literal |
| `"Total: $($items.Count)"` | `$( )` evaluates an expression inside a string |
| `"{0} of {1}" -f $a, $b` | The format operator |
| `` "col1`tcol2`n" `` | `` `t `` tab, `` `n `` newline, `` `` ` `` `` escapes |
| `@"…"@` / `@'…'@` | Here-string, expanding / literal |
| `"abc".ToUpper()` | .NET methods work on strings |
| `$s -split ','` / `$a -join ','` | Split and join |
| `$s.Trim()` / `.PadLeft(5)` | Trim and pad |
| `$s -replace 'a','b'` | Regex replace |
### Assignment, Logical, Comparison
## 💬 Comments & Escaping
| Syntax | Description |
|--------|-------------|
| `= += == *= /= %= ++ --` | Assignment |
| `-and -or -xor -not !` | Logical |
| `-eq -ne` | Equal / not equal |
| `-gt -ge` | Greater than / ≥ |
| `-lt -le` | Less than / ≤ |
| `-replace` | Replace text |
| `-match -notmatch` | Regex |
| `-like -notlike` | Wildcards |
| `-contains -notcontains` | Array contains |
| `-in -notin` | Reverse contains |
```powershell
# a single-line comment
## Other Operators
<#
a block comment, also used for help text
#>
| Syntax | Description |
|--------|-------------|
| `-split` | Split string |
| `-join` | Join strings |
| `..` | Range |
| `-is -isnot` | Type check |
| `-as` | Cast |
| `-f` | Format strings |
| `[type]` | Cast operator |
| `$( )` | Subexpression |
| `@()` | Array subexpression |
| `&` | Invocation |
# the backtick is the escape character
Write-Output "He said `"hi`""
Write-Output "line one`nline two"
## Objects
# and the line-continuation character
Get-ChildItem -Path C:\ `
-Recurse `
-Filter *.log
```
| Syntax | Description |
|--------|-------------|
| `(Get-Date).Date` | Date property |
| `Get-Date \| Get-Member` | List members |
| `[DateTime]::Now` | Static property |
| `"string".ToUpper()` | Method call |
| `[system.Net.Dns]::GetHostByAddress("127.0.0.1")` | Static method |
| `new-object -com excel.application` | Excel COM object |
## 📁 Files & Paths
## Filter, Sort, Group, Format
| Command | Does |
|-----------------------------------------------------------|---------------------------------------|
| `Get-Location` (`pwd`) | Current directory |
| `Set-Location` (`cd`) | Change directory |
| `Get-ChildItem` (`ls`, `dir`) | List a directory |
| `Get-Content` (`cat`) | Read a file |
| `Set-Content` / `Add-Content` | Overwrite / append |
| `Out-File` / `Out-Null` / `Out-String` | Write to a file / discard / stringify |
| `New-Item -ItemType Directory` | Create a file or folder |
| `Copy-Item` / `Move-Item` / `Rename-Item` / `Remove-Item` | Copy, move, rename, delete |
| `Test-Path` | Does it exist? |
| `Split-Path -Parent` | The directory part of a path |
| `Join-Path a b` | Build a path portably |
| `Resolve-Path` | Expand to a full path |
| `Get-FileHash file.zip` | Checksum a file |
| Command | Description |
|---------|-------------|
| `dir C:\pub \| where LastWriteTime -gt (Get-Date).AddDays(-1)` | Files modified yesterday |
| `ps \| where { $_.path -like "C:\windows\system32*" -and $_.company -notlike "Microsoft*" }` | Filter processes |
| `ps Explorer \| select ProcessName -ExpandProperty Modules \| format-list` | Expand modules |
| `ps \| sort WorkingSet \| select -Last 5` | Top 5 memory |
| `"a","b","a" \| select -Unique` | Unique values |
| `Get-Service \| Group-Object Status` | Group by status |
| `dir \| Group-Object { $_.Length -gt 100KB }` | Group by size |
| `Get-Content pcs.txt \| Select-String "q-" \| sls "win7"` | Search text |
| `ps \| Format-Table Name,StartTime -AutoSize` | Format table |
| `ps \| Format-Table ProcessName, @{Label="Total Run Time"; Expression={(Get-Date)-$_.StartTime}}` | Custom column |
| `Get-EventLog -Log System \| select -First 5 \| ft -wrap` | First 5 events |
| `gi C:\Users \| fl *` | All properties |
| `"{0}``t{1}``n" -f $a,5` | Format operator |
```powershell
# read a large file line by line without loading it all
Get-Content big.log -ReadCount 1000 | ForEach-Object { $_ }
## Common Commands
# tail -f
Get-Content app.log -Wait -Tail 20
- `Get-EventLog`
- `Get-WinEvent`
- `Get-CimInstance`
- `Get-Date`
- `Start-Sleep`
- `Compare-Object`
- `Start-Job`
- `Get-Credential`
- `Test-Connection`
- `New-PSSession`
- `Test-Path`
- `Split-Path`
# every .log modified in the last day
Get-ChildItem C:\logs -Recurse -Filter *.log |
Where-Object LastWriteTime -gt (Get-Date).AddDays(-1)
```
## Importing, Exporting, Converting
## 🔀 Flow Control
- `Export-Clixml`
- `Import-Clixml`
- `ConvertTo-Xml`
- `ConvertTo-Html`
- `Export-Csv`
- `Import-Csv`
- `ConvertTo-Csv`
- `ConvertFrom-Csv`
```powershell
if ($x -eq 5) { "five" } elseif ($x -gt 5) { "more" } else { "less" }
## Automatic Variables
while ($x -lt 10) { $x; $x++ }
| Variable | Description |
|----------|-------------|
| `$_` / `$PSItem` | Pipeline object |
| `$Args` | Script args |
| `$Error` | Errors |
| `$True` / `$False` | Boolean |
| `$null` | Null |
| `$profile` | Profile paths |
do { $x++ } while ($x -lt 10)
## PSDrives
for ($i = 0; $i -lt 10; $i++) { $i }
| Drive | Description |
|-------|-------------|
| `Alias:` | Aliases |
| `Cert:` | Certificates |
| `Env:` | Environment variables |
| `Function:` | Functions |
| `HKLM:` | Registry (Local Machine) |
| `HKCU:` | Registry (Current User) |
| `Variable:` | Variables |
| `WSMan:` | WinRM |
| `AD:` | Active Directory |
| `Set-Location HKLM:` | Navigate registry |
| `gci variable:` | List variables |
foreach ($file in Get-ChildItem C:\) { $file.Name }
## Regular Expressions
1..10 | ForEach-Object { $_ * 2 }
| Pattern | Meaning |
|---------|---------|
| `\w` | Word char `[a-zA-Z0-9]` |
| `\W` | Non‑word char |
| `\s` | Whitespace |
| `\S` | Non‑whitespace |
| `\d` / `\D` | Digit / non‑digit |
| `{n} {n,} {n,m}` | Quantifiers |
| More | Google .NET Regex |
# parallel, PowerShell 7+
1..10 | ForEach-Object -Parallel { Start-Sleep 1; $_ } -ThrottleLimit 5
switch ($value) {
'a' { 'letter a'; break }
{ $_ -gt 10 } { 'big' }
default { 'something else' }
}
```
## ⚖️ Operators
| Operator | Meaning |
|--------------------------------|-----------------------------------------|
| `= += -= *= /= %= ++ --` | Assignment |
| `-eq` / `-ne` | Equal / not equal |
| `-gt` `-ge` `-lt` `-le` | Numeric comparison |
| `-and` `-or` `-xor` `-not` `!` | Logical |
| `-like` / `-notlike` | Wildcard match (`*`, `?`) |
| `-match` / `-notmatch` | Regex match — fills `$Matches` |
| `-replace 'a','b'` | Regex replace |
| `-contains` / `-in` | Array membership, either way round |
| `-split` / `-join` | Split a string / join an array |
| `-is` / `-isnot` / `-as` | Type test and conversion |
| `-f` | Format a string |
| `..` | Range |
| `$( )` / `@( )` | Sub-expression / array sub-expression |
| `&` / `.` | Invoke a command / dot-source a script |
| `??` / `??=` | Null-coalescing (PowerShell 7+) |
| `?.` / `?[ ]` | Null-conditional access (PowerShell 7+) |
> ⚠️ Comparisons are **case-insensitive** by default. Use `-ceq`, `-clike`, `-cmatch` when case matters.
## 🧱 Objects & the Pipeline
```powershell
# properties and methods
(Get-Date).Date
(Get-Date).AddDays(-7)
"string".ToUpper()
# static members
[DateTime]::Now
[Math]::Round(3.14159, 2)
[System.Net.Dns]::GetHostByAddress("127.0.0.1")
# build your own objects
[pscustomobject]@{ Name = 'web01'; Status = 'up' }
# add a calculated property
Get-Process | Select-Object Name, @{ Name = 'MB'; Expression = { [math]::Round($_.WorkingSet / 1MB, 1) } }
```
## 🔍 Filter, Sort, Group, Format
| Example | Does |
|-----------------------------------------------------------|---------------------------|
| `Get-Process \| Where-Object CPU -gt 100` | Filter (simple syntax) |
| `Get-Process \| Where-Object { $_.Name -like "chrome*" }` | Filter (script block) |
| `Get-Process \| Sort-Object WorkingSet -Descending` | Sort |
| `Get-Process \| Select-Object -First 5` | Take the first few |
| `"a","b","a" \| Select-Object -Unique` | Deduplicate |
| `Get-Service \| Group-Object Status` | Group |
| `Get-Process \| Measure-Object WorkingSet -Sum -Average` | Aggregate |
| `Get-Process \| Select-Object -ExpandProperty Modules` | Flatten a nested property |
| `Get-Process \| Format-Table Name, Id -AutoSize` | Table output |
| `Get-Item C:\ \| Format-List *` | Every property |
| `Get-Content log.txt \| Select-String "error"` | grep |
| `Compare-Object $a $b` | Diff two collections |
> 💡 `Format-*` is always the **last** step — its output is display text, not objects, so nothing downstream can filter it.
## 🧰 Functions & Scripts
```powershell
function Get-Square {
param(
[Parameter(Mandatory)][int]$Number,
[switch]$Verbose
)
$Number * $Number
}
Get-Square -Number 7
# an advanced function: pipeline input, -WhatIf, -Verbose for free
function Remove-OldLog {
[CmdletBinding(SupportsShouldProcess)]
param(
[Parameter(ValueFromPipeline)][string]$Path,
[int]$Days = 30
)
process {
if ($PSCmdlet.ShouldProcess($Path, "delete")) {
Remove-Item $Path
}
}
}
```
## 🚨 Error Handling
```powershell
try {
Get-Content missing.txt -ErrorAction Stop
}
catch [System.IO.FileNotFoundException] {
Write-Warning "no such file"
}
catch {
Write-Error "unexpected: $($_.Exception.Message)"
}
finally {
"always runs"
}
# make non-terminating errors terminate, so catch can see them
Get-Item missing -ErrorAction Stop
# per-session default
$ErrorActionPreference = 'Stop'
# the most recent error, and the exit code of the last native command
$Error[0]
$LASTEXITCODE
$?
```
## 🌐 Web & Data
```powershell
# REST calls, parsed into objects
$data = Invoke-RestMethod https://api.github.com/repos/vuejs/core
$data.stargazers_count
# with headers and a JSON body
Invoke-RestMethod -Uri $url -Method Post `
-Headers @{ Authorization = "Bearer $token" } `
-ContentType 'application/json' `
-Body (@{ name = 'demo' } | ConvertTo-Json)
# the raw response, for scraping or downloads
Invoke-WebRequest $url -OutFile page.html
# convert between formats
Get-Process | Select-Object Name, Id | ConvertTo-Json
Get-Content data.json | ConvertFrom-Json
Import-Csv users.csv | Where-Object Dept -eq 'IT' | Export-Csv it.csv -NoTypeInformation
Export-Clixml / Import-Clixml # round-trips real objects
```
## 🖥 System Administration
```powershell
# processes
Get-Process | Sort-Object CPU -Descending | Select-Object -First 10
Stop-Process -Name notepad -Force
# services
Get-Service | Where-Object Status -eq 'Running'
Restart-Service -Name Spooler
Set-Service -Name Spooler -StartupType Manual
# background jobs
$job = Start-Job { Start-Sleep 10; "done" }
Receive-Job $job -Wait
# scheduled tasks, event logs, system info
Get-ScheduledTask
Get-WinEvent -LogName System -MaxEvents 20
Get-CimInstance Win32_LogicalDisk | Select-Object DeviceID, FreeSpace
Test-Connection example.com -Count 2
```
## 🔗 Remoting
```powershell
# one command on many machines
Invoke-Command -ComputerName web01, web02 -ScriptBlock { Get-Service Spooler }
# an interactive session
Enter-PSSession -ComputerName web01
Exit-PSSession
# a reusable session
$s = New-PSSession -ComputerName web01 -Credential (Get-Credential)
Invoke-Command -Session $s { hostname }
Remove-PSSession $s
```
## 📦 Modules & Execution Policy
```powershell
# find, install, load
Find-Module Pester
Install-Module Pester -Scope CurrentUser
Import-Module Pester
Get-Module -ListAvailable
# where does PowerShell look for modules?
$env:PSModulePath -split [IO.Path]::PathSeparator
# scripts refuse to run? check and relax the policy
Get-ExecutionPolicy -List
Set-ExecutionPolicy RemoteSigned -Scope CurrentUser
# your profile script
notepad $PROFILE
```
## 🧾 Automatic Variables
| Variable | Holds |
|--------------------------|-------------------------------------------|
| `$_` / `$PSItem` | The current pipeline object |
| `$Args` | Unbound arguments to a script or function |
| `$Error` | The error history, newest first |
| `$Matches` | Capture groups from the last `-match` |
| `$PSVersionTable` | Version and edition information |
| `$PWD` / `$HOME` | Current directory / home directory |
| `$PSScriptRoot` | The folder the running script lives in |
| `$LASTEXITCODE` | Exit code of the last native command |
| `$?` | Did the last command succeed? |
| `$true` `$false` `$null` | Constants |
| `$PROFILE` | Path to your profile script |
## 💾 PSDrives
| Drive | Contents |
|-------------------|-----------------------|
| `Env:` | Environment variables |
| `Alias:` | Command aliases |
| `Function:` | Defined functions |
| `Variable:` | Variables |
| `Cert:` | Certificate stores |
| `HKLM:` / `HKCU:` | The registry |
| `WSMan:` | WinRM configuration |
```powershell
Set-Location HKLM:\SOFTWARE
Get-ChildItem Env: | Sort-Object Name
Get-ChildItem Variable:
```
## ⌨️ Aliases Worth Knowing
| Alias | Real command | Alias | Real command |
|--------------------|-----------------|-------------|--------------------------------|
| `ls`, `dir`, `gci` | `Get-ChildItem` | `%` | `ForEach-Object` |
| `cat`, `gc` | `Get-Content` | `?` | `Where-Object` |
| `cd`, `sl` | `Set-Location` | `select` | `Select-Object` |
| `cp`, `copy` | `Copy-Item` | `sort` | `Sort-Object` |
| `rm`, `del` | `Remove-Item` | `ft` / `fl` | `Format-Table` / `Format-List` |
| `ps`, `gps` | `Get-Process` | `sls` | `Select-String` |
| `kill` | `Stop-Process` | `gm` | `Get-Member` |
> ⚠️ Aliases are for the console, not for scripts — write the full cmdlet name in anything you commit.
## 🔣 Regular Expressions
| Pattern | Matches |
|----------------------|-----------------------------|
| `\w` / `\W` | Word character / non-word |
| `\s` / `\S` | Whitespace / non-whitespace |
| `\d` / `\D` | Digit / non-digit |
| `{n}` `{n,}` `{n,m}` | Quantifiers |
| `^` / `$` | Start / end of the string |
| `(?<name>…)` | Named capture group |
```powershell
# -match fills $Matches
if ("build-1234" -match '(?<name>\w+)-(?<id>\d+)') { $Matches.id }
# extract every match from a file
Select-String -Path app.log -Pattern '\b\d{1,3}(\.\d{1,3}){3}\b' -AllMatches |
ForEach-Object { $_.Matches.Value } | Sort-Object -Unique
```
PowerShell uses the .NET regex engine — see [.NET regular expressions](https://learn.microsoft.com/dotnet/standard/base-types/regular-expressions).
## 📚 Resources
- [PowerShell documentation](https://learn.microsoft.com/powershell/)
- [Cmdlet reference](https://learn.microsoft.com/powershell/module/)
- [PowerShell Gallery](https://www.powershellgallery.com/)
- [Style guide (community)](https://poshcode.gitbook.io/powershell-practice-and-style/)
+148 -95
View File
@@ -1,121 +1,174 @@
### Normal characters
**Regular expressions** describe a pattern that text either matches or does not. The syntax below is the JavaScript flavour; PCRE, Python, Go and Java agree on almost all of it — the differences are called out where they bite.
Expression | Description
:--|:--
`.` or `[^\n\r]` | any character *excluding* a newline or carriage return
`[A-Za-z]` | alphabet
`[a-z]` | lowercase alphabet
`[A-Z]` | uppercase alphabet
`\d` or `[0-9]` | digit
`\D` or `[^0-9]` | non-digit
`_` | underscore
`\w` or `[A-Za-z0-9_]` | alphabet, digit or underscore
`\W` or `[^A-Za-z0-9_]` | inverse of `\w`
`\S` | inverse of `\s`
```js
/^\d{4}-\d{2}-\d{2}$/.test('2026-08-22') // true
'a1b2'.replace(/\d/g, '#') // 'a#b#'
```
### Whitespace characters
## 🔡 Character Classes
Expression | Description
:--|:--
` ` | space
`\t` | tab
`\n` | newline
`\r` | carriage return
`\s` | space, tab, newline or carriage return
| Expression | Matches |
|------------------------|---------------------------------------------------------------|
| `.` | Any character except a newline (any at all with the `s` flag) |
| `[A-Za-z]` | A letter |
| `[a-z]` / `[A-Z]` | A lower-case / upper-case letter |
| `\d` or `[0-9]` | A digit |
| `\D` or `[^0-9]` | Anything but a digit |
| `\w` or `[A-Za-z0-9_]` | A letter, digit or underscore |
| `\W` | The inverse of `\w` |
| `\s` | Whitespace: space, tab, newline, carriage return |
| `\S` | The inverse of `\s` |
| `\p{L}` / `\p{Nd}` | Any Unicode letter / decimal digit (needs the `u` flag) |
| `\x41` / `\u00E9` | A character by hex code, or by Unicode code point |
### Character set
## ␣ Whitespace
Expression | Description
:--|:--
`[xyz]` | either `x`, `y` or `z`
`[^xyz]` | neither `x`, `y` nor `z`
`[1-3]` | either `1`, `2` or `3`
`[^1-3]` | neither `1`, `2` nor `3`
| Expression | Matches |
|------------|-------------------------|
| ` ` | A literal space |
| `\t` | Tab |
| `\n` | Newline |
| `\r` | Carriage return |
| `\f` `\v` | Form feed, vertical tab |
| `\s` | Any of the above |
- Think of a character set as an `OR` operation on the single characters that are enclosed between the square brackets.
- Use `^` after the opening `[` to “negate” the character set.
- Within a character set, `.` means a literal period.
## 🎯 Character Sets
### Characters that require escaping
| Expression | Matches |
|-------------|-------------------------------------------|
| `[xyz]` | `x`, `y` or `z` |
| `[^xyz]` | Anything except `x`, `y`, `z` |
| `[1-3]` | `1`, `2` or `3` |
| `[^1-3]` | Anything except `1`, `2`, `3` |
| `[a-z0-9_]` | Ranges and single characters can be mixed |
#### Outside a character set
- A set is an **or** over single characters — `[abc]` matches one character, not the string `abc`.
- `^` right after the opening `[` negates the set.
- Inside a set, most metacharacters lose their meaning: `.` is a literal period, `+` a literal plus.
Expression | Description
:--|:--
`\.` | period
`\^` | caret
`\$` | dollar sign
`\|` | pipe
`\\` | back slash
`\/` | forward slash
`\(` | opening bracket
`\)` | closing bracket
`\[` | opening square bracket
`\]` | closing square bracket
`\{` | opening curly bracket
`\}` | closing curly bracket
## 🔒 Escaping
#### Inside a character set
Outside a character set, these need a backslash to be taken literally:
Expression | Description
:--|:--
`\\` | back slash
`\]` | closing square bracket
| Escape | Character | Escape | Character |
|--------|----------------------------------------|--------|---------------------|
| `\.` | Period | `\(` | Opening parenthesis |
| `\^` | Caret | `\)` | Closing parenthesis |
| `\$` | Dollar sign | `\[` | Opening bracket |
| `\\|` | Pipe | `\]` | Closing bracket |
| `\\` | Backslash | `\{` | Opening brace |
| `\/` | Forward slash (only in `/…/` literals) | `\}` | Closing brace |
- A `^` must be escaped only if it occurs immediately after the opening `[` of the character set.
- A `-` must be escaped only if it occurs between two alphabets or two digits.
Inside a character set only `\\` and `\]` always need escaping. A `^` needs it only immediately after `[`, and a `-` only between two other characters.
### Quantifiers
## 🔢 Quantifiers
Expression | Description
:--|:--
`{2}` | exactly 2
`{2,}` | at least 2
`{2,7}` | at least 2 but no more than 7
`*` | 0 or more
`+` | 1 or more
`?` | exactly 0 or 1
| Expression | Repeats the preceding item |
|-------------------------|-------------------------------------------------------|
| `{2}` | Exactly twice |
| `{2,}` | Two or more times |
| `{2,7}` | Between two and seven times |
| `*` | Zero or more |
| `+` | One or more |
| `?` | Zero or once |
| `*?` `+?` `??` `{2,7}?` | The lazy form — match as *few* characters as possible |
- The quantifier goes *after* the expression to be quantified.
```js
'<b>bold</b>'.match(/<.+>/)[0] // '<b>bold</b>' — greedy
'<b>bold</b>'.match(/<.+?>/)[0] // '<b>' — lazy
```
### Boundaries
## 📍 Anchors & Boundaries
Expression | Description
:--|:--
`^` | start of string
`$` | end of string
`\b` | word boundary
| Expression | Matches at |
|------------|-------------------------------------------------------------------------|
| `^` | Start of the string (start of a line with the `m` flag) |
| `$` | End of the string (end of a line with the `m` flag) |
| `\b` | A word boundary |
| `\B` | Anywhere that is *not* a word boundary |
| `\A` `\z` | Start / end of the string in PCRE, Python, Java — **not** in JavaScript |
- How word boundary matching works:
- At the beginning of the string if the first character is `\w`.
- Between two adjacent characters within the string, if the first character is `\w` and the second character is `\W`.
- At the end of the string if the last character is `\w`.
A word boundary sits between a `\w` and a `\W` (or the edge of the string), so `\bcat\b` matches `cat` in `the cat sat` but not in `concatenate`.
### Matching
## 🔀 Alternation & Lookaround
Expression | Description
:--|:--
`foo\|bar` | match either `foo` or `bar`
`foo(?=bar)` | match `foo` if it’s before `bar`
`foo(?!bar)` | match `foo` if it’s *not* before `bar`
`(?<=bar)foo` | match `foo` if it’s after `bar`
`(?<!bar)foo` | match `foo` if it’s *not* after `bar`
| Expression | Matches |
|---------------|-------------------------------------------|
| `foo\|bar` | Either `foo` or `bar` |
| `foo(?=bar)` | `foo` only when followed by `bar` |
| `foo(?!bar)` | `foo` only when **not** followed by `bar` |
| `(?<=bar)foo` | `foo` only when preceded by `bar` |
| `(?<!bar)foo` | `foo` only when **not** preceded by `bar` |
### Grouping and capturing
Lookaround is *zero-width*: it tests the surroundings without consuming them, so nothing it matches ends up in the result.
Expression | Description
:--|:--
`(foo)` | capturing group; match and capture `foo`
`(?:foo)` | non-capturing group; match `foo` but *without* capturing `foo`
`(foo)bar\1` | `\1` is a backreference to the 1st capturing group; match `foobarfoo`
## 🎁 Groups & Backreferences
- Capturing groups are only relevant in the following methods:
- `string.match(regexp)`
- `string.matchAll(regexp)`
- `string.replace(regexp, callback)`
- `\N` is a backreference to the `Nth` capturing group. Capturing groups are numbered starting from 1.
| Expression | Meaning |
|------------------|--------------------------------------------------|
| `(foo)` | Capturing group — matches `foo` and remembers it |
| `(?:foo)` | Non-capturing group — groups without capturing |
| `(?<year>\d{4})` | Named capturing group |
| `\1` | Backreference to the first group |
| `\k<year>` | Backreference to a named group |
## References and tools
```js
// named groups make replacements readable
'2026-08-22'.replace(/(?<y>\d{4})-(?<m>\d{2})-(?<d>\d{2})/, '$<d>/$<m>/$<y>')
// '22/08/2026'
- [MDN](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Guide/Regular_Expressions)
const { groups } = /(?<user>\w+)@(?<host>[\w.]+)/.exec('jane@example.com');
// groups.user === 'jane', groups.host === 'example.com'
```
## 🚩 Flags
| Flag | Name | Effect |
|------|-------------|-----------------------------------------------------------|
| `g` | global | Find every match, not just the first |
| `i` | ignore case | Case-insensitive matching |
| `m` | multiline | `^` and `$` match at line breaks |
| `s` | dotAll | `.` also matches newlines |
| `u` | unicode | Treat the pattern as Unicode code points; enables `\p{…}` |
| `y` | sticky | Match only from `lastIndex`, no scanning ahead |
| `d` | indices | Report the start/end index of every group |
## 🔁 Replacement Patterns
| Token | Inserts |
|------------|-----------------------------|
| `$&` | The whole match |
| `` $` `` | Everything before the match |
| `$'` | Everything after the match |
| `$1`, `$2` | Capture group 1, 2, … |
| `$<name>` | A named capture group |
| `$$` | A literal dollar sign |
> ℹ️ Other tools spell these differently: `\1` in `sed`, `\g<1>` in Python, `${1}` in Go.
## 🧪 Where the Groups Show Up (JavaScript)
```js
'2026-08-22'.match(/\d+/g) // ['2026', '08', '22'] — no groups with /g
[...'a1b2'.matchAll(/([a-z])(\d)/g)] // every match, with its groups
/(\d+)/.exec('abc 42') // ['42', '42', index: 4, ...]
/\d/.test('abc') // false
'a1'.replace(/(\w)(\d)/, (m, a, b) => b + a) // '1a'
'a, b;c'.split(/[,;]\s*/) // ['a', 'b', 'c']
```
## ⚠️ Gotchas
- **`.` does not match newlines** unless you set the `s` flag — a common surprise on multi-line input.
- **A `/g` regex is stateful.** Reusing the same object across `test()` calls advances `lastIndex` and skips matches; build a fresh one, or reset `lastIndex = 0`.
- **Nested quantifiers backtrack catastrophically.** `(a+)+$` against a long non-matching string can hang the process — avoid overlapping repetition, prefer possessive/atomic groups where the flavour has them.
- **Escape user input** before putting it in a pattern; in JS, `RegExp.escape()` where available, otherwise `s.replace(/[.*+?^${}()|[\]\\]/g, '\\$&')`.
- **Don't parse HTML with regex.** Use a parser; regex cannot handle nesting.
- **Backslashes double up in strings**: `new RegExp('\\d+')` is the same pattern as `/\d+/`.
## 📚 Resources
- [MDN — regular expressions](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Guide/Regular_expressions)
- [regex101 — build and explain patterns](https://regex101.com)
- [RegExplained](https://leaverou.github.io/regexplained/)
- [Regular-Expressions.info](https://www.regular-expressions.info/)
+80 -59
View File
@@ -1,15 +1,13 @@
This sheet gives you a complete, visual, color‑accurate reference for Cat6 wiring.
A complete, colour-accurate reference for terminating **RJ45 / Cat6** Ethernet cables: the two pinout standards, what each pair does, PoE, and the rules that decide whether the run passes a certifier.
## 1. T568A & T568B Pinouts
## 🎨 T568A & T568B Pinouts
These are the two official wiring standards for Ethernet cables.
Both work electrically the same — the only difference is the **color order**.
Most modern installations use **T568B**, but government and residential specs sometimes require **T568A**.
Both standards are electrically identical — only the colour order differs. **T568B** is the default in most commercial installations; **T568A** turns up in US government and residential specifications. Pick one and use it on both ends of every cable in the building.
### T568A Pinout
### T568A
| Pin | Color block | Name |
|-----|-------------|------|
| Pin | Colour | Name |
|-----|--------|------|
| 1 | <span style="display:inline-block;width:14px;height:14px;background:linear-gradient(90deg,#ffffff 50%,#008000 50%);border:1px solid #000;"></span> | White/Green |
| 2 | <span style="display:inline-block;width:14px;height:14px;background:#008000;border:1px solid #000;"></span> | Green |
| 3 | <span style="display:inline-block;width:14px;height:14px;background:linear-gradient(90deg,#ffffff 50%,#ff8c00 50%);border:1px solid #000;"></span> | White/Orange |
@@ -19,14 +17,12 @@ Most modern installations use **T568B**, but government and residential specs so
| 7 | <span style="display:inline-block;width:14px;height:14px;background:linear-gradient(90deg,#ffffff 50%,#8b4513 50%);border:1px solid #000;"></span> | White/Brown |
| 8 | <span style="display:inline-block;width:14px;height:14px;background:#8b4513;border:1px solid #000;"></span> | Brown |
### T568B Pinout (Most Common)
This is the standard used in most commercial networks, patch panels, and pre‑made cables.
### T568B (most common)
![T568B Pinout](data:image/jpeg;base64,/9j/4AAQSkZJRgABAQEAYABgAAD/4QBoRXhpZgAATU0AKgAAAAgABAEaAAUAAAABAAAAPgEbAAUAAAABAAAARgEoAAMAAAABAAMAAAExAAIAAAARAAAATgAAAAAAAJOoAAAD6AAAk6gAAAPocGFpbnQubmV0IDUuMC4xMwAA/9sAQwAGBAUGBQQGBgUGBwcGCAoQCgoJCQoUDg8MEBcUGBgXFBYWGh0lHxobIxwWFiAsICMmJykqKRkfLTAtKDAlKCko/9sAQwEHBwcKCAoTCgoTKBoWGigoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgo/8AAEQgBkAGQAwESAAIRAQMRAf/EAB8AAAEFAQEBAQEBAAAAAAAAAAABAgMEBQYHCAkKC//EALUQAAIBAwMCBAMFBQQEAAABfQECAwAEEQUSITFBBhNRYQcicRQygZGhCCNCscEVUtHwJDNicoIJChYXGBkaJSYnKCkqNDU2Nzg5OkNERUZHSElKU1RVVldYWVpjZGVmZ2hpanN0dXZ3eHl6g4SFhoeIiYqSk5SVlpeYmZqio6Slpqeoqaqys7S1tre4ubrCw8TFxsfIycrS09TV1tfY2drh4uPk5ebn6Onq8fLz9PX29/j5+v/EAB8BAAMBAQEBAQEBAQEAAAAAAAABAgMEBQYHCAkKC//EALURAAIBAgQEAwQHBQQEAAECdwABAgMRBAUhMQYSQVEHYXETIjKBCBRCkaGxwQkjM1LwFWJy0QoWJDThJfEXGBkaJicoKSo1Njc4OTpDREVGR0hJSlNUVVZXWFlaY2RlZmdoaWpzdHV2d3h5eoKDhIWGh4iJipKTlJWWl5iZmqKjpKWmp6ipqrKztLW2t7i5usLDxMXGx8jJytLT1NXW19jZ2uLj5OXm5+jp6vLz9PX29/j5+v/aAAwDAQACEQMRAD8A+qaKACigAooAKKACigAooAKKACigAooAKKACigAooAKKACigAooAKKACigAooAKKACigAooAKKACigAooAKKACigAooAKKACigAooAKKACigAooAKKACigAooAKKACigAooAKKACigAooAKKACigAooAKKACigAooAKKACigAooAKKACigAooAKKACigAooAKKACigAooAKKACigAooAKKACigAooAKKACigAooAKKACigAooAKKACigAooAKKACigAooAKKACigAooAKKACigAooAKKACigAooAKKACigAooAKKACigAooAKKACigAooAKKACigAooAKKACigAooAKKACigAooAKztf1zTPD1gb7W763sLMMEM1w4RAx6DJoA0a4yL4p+BJZUjj8XaIXc4A+1oMn86AOzqO3niuYUmt5ElicZV0YMrD1BFAElYNv4w8PXPiB9Ct9YspNZjLK9ksoMqkDJyvXgUAb1FABWBceMvDlt4gTQ7jWrCPWXZVWyMw80lhkDb15HNAG/WT4j8R6N4atYrnX9TtNOt5X8tJLmQIrNgnAJ74B/KgDWriv+Fr+Af+hv0T/wAC1/xoA7WmQypNEkkTBo3AZWHcHoaAH1U1XUbPSdPnvtSuYrWzgXfLNK21UHqTQBbriv8Aha3gLIH/AAl+ic/9Paf40AdrVbTr+01O0jutPuYbq2kGUlhcOrfQjigCzXG6F8S/C+veL77wxpeoGfWLPf5sXlOF+QgMA5GCQT6+vpQB2VFABRQAUUAFFABRQAUUAFFABRQAUUAFFABRQAUUAFFABRQAUUAFFABRQAUUAFFABRQAUUAFFABRQAUUAFFABRQAUUAFFABRQAUUAFFABRQAUUAFFABRQAUUAFFAHi37Xf8AyRq6/wCvyD/0Kj9rv/kjV1/1+Qf+hUAcB4K+CXgzX/gnY69fi4stTmsnuJL0XB2qylsEqflxxXn+rfDbX0+BGm+K7DxBqF5YMgludKZ2EUEW4jco3EEAgE8Dgk9qAPT/ANibU9RuNE8RafOZG0u1mikt9xJVHcNvVfQcKcepz3r0z9nW68M3nwzsZfCNpFZR5xeQK+90uQBv3seWPTBP8OKAPHvBH/J5muf71x/6KFHgn/k8zXP964/9FCgD3v4t+ObP4feDLvWbra9xjyrSAtgzTEfKv07n2Br5z+I3jxLv49C68UaPql74f8OSvFZ2drGGEk6n/WNngjPPX+FfegDmfDWj6vpP7Q3g+TxLO82s6lLDqN0H6xvLuOw/QAfTp2pniz4m2Wq/HrR/GUelalFa2nkhrORFE77QegzjnPHNAHq/7bk+3wj4ct+73zyY/wB2Mj/2avU9Q8N+Gfi74W0HVPEGlTSW7ILu3iklaN4945Vtp9hke1AHlXhz4cfAzVRZWUOo2N1qckagxR6u253wM4G7rnPArI/aX+EnhLwt4IXXvDtp/Zt3FcRwmJZGZJlYkdGJwR1yPQ0AfUtrAltbxQR58uNAi55OAMCuK+B2qXms/Cfw1fanI8l3JahXkfln2kqGPqSADQBF8fP+SOeLP+vJv5ij4+f8kc8Wf9eTfzFAHhPwA+D/AIQ8cfDJ9V1+2uft7XM0IuI7hk2KoGDjO3jPcVzfwe+EGu+PfAb3ll4xn0vTJZ5IHsQsjo2MZJUOFOc+lAHRfsk3lxpnxP8AFPhuyuXudFWOV1fdlC0coRJAOgLKevfA9K90+EXwq0f4aWFwmnyy3d/dbftF3MACwHRVA+6vJ4/WgDiPhp4l8K33x08RaVpHhCHTdbi+0/aNTWbd5u2RQ2ExgbicnH61xHwT/wCTrfGP0vv/AEclAHqfxB+OWleBfHy+Hda064+zfZhcNexPuxkMQvl4zyVAznvXlPxgtYb39rPwvb3KB4mNkWU9Dh2P8xQB0mrftF61od3a3GufD3ULDRblv3U08jJI6+oBQLuxzjNdD+2DGjfCFnZQWS+gKkjkHkcfhQB6oninSW8Ip4mN0qaM1qLzz24xGV3Zx6+3rXz74yaZf2MdKMJYKYLUSY/u+cP0zigC+f2iNa1eS9ufB/w/1DVdItDh7oyMDxzyFRgOOcZJ5ruP2XktV+Cmg/YwPmMxlx3k81s5/SgDY+EnxQ0f4laTNcaYklte2xAubOUgvHnowI6qcHB9q8G/Z4/dftJ+MYtOLf2eRe5Cj5cC4Xb7cHOKAPZtF+LcWqfGC+8BjSHjltTJm8M4KsFUN93b3z615L4KZV/bL14MQCxnAyep8paAPYvjV8UovhhY6ZczaU+oi9keMKkwj2FQD3Bz1ryn9t+4i/srwrblx5xmnkC552hVGfzIoA7/AOOnxJn8MfDlZrDT79r/AFazPlXECExWe5QN7vjAI3cepqL46f8AJt15/wBeVp/6FHQBwX7P/wAWdah0TQtAu/Det6qtxdmJ9ad3kjAeTqWKnhc469u1eh/sn/8AJEtH/wCu1z/6OagD1+igAooAKKACigAooAKwPEGo6hZ6vokVtHbfYbq68id3JLnKOwCgcDlRyc+mO9AG/UN4JjbyC2aNZ9p2GQEqD2yAQSKAJqwvDt9fTajqdneyQXK2hjAuYYzGCzAloyuTyvynOf4h6UAbtcp411e90W2N1b3logCfuraW3Z2uJM/c3hgEByBnHGcngUAdXUZlCQ+ZKVjAXcxJ4XjnmgCSua8Na3d6prmqwzQrFZRQwTWuQQ7I5kG5vTOzIHYHnnoAdLXI+LdYvtP1S2ihvoNPsvs7yy3E1hJcLuDAAFlYBABkkmgDrqry3KQWMly7Bo44zIWXoQBnIoAsVy3hHXZNYCyXF1CZJoFuI7VLd02Icch2P7wDIBZRjNAHU1geLJtRsbGW+sdRtrW3t4meVJbMzs5HQLh157Y5ySKAN+qGkNff2PaNqYibUTCpnEQ2p5mOQAScDNAF+uK8F+JbzWtRMM0ttKBbmW4jjiaNrSXftEeST5gOH+YcfJn+IUAdrRQAUUAFFABRQAUUAFFABRQAUUAFFABRQAUUAcN8ZvBEvxC8FSaDBfR2LSTxymZ4zIAFOcYBFdzQBynw/wDCY8N/D/TvDN/LFfJbQNbyPs2rKpJz8pJ7GuroA8B8HeCrX4JeKrvUbzxnY2fhnU3dV0+8QpwOUxIW5dRx05Gfw9U8f+AtA8e2dra+JbaW4htpDLEI5mjwxGDypGeKAPnD4LXqeLf2ote8QaUsjabtuJhKVIBQ4RSfTd1APNfSvgrwR4f8E2D2nhrTorOOQgyMCWeQjpuYkk0AdHtHoPypaAPIfEvwluNZ+NGl+OY9VhihsvKzZmAlnCA5+bOBnPpXqX9p2OTm9tR/22X/ABoA8/8Ai14H8T+JdQ0rUfB3il9Bu7GORGQK2yfcQRuwccYPUHrXoP8Aadh/z+2v/f1f8aAPnq++BnjXxje2n/CxvHX23T7dtwgtY+Tx2yFUH3wTivo5GDqGUgqRkEHINAFXR9NtdH0q007Tolhs7WJYYY16KqjAFWJ5o4E3zSJGg6s7AAfiaAMD4ieHpPFfgrV9ChuFtnv4DCJmUsEyRzgda2P7TsP+f21/7+r/AI0Acb8FvAk3w78G/wBh3F/HfsLh5xMkZjGGxxgk+ld2jh1DKQVIyCDkGgB1RXFxDbqGnljjU8Zdgo/WgDyLwH8Irrwx8W9a8ZzaxBcx6h9oxapAVMYkkDD5txzgDHSvX4Zo50DwukiHoyMCD+IoA8j8W/CS5174yaT43j1eGGKxMGbRoCzOIySfmzxnPpXrE93BbkCeaKInoHcLn86AOL+M/gab4h+DG0KC+jsWa4jmMzxmQALnjAI9a7pGDqGUgqeQQcg0AcZ4c8DW1p8L7XwZrTpf2qWZs5nQFPMU55HOVP48Gurlv7SGQpNdQRuOqtIAR+FAHztafA/x54YW707wR4++yaHdMS0UyFXUHjsCN2O67c19GW9zDcAmCaKUDqUcNj8qAPOfgr8J7D4aWN2y3b6hq97jLine truncated
| Pin | Color block | Name |
|-----|-------------|------|
| Pin | Colour | Name |
|-----|--------|------|
| 1 | <span style="display:inline-block;width:14px;height:14px;background:linear-gradient(90deg,#ffffff 50%,#ff8c00 50%);border:1px solid #000;"></span> | White/Orange |
| 2 | <span style="display:inline-block;width:14px;height:14px;background:#ff8c00;border:1px solid #000;"></span> | Orange |
| 3 | <span style="display:inline-block;width:14px;height:14px;background:linear-gradient(90deg,#ffffff 50%,#008000 50%);border:1px solid #000;"></span> | White/Green |
@@ -36,14 +32,22 @@ This is the standard used in most commercial networks, patch panels, and pre‑m
| 7 | <span style="display:inline-block;width:14px;height:14px;background:linear-gradient(90deg,#ffffff 50%,#8b4513 50%);border:1px solid #000;"></span> | White/Brown |
| 8 | <span style="display:inline-block;width:14px;height:14px;background:#8b4513;border:1px solid #000;"></span> | Brown |
## 2. Straight‑Through Cable (T568B ↔ T568B)
> 💡 The mnemonic for T568B: **orange, green, blue, brown** — striped before solid, except that the blue pair sits inside the green pair on pins 4 and 5.
This is the **standard Ethernet patch cable** used to connect:
- computers → switches
- switches → routers
- wall jacks → patch panels
## 🔌 What Each Pin Does
Both ends use the **same** wiring standard.
| Pins | Pair | 10/100BASE-TX | 1000BASE-T and faster |
|-------|------|----------------------|-----------------------|
| 1 – 2 | 2 | Transmit (TX+ / TX−) | Bi-directional pair A |
| 3 – 6 | 3 | Receive (RX+ / RX−) | Bi-directional pair B |
| 4 – 5 | 1 | Unused | Bi-directional pair C |
| 7 – 8 | 4 | Unused | Bi-directional pair D |
Gigabit and 10 Gigabit Ethernet use **all four pairs in both directions** at once — which is why a "Fast Ethernet works, Gigabit doesn't" fault is almost always a bad termination on pins 4, 5, 7 or 8.
## 🔗 Straight-Through Cable (T568B ↔ T568B)
The standard patch cable: computer → switch, switch → router, wall jack → patch panel. Both ends use the same standard, so every pin lands on the same pin.
```text
End A (T568B) End B (T568B)
@@ -58,17 +62,9 @@ End A (T568B) End B (T568B)
8 Brown ------------------ Brown 8
```
A straight‑through cable keeps all pins aligned. Modern devices use auto‑MDI/MDI‑X, so this is the only cable type you typically need today.
## 🔀 Crossover Cable (T568A ↔ T568B)
## 3. Crossover Cable (T568A ↔ T568B)
This cable swaps the transmit and receive pairs.
It was historically used for:
- switch ↔ switch
- computer ↔ computer
- router ↔ router
Today, almost all hardware auto‑detects, so crossover cables are mostly legacy.
Historically needed for switch ↔ switch, PC ↔ PC and router ↔ router links. Every device made in the last twenty years negotiates **Auto-MDI/MDI-X**, so this is now a legacy cable — but it is still what you grab for two ancient devices, or a lab exercise.
```text
End A (T568A) End B (T568B)
@@ -78,46 +74,71 @@ End A (T568A) End B (T568B)
3 White/Orange ------------------ White/Green 3
6 Orange ------------------ Green 6
Pins 4, 5, 7, 8 remain straight‑through.
Pins 4, 5, 7 and 8 stay straight-through for 10/100.
```
Only the pairs used for data (1–2 and 3–6) are swapped.
Gigabit Ethernet uses all four pairs, but still auto‑corrects internally.
A true **gigabit crossover** also swaps the other two pairs: 4↔7 and 5↔8.
## 4. Pair Mapping (Color‑Block Reference)
## 🧵 Pair Mapping
Ethernet uses **four twisted pairs**, each with a solid and striped conductor.
This table helps you quickly identify which colors belong together.
Four twisted pairs, each a solid conductor and its white-striped partner. Keeping a pair together is what cancels noise.
| Pair | T568A colors | T568B colors | Explanation |
|------|--------------|--------------|-------------|
| Pair 1 (4–5) | 🔵 / 🔵⬜ | same | Used for PoE power delivery |
| Pair 2 (1–2) | 🟧⬜ / 🟧 | 🟧⬜ / 🟧 | Primary transmit/receive pair |
| Pair 3 (3–6) | 🟩⬜ / 🟩 | 🟩⬜ / 🟩 | Secondary transmit/receive pair |
| Pair 4 (7–8) | 🟫⬜ / 🟫 | same | Not used in 10/100 Mbps, used in Gigabit |
| Pair | Pins | T568A colours | T568B colours | Notes |
|------|-------|---------------|---------------|-----------------------------------------|
| 1 | 4 – 5 | 🔵⬜ / 🔵 | same | Sits inside the middle pair |
| 2 | 1 – 2 | 🟩⬜ / 🟩 | 🟧⬜ / 🟧 | The pair that swaps between the standards |
| 3 | 3 – 6 | 🟧⬜ / 🟧 | 🟩⬜ / 🟩 | Split across pins 3 and 6 |
| 4 | 7 – 8 | 🟫⬜ / 🟫 | same | Idle on 10/100, active on Gigabit |
Pair grouping matters for signal integrity — never untwist more than ~13 mm when terminating.
> ⚠️ Never untwist more than about **13 mm (½ inch)** at the connector, and never split a pair across two pairs' positions — a "works but slow" cable is usually a split pair, and only a certifier will tell you.
## 5. Cat5e / Cat6 / Cat6A Comparison
## ⚡ Power over Ethernet
This helps you choose the right cable for your installation.
| Standard | Name | Power at the port | Pairs used |
|-----------------|---------|-------------------|-----------------------|
| IEEE 802.3af | PoE | 15.4 W | Two pairs (Mode A or B) |
| IEEE 802.3at | PoE+ | 30 W | Two pairs |
| IEEE 802.3bt T3 | PoE++ | 60 W | All four pairs |
| IEEE 802.3bt T4 | PoE++ | 100 W | All four pairs |
| Feature | Cat5e | Cat6 | Cat6A | Explanation |
|--------|-------|------|-------|-------------|
| Max speed | 1 Gbps | 10 Gbps (≤55 m) | 10 Gbps (100 m) | Cat6A is the only one supporting full‑distance 10G |
| Bandwidth | 100 MHz | 250 MHz | 500 MHz | Higher bandwidth = better noise resistance |
| Shielding | UTP/STP | UTP/STP | Often STP | Cat6A often shielded for long‑distance 10G |
| PoE | Yes | Yes | Yes | All support PoE/PoE+/PoE++ |
| Typical use | Home | Office | Data center | Cat6A is bulkier but future‑proof |
- **Mode A** puts power on the data pairs 1–2 and 3–6; **Mode B** uses the spare pairs 4–5 and 7–8. A powered device must accept either.
- Because 802.3bt energises all four pairs, sloppy terminations on 4/5/7/8 now break power as well as gigabit speed.
- Power raises cable temperature; in a full, bundled tray, derate the length or move to a shielded, larger-gauge cable.
## 6. Termination Best Practices
## 📏 Categories & Limits
These rules ensure your cable meets Cat6 performance:
| Feature | Cat5e | Cat6 | Cat6A | Cat8 |
|-------------|-----------|-------------------|------------------|-----------------|
| Max speed | 1 Gbps | 10 Gbps (≤ 55 m) | 10 Gbps (100 m) | 25/40 Gbps (30 m) |
| Bandwidth | 100 MHz | 250 MHz | 500 MHz | 2000 MHz |
| Shielding | UTP/STP | UTP/STP | Often F/UTP | Always shielded |
| Typical use | Home, PoE cameras | Office floor | 10G to the desk, dense PoE | Data-centre top-of-rack |
- Keep twist pairs intact as long as possible
- Use Cat6‑rated connectors (larger conductor spacing)
- Ensure the jacket is inside the strain‑relief clamp
- Test with a cable tester
- Use T568B unless your environment requires T568A
- Maximum channel length is **100 m**: 90 m of solid-core in the wall plus 10 m of stranded patch leads.
- Use **solid** conductor for permanent runs and **stranded** for patch cords — stranded flexes, solid does not survive repeated bending; connectors are rated for one or the other.
Cat6 performance depends heavily on maintaining pair geometry and minimizing untwist.
## 🔧 Termination Checklist
- Strip only the jacket, never nick the insulation of the conductors.
- Keep the pairs twisted right up to the point of termination.
- The jacket must sit **inside** the connector's strain-relief crimp.
- Use Cat6-rated plugs (wider conductor spacing) with a load bar for 23 AWG cable.
- For keystone jacks, follow the colour code printed on the jack and punch down with a 110 tool — do not push the wires in with a screwdriver.
- Terminate to **T568B** unless the site specification says otherwise, and label both ends.
- Test every run: a cheap continuity tester catches swaps and opens; only a certifier catches split pairs, NEXT and length problems.
## 🧯 Common Faults
| Symptom | Likely cause |
|-------------------------------------|-------------------------------------------------------|
| No link at all | Open or crossed pair on 1–2 / 3–6, or a bad crimp |
| 100 Mbps instead of 1 Gbps | One of pins 4, 5, 7, 8 not terminated properly |
| Link drops under load | Split pair, excessive untwist, or a run over 100 m |
| PoE device does not power up | Break on the pairs the injector uses, or a budget limit on the switch |
| Errors only on hot afternoons | Bundled PoE runs overheating, or cable near a power line |
## 📚 Resources
- [TIA-568 wiring standards overview](https://en.wikipedia.org/wiki/TIA-568)
- [Power over Ethernet standards](https://en.wikipedia.org/wiki/Power_over_Ethernet)
- [Ethernet physical layers and distances](https://en.wikipedia.org/wiki/Ethernet_physical_layer)
+119 -109
View File
@@ -1,129 +1,139 @@
# 🖥️ GNU Screen Command Cheatsheet
**GNU Screen** is a terminal multiplexer: it runs several shell sessions inside one terminal, keeps them alive after you disconnect, and lets you reattach later from anywhere.
GNU Screen is a terminal multiplexer that lets you run multiple shell sessions inside a single terminal window, detach them, and reattach later.
> ℹ️ Every shortcut starts with the **prefix**, `Ctrl+a` by default: press the prefix, release it, then press the key. In the tables below `Ctrl+a c` means "prefix, then `c`".
## 📦 Starting & Managing Screen Sessions
## 🚀 Sessions from the Shell
- **Start a new session**
```bash
screen
```
- **Start with a session name**
```bash
screen -S session_name
```
- **List running sessions**
```bash
screen -ls
```
- **Attach to a session**
```bash
screen -r session_name
```
- **Attach to the last detached session**
```bash
screen -r
```
- **Kill a session**
```bash
screen -X -S session_name quit
```
```bash
# start a session
screen
## ⌨️ Essential Key Bindings
# start a named session
screen -S <name>
> Default prefix key: `Ctrl-a` (press before other keys)
# list running sessions
screen -ls
| Key Binding | Action |
|-|--|
| `Ctrl-a c` | Create new window |
| `Ctrl-a n` | Next window |
| `Ctrl-a p` | Previous window |
| `Ctrl-a "` | List all windows |
| `Ctrl-a 0..9` | Switch to window by number |
| `Ctrl-a A` | Rename current window |
| `Ctrl-a d` | Detach session |
| `Ctrl-a ?` | Help (show key bindings) |
| `Ctrl-a k` | Kill current window |
| `Ctrl-a \` | Kill all windows & exit screen |
| `Ctrl-a x` | Lock screen |
| `Ctrl-a S` | Split screen horizontally |
| `Ctrl-a |` | Split screen vertically |
| `Ctrl-a tab` | Switch focus between regions |
| `Ctrl-a Q` | Close all regions except current |
# reattach to a session
screen -r <name>
## 🔀 Window & Region Management
# reattach, detaching whoever else is attached
screen -d -r <name>
- **Split horizontally**
```bash
Ctrl-a S
```
- **Split vertically**
```bash
Ctrl-a |
```
- **Switch region focus**
```bash
Ctrl-a tab
```
- **Remove all splits except current**
```bash
Ctrl-a Q
```
# reattach if possible, otherwise create the session
screen -R <name>
## 📂 Session Persistence
# attach in parallel with another client (shared screen)
screen -x <name>
- **Detach session (keep running in background)**
```bash
Ctrl-a d
```
- **Reattach session**
```bash
screen -r session_name
```
# start a detached session running a command
screen -dmS <name> <command>
# run a command inside an existing session
screen -S <name> -X stuff "make test\n"
# quit a session from outside
screen -X -S <name> quit
# clean up dead sessions
screen -wipe
```
## ⌨️ Windows
| Keys | Action |
|-----------------|-------------------------------------|
| `Ctrl+a c` | Create a window |
| `Ctrl+a A` | Rename the current window |
| `Ctrl+a n` | Next window |
| `Ctrl+a p` | Previous window |
| `Ctrl+a Ctrl+a` | Toggle between the last two windows |
| `Ctrl+a 0…9` | Jump to a window by number |
| `Ctrl+a "` | Choose a window from a list |
| `Ctrl+a w` | Show the window bar |
| `Ctrl+a k` | Kill the current window |
## 🪟 Split Regions
| Keys | Action |
|--------------|-------------------------------------------|
| `Ctrl+a S` | Split into top / bottom regions |
| `Ctrl+a \|` | Split into left / right regions |
| `Ctrl+a Tab` | Move the focus to the next region |
| `Ctrl+a X` | Close the focused region |
| `Ctrl+a Q` | Close every region except the focused one |
> 💡 A new region starts empty — press `Ctrl+a c` to open a shell in it, or `Ctrl+a "` to move an existing window there.
## 🔌 Session Control
| Keys | Action |
|--------------|------------------------------------------|
| `Ctrl+a d` | Detach and leave everything running |
| `Ctrl+a D D` | Detach and log out |
| `Ctrl+a x` | Lock the terminal (password required) |
| `Ctrl+a :` | Command prompt (`:quit`, `:sessionname`) |
| `Ctrl+a ?` | List every key binding |
| `Ctrl+a \` | Kill every window and quit screen |
| `Ctrl+a a` | Send a literal `Ctrl+a` (nested screens) |
## 📋 Copy Mode & Scrollback
| Keys | Action |
|-------------|-------------------------------------------------|
| `Ctrl+a [` | Enter copy/scrollback mode |
| `space` | Start the selection, then `space` again to copy |
| `Ctrl+a ]` | Paste the buffer |
| `/` and `?` | Search forward / backward |
| `Ctrl+a >` | Write the buffer to a file |
| `Esc` | Leave copy mode |
## 📝 Logging & Monitoring
| Keys | Action |
|------------|-----------------------------------------------|
| `Ctrl+a H` | Toggle logging of the window to `screenlog.n` |
| `Ctrl+a M` | Notify me when this window shows activity |
| `Ctrl+a _` | Notify me when this window goes quiet |
| `Ctrl+a C` | Clear the window |
```bash
# start a session with logging enabled
screen -L -S <name>
# keep 5000 lines of scrollback
screen -h 5000
```
## ⚙️ Configuration
- **Default config file:** `~/.screenrc`
- Example `.screenrc`:
```bash
# Start with a named session
startup_message off
defscrollback 5000
hardstatus alwayslastline
hardstatus string '%{= kG}[%H] %{= kw}%?%-Lw%?%{= kR}%n*%f %t%?(%u)%?%{= kw}%?%+Lw%? %{= kG}[%H]'
```
Settings live in `~/.screenrc`:
## 🛠️ Useful Options
```bash
# no splash screen, plenty of scrollback
startup_message off
defscrollback 10000
- **Scrollback buffer size**
```bash
screen -h 5000
```
- **Execute command in new screen**
```bash
screen -dmS session_name command
```
- **Log output**
```bash
Ctrl-a H
```
(toggle logging in current window)
# a status line with the window list and the hostname
hardstatus alwayslastline
hardstatus string '%{= kG}[%H] %{= kw}%?%-Lw%?%{= kR}%n*%f %t%?(%u)%?%{= kw}%?%+Lw%?'
## 🚪 Exiting Screen
# use Ctrl+z as the prefix instead of Ctrl+a
# escape ^z^z
- **Kill current window**
```bash
Ctrl-a k
```
- **Kill all windows & quit**
```bash
Ctrl-a \
```
- **Exit normally**
- Close all programs in the window, then type `exit`.
# turn on mouse-wheel scrolling in the scrollback
termcapinfo xterm* ti@:te@
```
## 🧠 Tips
- Use **named sessions** (`screen -S`) to avoid confusion.
- Combine with **SSH** to keep remote jobs running after disconnect.
- Use `.screenrc` to customize behavior and status line.
- Name your sessions (`screen -S deploy`) — `screen -ls` is unreadable once you have three sessions called `12345.pts-0.host`.
- Over SSH, start work inside screen: the job survives a dropped connection, and `screen -d -r` picks it back up.
- Nested screens (local + remote): press `Ctrl+a a` to send the prefix through to the inner session.
- `screen -x` attaches several clients to the same session — handy for pair debugging.
## 📚 Resources
- [GNU Screen manual](https://www.gnu.org/software/screen/manual/screen.html)
- [Manual page](https://man7.org/linux/man-pages/man1/screen.1.html)
- Built-in help: `Ctrl+a ?`
+229 -167
View File
@@ -1,211 +1,273 @@
# 🧾 `sed` Command Cheatsheet (Stream Editor)
`sed` is a powerful Unix utility for parsing and transforming text using **stream editing**.
## 📌 Basic Syntax
**`sed`** is the stream editor: it reads input line by line into the *pattern space*, applies a script to each line, and prints the result. It never needs the whole file in memory, so it happily edits gigabytes.
```bash
sed [OPTIONS] 'script' [file...]
sed [options] 'script' [file...]
```
- `'script'`: One or more editing commands.
- `[file]`: Input file(s). If omitted, `sed` reads from standard input.
> ℹ️ Examples here use **GNU sed** (Linux). BSD/macOS differs on `-i`, `-E` and the `a`/`i`/`c` commands — see the callouts. `brew install gnu-sed` gives you `gsed` on macOS.
## ⚙️ Common Options
## ⚙️ Options
| Option | Description |
|--------|-------------|
| `-n` | Suppress automatic printing of pattern space. Use `p` to print explicitly. |
| `-e` | Add multiple editing commands. |
| `-i` | Edit files in-place (with optional backup: `-i.bak`). |
| `-f` | Read commands from a file. |
| Option | Description |
|----------------|--------------------------------------------------------------------|
| `-n` | Do not print automatically — pair it with `p` |
| `-e <script>` | Add a script; repeat for several |
| `-f <file>` | Read the script from a file |
| `-i[SUFFIX]` | Edit files in place, optionally keeping a backup |
| `-E` (or `-r`) | Extended regular expressions — no backslashes before `(`, `+`, `?` |
| `-s` | Treat each file separately instead of one long stream |
| `-z` | Split on NUL instead of newline (pairs with `find -print0`) |
| `--debug` | Show how sed executes the script (GNU 4.6+) |
## ✂️ Basic Commands
## 📍 Addressing
| Command | Description |
|---------|-------------|
| `p` | Print the current pattern space. |
| `d` | Delete the current pattern space. |
| `s` | Substitute text using regex. |
| `q` | Quit after processing first match. |
| `a` | Append text after current line. |
| `i` | Insert text before current line. |
| `c` | Replace line with new text. |
| `y` | Translate characters (like `tr`). |
An address decides *which* lines a command applies to. Without one, the command applies to every line.
## 🔍 Substitution Syntax
| Address | Matches |
|-----------------|-----------------------------------------------------|
| `5` | Line 5 |
| `$` | The last line |
| `/regex/` | Every line matching the pattern |
| `\%regex%` | Same, with `%` as the delimiter (handy for paths) |
| `2,7` | Lines 2 to 7 |
| `2,$` | Line 2 to the end |
| `/start/,/end/` | From the first `start` to the next `end`, inclusive |
| `2,+3` | Line 2 and the three lines after it |
| `0~3` | Every third line (GNU) |
| `2~5` | Line 2, then every fifth line (GNU) |
| `/regex/!` | Invert — every line that does **not** match |
```bash
# print line 5
sed -n '5p' file
# print lines 3 to 5
sed -n '3,5p' file
# print everything between two markers
sed -n '/BEGIN/,/END/p' file
# delete every line except the ones matching
sed -n '/keep/p' file
sed '/keep/!d' file
# count the lines (like wc -l)
sed -n '$=' file
```
## 🔁 Substitution
```bash
sed 's/pattern/replacement/flags' file
```
### Flags
| Flag | Description |
|------|-------------|
| `g` | Global replacement (all matches in line). |
| `i` | Case-insensitive matching. |
| `p` | Print the line if substitution occurred. |
| `n` | Replace nth occurrence only. |
### Examples
| Flag | Effect |
|--------|------------------------------------------------------|
| *none* | Replace the first match on each line |
| `g` | Replace every match on the line |
| `i` | Case-insensitive matching |
| `p` | Print the line when a replacement happened |
| `2` | Replace only the second match — any number works |
| `2g` | Replace from the second match to the end of the line |
| `w f` | Write the changed lines to file `f` |
| `e` | Run the result as a shell command (GNU) |
```bash
sed 's/foo/bar/' file # Replace first 'foo' with 'bar'
sed 's/foo/bar/g' file # Replace all 'foo' with 'bar'
sed 's/foo/bar/2' file # Replace second 'foo' only
sed 's/foo/bar/ip' file # Case-insensitive + print
```
# first match on each line
sed 's/foo/bar/' file
## 🗑️ Deletion with `sed`
# every match on the line
sed 's/foo/bar/g' file
`sed` can delete lines based on line numbers, patterns, or ranges.
# only the second match
sed 's/foo/bar/2' file
### 🔢 Delete by Line Number
# every match, ignoring case
sed 's/foo/bar/gI' file
```bash
sed '2d' file # Delete line 2
sed '5,10d' file # Delete lines 5 through 10
```
# & stands for the whole match — wrap every number in brackets
sed 's/[0-9]\+/[&]/g' file
### 🔍 Delete by Pattern
```bash
sed '/error/d' file # Delete lines containing 'error'
sed '/^$/d' file # Delete blank lines
sed '/^#/d' file # Delete comment lines (starting with #)
```
### 🧮 Delete by Range and Pattern
```bash
sed '1,/pattern/d' file # Delete from line 1 to first match of 'pattern'
sed '/start/,/end/d' file # Delete lines between 'start' and 'end' (inclusive)
```
### 🧠 Conditional Deletion
```bash
sed -n '/pattern/!p' file # Print only lines NOT matching 'pattern'
```
### 🧹 Delete Last Line
```bash
sed '$d' file # Delete the last line
```
## 📍 Addressing Lines
### Line Numbers
```bash
sed '2d' file # Delete line 2
sed '3,5p' file # Print lines 3 to 5
```
### Patterns
```bash
sed '/error/d' file # Delete lines containing 'error'
sed '/^#/d' file # Delete comment lines
```
### Combined
```bash
sed '1,/pattern/d' file # Delete from line 1 to first match of 'pattern'
```
## 🧪 Advanced Substitution
### Using Capture Groups
```bash
# capture groups: \( \) in basic regex, ( ) with -E
sed 's/\(foo\)bar/\1baz/' file
```
sed -E 's/(\w+)@(\w+)/\2 at \1/' file
- `\(...\)` captures a group.
- `\1`, `\2`, etc., refer to captured groups.
### Escaping Special Characters
```bash
sed 's/\/usr\/bin/\/usr\/local\/bin/' file
```
Or use alternate delimiter:
```bash
# any character can be the delimiter — pick one that is not in the text
sed 's|/usr/bin|/usr/local/bin|' file
sed 's#http://#https://#' file
# GNU case conversion: upper-case the first word
sed 's/\w\+/\U&/' file
# ...and lower-case a whole line
sed -E 's/(.*)/\L\1/' file
# only substitute on lines that match an address
sed '/^deb /s/http:/https:/' sources.list
```
## ✂️ Command Reference
| Command | Does |
|----------------------|-----------------------------------------------------------------|
| `p` | Print the pattern space |
| `d` | Delete it and start the next cycle |
| `s` | Substitute |
| `y` | Transliterate characters, like `tr` |
| `a text` | Append a line after the current one |
| `i text` | Insert a line before the current one |
| `c text` | Replace the matched line(s) |
| `q` | Quit — `q5` also sets the exit status |
| `Q` | Quit without printing the current line |
| `r file` | Read a file in after the current line |
| `w file` | Write the pattern space to a file |
| `=` | Print the current line number |
| `n` / `N` | Load the next line, replacing / appending to the pattern space |
| `D` / `P` | Delete / print up to the first newline of the pattern space |
| `h` `H` | Copy / append the pattern space into the hold space |
| `g` `G` | Copy / append the hold space into the pattern space |
| `x` | Swap the pattern and hold spaces |
| `b` `t` `T` `:label` | Branching — jump, jump if a substitution happened, jump if none |
## 🗑 Deleting & Printing
```bash
# by line number
sed '2d' file
sed '5,10d' file
sed '$d' file
# lines containing a word
sed '/error/d' file
# blank lines
sed '/^$/d' file
# comment lines
sed '/^#/d' file
# blank or whitespace-only lines
sed '/^\s*$/d' file
# strip comments and blank lines from a config
sed -e '/^#/d' -e '/^$/d' /etc/ssh/sshd_config
# from the top up to the first match
sed '1,/pattern/d' file
# everything between two markers
sed '/start/,/end/d' file
# keep only what matches
sed -n '/pattern/p' file
```
## 🧵 Insert, Append & Change
```bash
# GNU one-line form
sed '/pattern/a appended line' file
sed '/pattern/i inserted line' file
sed '/pattern/c replacement line' file
# portable form (works on BSD/macOS too)
sed '/pattern/a\
appended line' file
# add a line at the very top or bottom
sed '1i #!/bin/bash' file
sed '$a # end of file' file
# insert the contents of another file after a marker
sed '/INCLUDE HERE/r snippet.txt' file
```
## 🧠 Multi-line & Hold Space
The hold space is a second buffer that survives between lines — that is what makes sed more than a line-at-a-time filter.
```bash
# join every pair of lines
sed 'N;s/\n/ /' file
# reverse the file (what tac does)
sed -n '1!G;h;$p' file
# print the line before each match
sed -n '/pattern/{x;p;x};h' file
# print the line after each match
sed -n '/pattern/{n;p}' file
# squeeze runs of blank lines into one
sed '/^$/{N;/^\n$/D}' file
# delete the last two lines
sed 'N;$!P;$!D;$d' file
```
## 🧨 In-Place Editing
```bash
sed -i 's/foo/bar/g' file # Edit file directly
sed -i.bak 's/foo/bar/g' file # Backup original as file.bak
# GNU: edit in place
sed -i 's/foo/bar/g' file
# keep a backup as file.bak
sed -i.bak 's/foo/bar/g' file
# several files at once
sed -i 's/foo/bar/g' *.conf
# every matching file in a tree
find . -name '*.py' -exec sed -i 's/old_api/new_api/g' {} +
```
## 📂 Multiple Commands
> ⚠️ **BSD/macOS `sed` requires an argument to `-i`.** `sed -i '' 's/foo/bar/' file` edits without a backup there, while the same command on GNU sed reads `''` as the script. Write `sed -i.bak` for something that works on both, or use `gsed`.
>
> 💡 Always run the command without `-i` first and read the output. `-i` has no undo.
### Inline
## 🧰 Recipes
```bash
sed -e 's/foo/bar/' -e '/baz/d' file
```
# trim leading and trailing whitespace
sed 's/^[[:space:]]*//; s/[[:space:]]*$//' file
### Block Syntax
# collapse repeated spaces
sed 's/ */ /g' file
```bash
sed '
s/foo/bar/
s/baz/qux/
' file
```
## 🧵 Append, Insert, Change
```bash
sed '/pattern/a\Text to append' file
sed '/pattern/i\Text to insert' file
sed '/pattern/c\New line content' file
```
## 🔄 Translate Characters
```bash
sed 'y/abc/ABC/' file # a→A, b→B, c→C
```
## 🧠 Useful Recipes
### Remove Blank Lines
```bash
sed '/^$/d' file
```
### Remove Leading/Trailing Whitespace
```bash
sed 's/^[ \t]*//' file # Remove leading
sed 's/[ \t]*$//' file # Remove trailing
```
### Replace Tabs with Spaces
```bash
# tabs to four spaces
sed 's/\t/ /g' file
```
### Number Lines
# strip Windows carriage returns
sed 's/\r$//' file
```bash
# strip HTML tags
sed -e 's/<[^>]*>//g' page.html
# number the lines
sed = file | sed 'N;s/\n/\t/'
# print a specific line and quit early (fast on huge files)
sed -n '1000{p;q}' file
# extract the value of a key from a config file
sed -n 's/^Port[[:space:]]*//p' /etc/ssh/sshd_config
# replace only in lines between two markers
sed '/BEGIN/,/END/ s/old/new/g' file
# change a value only on the first match, then stop
sed '0,/version:/s//version: 2/' file
# comment out a line
sed '/^ExecStart/s/^/#/' unit.service
# uncomment a line
sed '/^#ExecStart/s/^#//' unit.service
```
## 📚 Resources
- `man sed`
- GNU sed manual: [https://www.gnu.org/software/sed/manual/sed.html](https://www.gnu.org/software/sed/manual/sed.html)
- [GNU sed manual](https://www.gnu.org/software/sed/manual/sed.html)
- [One-liners, annotated (sed1line)](https://sed.sourceforge.io/sed1line.txt)
- Manual page: `man 1 sed`
+160 -47
View File
@@ -1,57 +1,170 @@
# 🖥️ tmux Linux Command Cheatsheet
**tmux** is a terminal multiplexer: it keeps shell sessions alive on the server, splits one terminal into windows and panes, and lets you detach and reattach without losing anything.
## 🔑 Basics
- **Start new session**: `tmux`
- **Start named session**: `tmux new -s <name>`
- **List sessions**: `tmux ls`
- **Attach to last session**: `tmux attach`
- **Attach to named session**: `tmux attach -t <name>`
- **Detach from session**: `Ctrl+b d`
- **Kill session**: `tmux kill-session -t <name>`
- **Rename session**: `Ctrl+b $`
> ℹ️ Almost every shortcut starts with the **prefix**, `Ctrl+b` by default: press the prefix, release it, then press the key. In the tables below `Ctrl+b c` means "prefix, then `c`".
## 📂 Windows (like tabs)
- **Create new window**: `Ctrl+b c`
- **Rename window**: `Ctrl+b ,`
- **Next window**: `Ctrl+b n`
- **Previous window**: `Ctrl+b p`
- **List windows**: `Ctrl+b w`
- **Switch to window #**: `Ctrl+b <0-9>`
- **Kill current window**: `Ctrl+b &`
## 🚀 Sessions from the Shell
```bash
# start a session
tmux
# start a named session
tmux new -s <name>
# start a named session in the background
tmux new -s <name> -d
# attach if the session exists, create it otherwise
tmux new-session -A -s <name>
# list sessions
tmux ls
# attach to the most recent session
tmux attach
# attach to a named session
tmux attach -t <name>
# attach and detach any other client (steal the session)
tmux attach -d -t <name>
# rename a session
tmux rename-session -t <old> <new>
# kill one session
tmux kill-session -t <name>
# kill the server and every session on it
tmux kill-server
```
## ⌨️ Sessions
| Keys | Action |
|------------|----------------------------------|
| `Ctrl+b d` | Detach from the session |
| `Ctrl+b s` | Interactive session list |
| `Ctrl+b $` | Rename the current session |
| `Ctrl+b (` | Previous session |
| `Ctrl+b )` | Next session |
| `Ctrl+b w` | Tree of every session and window |
## 📂 Windows (tabs)
| Keys | Action |
|--------------|----------------------------------|
| `Ctrl+b c` | Create a window |
| `Ctrl+b ,` | Rename the current window |
| `Ctrl+b n` | Next window |
| `Ctrl+b p` | Previous window |
| `Ctrl+b l` | Last (previously used) window |
| `Ctrl+b 0…9` | Jump to a window by number |
| `Ctrl+b w` | Choose a window from a list |
| `Ctrl+b f` | Find a window by name |
| `Ctrl+b .` | Move the window to another index |
| `Ctrl+b &` | Kill the current window |
## 🪟 Panes (splits)
- **Split vertically**: `Ctrl+b %`
- **Split horizontally**: `Ctrl+b "`
- **Move between panes**: `Ctrl+b <arrow keys>`
- **Cycle panes**: `Ctrl+b o`
- **Toggle pane zoom**: `Ctrl+b z`
- **Kill pane**: `Ctrl+b x`
- **Swap panes**: `Ctrl+b {` (swap with previous), `Ctrl+b }` (swap with next)
## 📋 Copy Mode
- **Enter copy mode**: `Ctrl+b [`
- **Start selection**: `<Space>`
- **Copy selection**: `<Enter>`
- **Quit copy mode**: `q`
- **Paste buffer**: `Ctrl+b ]`
| Keys | Action |
|-------------------|--------------------------------------------|
| `Ctrl+b %` | Split into left / right panes |
| `Ctrl+b "` | Split into top / bottom panes |
| `Ctrl+b ←↑↓→` | Move the focus in that direction |
| `Ctrl+b o` | Cycle through the panes |
| `Ctrl+b ;` | Jump to the previously focused pane |
| `Ctrl+b q` | Show pane numbers (press one to jump) |
| `Ctrl+b z` | Zoom the pane to full screen, and back |
| `Ctrl+b x` | Kill the current pane |
| `Ctrl+b {` / `}` | Swap the pane with the previous / next one |
| `Ctrl+b space` | Cycle through the preset layouts |
| `Ctrl+b !` | Break the pane out into its own window |
| `Ctrl+b Alt+←↑↓→` | Resize the pane in that direction |
## 📋 Copy Mode & Scrollback
| Keys | Action |
|-------------|-----------------------------------------------|
| `Ctrl+b [` | Enter copy mode (scroll with the arrows/PgUp) |
| `space` | Start the selection |
| `Enter` | Copy the selection and leave copy mode |
| `Ctrl+b ]` | Paste the buffer |
| `/` and `?` | Search forward / backward (vi mode) |
| `q` | Leave copy mode |
> 💡 `setw -g mode-keys vi` gives you vi navigation in copy mode; `v` then starts the selection and `y` yanks it.
## ⚙️ Configuration
- **Reload config**: `tmux source-file ~/.tmux.conf`
- **Show options**: `tmux show-options -g`
- **List all shortcuts**: `Ctrl+b ?`
## 🧭 Useful Commands
- **Command prompt**: `Ctrl+b :`
- **List sessions/windows/panes**: `tmux info`
- **Kill server (all sessions)**: `tmux kill-server`
```bash
# reload the configuration without restarting
tmux source-file ~/.tmux.conf
## 📝 Notes
- **Prefix key**: Default is `Ctrl+b`. Press it before any shortcut.
- **Custom prefix**: Can be changed in `~/.tmux.conf` (e.g., `set -g prefix C-a`).
# every key binding currently active
tmux list-keys
### Example Workflow
1. Start a named session: `tmux new -s dev`
2. Split into panes: `Ctrl+b %` and `Ctrl+b "`
3. Create multiple windows: `Ctrl+b c`
4. Detach: `Ctrl+b d`
5. Reattach later: `tmux attach -t dev`
# every option and its value
tmux show-options -g
```
A reasonable `~/.tmux.conf` starting point:
```bash
# use Ctrl+a as the prefix, like GNU Screen
set -g prefix C-a
unbind C-b
bind C-a send-prefix
# number windows from 1 and close the gaps
set -g base-index 1
set -g renumber-windows on
# mouse: focus panes, resize splits, scroll
set -g mouse on
# vi keys in copy mode, and a longer scrollback
setw -g mode-keys vi
set -g history-limit 10000
# reload with prefix + r
bind r source-file ~/.tmux.conf \; display "Config reloaded"
```
## 🧭 Scripting & Automation
```bash
# run a command in a target pane
tmux send-keys -t <session>:<window>.<pane> "make test" Enter
# print the contents of a pane
tmux capture-pane -p -t <session>:<window>
# open the command prompt inside tmux (prefix + :)
tmux command-prompt
# build a session non-interactively
tmux new -d -s dev -n editor
tmux split-window -t dev -h
tmux attach -t dev
```
## 📄 Example Workflow
```bash
# 1. start a named session
tmux new -s dev
# 2. split the window: prefix % (left/right), prefix " (top/bottom)
# 3. add another window with prefix c
# 4. detach with prefix d — everything keeps running
# 5. come back later
tmux attach -t dev
```
## 📚 Resources
- [tmux wiki and getting-started guide](https://github.com/tmux/tmux/wiki)
- [Manual page](https://man7.org/linux/man-pages/man1/tmux.1.html)
- Built-in help: `Ctrl+b ?` lists every active binding
File diff suppressed because it is too large. Load diff
+247 -248
View File
@@ -1,248 +1,247 @@
> Credit to the original author: https://gist.github.com/jmaccabee/550a0b9fcfdc7e6b170cd34c6ec7bc56
## 0. XPath Examples.
> More: http://xpath.alephzarro.com/content/cheatsheet.html
| XPath | Description |
| -------------- | ----------- |
| `//hr[@class="edge" and position()=1]` | every first hr of 'edge' class
| `//table[count(tr)=1 and count(tr/td)=2]` | all tables with 1 row and 2 cols
| `//div/form/parent::*` | all divs that have form
| `./div/b` | a relative path
| `//table[parent::div[@class="pad"] and not(@id)]//a` | any anchor in a table without id, contained in a div of "pad" class
| `/html/body/div/*[preceding-sibling::h4]` | give me whatever after h4
| `//tr/td[font[@class="head" and text()="TRACK"]]` | all td that has font of a "head" class and text "TRACK"
| `./table/tr[last()]` | the last row of a table
| `//rdf:Seq/rdf:li/em:id` | using namespaces
| `//a/@href` | hrefs of all anchors
| `//*[count(*)=3]` | all nodes with 3 children
| `//var\|//acronym` | all vars and acronyms
## 1. General.
| XPath | Description |
| -------------- | ----------- |
| `/html` | whole web page (css: html)
| `/html/body` | whole web page body (css: body)
| `//text()` | all text nodes of web page
| `/html/body/.../.../.../E` | element \<E\> by absolute reference (css: body > … > … > … > E)
## 2. Tag.
| XPath | Description |
| -------------- | ----------- |
| `//E` | element \<E\> by relative reference (css: E)
| `(//E)[2]` | second \<E\> element anywhere on page
| `//img` | image element (css: img)
| `//E[@A]` | element \<E\> with attribute A (css: E[A])
| `//E[@A="t"]` | element \<E\> with attribute A containing text 't' exactly (css: E[A='t'])
| `//E[contains(@A,"t")]` | element \<E\> with attribute A containing text 't' (css: E[A*='t'])
| `//E[starts-with(@A, "t")]` | element \<E\> whose attribute A begins with 't' (css: E[A^='t'])
| `//E[ends-with(@A, "t")]` | element \<E\> whose attribute A ends with 't' (css: E[A$='t'])
| `//E[contains(concat(" ", @A, " "), " w ")` | element \<E\> with attribute A containing word 'w' (css: E[A~='w'])
| `//E[matches(@A, "r")]` | element \<E\> with attribute A matching regex ‘r’
| `//E1[@id=I1] \| //E2[@id=I2]` | element \<E1\> with id I1 or element \<E2\> with id I2 (css: E1#I1, E2#I2)
| `//E1[@id=I1 or @id=I2]` | element \<E1\> with id I1 or id I2 (css: E1#I1, E1#I2)
## 3. Attribute.
| XPath | Description |
| -------------- | ----------- |
| `//E/@A` | attribute A of element \<E\> (css: E@A)
| `//*/@A` | attribute A of any element (css: *@A)
| `//E[@A2="t"]/@A1` | attribute A1 of element \<E\> where attribute A2 is 't' exactly (css: E[A2='t']@A1)
| `//E[contains(@A,"t")]/@A` | attribute A of element \<E\> where A contains 't' (css: E[A*='t']@A)
## 4. ID and Name.
| XPath | Description |
| -------------- | ----------- |
| `//*[@id="I"]` | element with id I (css: #I)
| `//E[@id="I"]` | element \<E\> with id I (css: E#I)
| `//*[@name="N"]` | element with name (css: [name='N'])
| `//E[@name="N"]` | element \<E\> with name (css: E[name='N'])
| `//*[@id="X" or @name="X"]` | element with id X or, failing that, a name X
| `//*[@name="N"][v+1]` | element with name N & specified 0-based index ‘v’ (css: [name='N']:nth-child(v+1))
| `//*[@name="N"][@value="v"]` | element with name N & specified value ‘v’ (css: *[name='N'][value='v’])
## 5. Lang and Class.
| XPath | Description |
| -------------- | ----------- |
| `//E[@lang="L" or starts-with(@lang, concat("L", "-"))]` | element \<E\> is explicitly in language L or subcode (css: E[lang\|=L])
| `//*[contains(concat(" ", @class, " "), " C ")]` | element with a class C (css: .C)
| `//E[contains(concat(" ", @class, " "), " C ")]` | element \<E\> with a class C (css: E.C)
## 6. Text and Link.
| XPath | Description |
| -------------- | ----------- |
| `//*[.="t"]` | element containing text 't' exactly
| `//E[contains(text(), "t")]` | element \<E\> containing text 't' (css: E:contains('t'))
| `//a` | link element (css: a)
| `//a[.="t"]` | element \<a\> containing text 't' exactly
| `//a[contains(text(), "t")]` | element \<a\> containing text 't' (css: a:contains('t'))
| `//a[@href="url"]` | \<a\> with target link 'url' (css: a[href='url'])
| `//a[.="t"]/@href` | link URL labeled with text 't' exactly
## 7. Parent and Child.
| XPath | Description |
| -------------- | ----------- |
| `//E/*[1]` | first child of element \<E\> (css: E > *:first-child)
| `//E[1]` | first \<E\> child (css: E:first-of-type)
| `//E/*[last()]` | last child of element E (css: E *:last-child)
| `//E[last()]` | last \<E\> child (css: E:last-of-type)
| `//E[2]` | second \<E\> child (css: E:nth-of-type(2))
| `//*[2][name()="E"]` | second child that is an \<E\> element (css: E:nth-child(2))
| `//E[last()-1]` | second-to-last \<E\> child (css: E:nth-last-of-type(2))
| `//*[last()-1][name()="E"]` | second-to-last child that is an \<E\> element (css: E:nth-last-child(2))
| `//E1/[E2 and not( *[not(self::E2)])]` | element \<E1\> with only \<E2\> children
| `//E/..` | parent of element \<E\>
| `//*[@id="I"]/.../.../.../E` | descendant \<E\> of element with id I using specific path (css: #I > … > … > … > E)
| `//*[@id="I"]//E` | descendant \<E\> of element with id I using unspecified path (css: #I E)
| `//E[count(*)=0]` | element \<E\> with no children (E:empty)
| `//E[count(*)=1]` | element \<E\> with an only child
| `//E[count(preceding-sibling::*)+count(following-sibling::*)=0]` | element \<E\> that is an only child (css: E:only-child)
| `//E[count(../E) = 1]` | element \<E\> with no \<E\> siblings (css: E:only-of-type)
| `//E[position() mod N = M + 1]` | every Nth element starting with the (M+1)th (css: E:nth-child(Nn+M))
## 8. Sibling.
| XPath | Description |
| -------------- | ----------- |
| `//E2/following-sibling::E1` | element \<E1\> following some sibling \<E2\> (css: E2 ~ E1)
| `//E2/following-sibling::*[1][name()="E1"]` | element \<E1\> immediately following sibling \<E2\> (css: E2 + E1)
| `//E2/following-sibling::*[2][name()="E1"]` | element \<E1\> following sibling \<E2\> with one intermediary (css: E2 + * + E1)
| `//E/following-sibling::*` | sibling element immediately following \<E\> (css: E + *)
| `//E2/preceding-sibling::E1` | element \<E1\> preceding some sibling \<E2\>
| `//E2/preceding-sibling::*[1][name()="E1"]` | element \<E1\> immediately preceding sibling \<E2\>
| `//E2/preceding-sibling::*[2][name()="E1"]` | element \<E1\> preceding sibling \<E2\> with one intermediary
| `//E/preceding-sibling::*[1]` | sibling element immediately preceding \<E\>
## 9. Table Cell.
| XPath | Description |
| -------------- | ----------- |
| `//*[@id="TestTable"]//tr[3]//td[2]` | cell by row and column (e.g. 3rd row, 2nd column) (css: #TestTable tr:nth-child(3) td:nth-child(2))
| `//td[preceding-sibling::td="t"]` | cell immediately following cell containing 't' exactly
| `td[preceding-sibling::td[contains(.,"t")]]` | cell immediately following cell containing 't' (css: td:contains('t') ~ td)
## 10. Dynamic.
| XPath | Description |
| -------------- | ----------- |
| `//E[@disabled]` | user interface element \<E\> that is disabled (css: E:disabled)
| `//*[not(@disabled)]` | user interface element that is enabled (css: E:enabled)
| `//*[@checked]` | checkbox (or radio button) that is checked (css: *:checked)
## 11. XPath Functions.
> https://developer.mozilla.org/en-US/docs/Web/XPath/Functions
### 11.1. Conversion.
| XPath | Description |
| -------------- | ----------- |
| `boolean(expression)` | evaluates an expression and returns true or false.
| `string([object])` | converts the given argument to a string.
| `number([object])` | converts an object to a number and returns the number.
### 11.2. Math.
| XPath | Description |
| -------------- | ----------- |
| `ceiling(number)` | evaluates a decimal number and returns the smallest integer greater than or equal to the decimal number.
| `floor(number)` | evaluates a decimal number and returns the largest integer less than or equal to the decimal number.
| `round(decimal)` | returns a number that is the nearest integer to the given number.
| `sum(node-set)` | returns a number that is the sum of the numeric values of each node in a given node-set.
### 11.3. Logic.
| XPath | Description |
| -------------- | ----------- |
| `true()` | returns a boolean value of true.
| `false()` | returns boolean false.
| `not(expression)` | evaluates a boolean expression and returns the opposite value.
### 11.4. Node.
| XPath | Description |
| -------------- | ----------- |
| `lang(string)` | determines whether the context node matches the given language and returns boolean true or false.
| `name([node-set])` | returns a string representing the QName of the first node in a given node-set.
| `namespace-uri([node-set])` | returns a string representing the namespace URI of the first node in a given node-set.
### 11.5. Context.
| XPath | Description |
| -------------- | ----------- |
| `count(node-set)` | counts the number of nodes in a node-set and returns an integer.
| `function-available(name)` | determines if a given function is available and returns boolean true or false.
| `last()` | returns a number equal to the context size from the expression evaluation context.
| `position()` | returns a number equal to the context position from the expression evaluation context.
### 11.6. String.
| XPath | Description |
| -------------- | ----------- |
| `contains(haystack-string, needle-string)` | determines whether the first argument string contains the second argument string and returns boolean true or false.
| `concat(string1, string2 [stringn]*)` | concatenates two or more strings and returns the resulting string.
| `normalize-space(string)` | strips leading and trailing white-space from a string, replaces sequences of whitespace characters by a single space, and returns the resulting string.
| `starts-with(haystack, needle)` | checks whether the first string starts with the second string and returns true or false.
| `string-length([string])` | returns a number equal to the number of characters in a given string.
| `substring(string, start [length])` | returns a part of a given string.
| `substring-after(haystack, needle)` | returns a string that is the rest of a given string after a given substring.
| `substring-before(haystack, needle)` | returns a string that is the rest of a given string before a given substring.
| `translate(string, abc, XYZ)` | evaluates a string and a set of characters to translate and returns the translated string.
## 12. XPath Axes.
| XPath | Description |
| -------------- | ----------- |
| `ancestor` | indicates all the ancestors of the context node beginning with the parent node and traveling through to the root node.
| `ancestor-or-self` | indicates the context node and all of its ancestors, including the root node.
| `attribute (@)` | indicates the attributes of the context node. Only elements have attributes. This axis can be abbreviated with the at sign (@).
| `child (/)` | indicates the children of the context node. If an XPath expression does not specify an axis, this is understood by default. Since only the root node or element nodes have children, any other use will select nothing.
| `descendant (//)` | indicates all of the children of the context node, and all of their children, and so forth. Attribute and namespace nodes are not included - the parent of an attribute node is an element node, but attribute nodes are not the children of their parents.
| `descendant-or-self` | indicates the context node and all of its descendants. Attribute and namespace nodes are not included - the parent of an attribute node is an element node, but attribute nodes are not the children of their parents.
| `following` | indicates all the nodes that appear after the context node, except any descendant, attribute, and namespace nodes.
| `following-sibling` | indicates all the nodes that have the same parent as the context node and appear after the context node in the source document.
| `parent(..)` | indicates the single node that is the parent of the context node. It can be abbreviated as two periods (..).
| `preceding` | indicates all the nodes that precede the context node in the document except any ancestor, attribute and namespace nodes.
| `preceding-sibling` | indicates all the nodes that have the same parent as the context node and appear before the context node in the source document.
| `self (.)` | indicates the context node itself. It can be abbreviated as a single period (.).
**XPath** navigates an XML or HTML document as a tree. Every step has the form `axis::node-test[predicate]`, and most of it has a short form: `//div[@class="card"]/a/@href` reads "the `href` of every `a` inside any `div` with class `card`".
> ⚠️ Browsers, Selenium and most scrapers implement **XPath 1.0**. `ends-with()`, `matches()`, `lower-case()` and `replace()` are 2.0+ and only work in tools like `xmllint`, Saxon or `lxml` with an XSLT 2.0 processor. Workarounds for 1.0 are given below.
## 🧭 Path Syntax
| XPath | Selects |
|------------|---------------------------------------------|
| `/` | From the document root — an absolute path |
| `//` | Anywhere in the document, at any depth |
| `.` | The current (context) node |
| `..` | The parent of the context node |
| `@attr` | An attribute |
| `*` | Any element |
| `@*` | Any attribute |
| `text()` | The text nodes of an element |
| `node()` | Any node at all, text and comments included |
| `[n]` | The nth match — **1-based**, not 0-based |
| `[last()]` | The last match |
| `a \| b` | The union of two expressions |
```xpath
/html/body (: absolute: the body of the page :)
//text() (: every text node :)
./div/b (: relative to the context node :)
(//p)[2] (: the second p in the whole document :)
//p[2] (: every p that is the second p of its parent :)
```
## 🏷 Elements & Attributes
| XPath | Description | CSS |
|-----------------------------------------------|-------------------------------------------|----------------|
| `//E` | Every `<E>` element | `E` |
| `(//E)[2]` | The second `<E>` in the document | — |
| `//E[@A]` | `<E>` carrying attribute `A` | `E[A]` |
| `//E[@A="t"]` | `<E>` where `A` is exactly `t` | `E[A='t']` |
| `//E[contains(@A,"t")]` | `<E>` where `A` contains `t` | `E[A*='t']` |
| `//E[starts-with(@A,"t")]` | `<E>` where `A` starts with `t` | `E[A^='t']` |
| `//E[substring(@A, string-length(@A)-2)="t"]` | `<E>` where `A` ends with `t` (1.0 trick) | `E[A$='t']` |
| `//E[ends-with(@A,"t")]` | The same, XPath 2.0+ only | `E[A$='t']` |
| `//E[matches(@A,"r")]` | `A` matches regex `r` (2.0+) | — |
| `//E/@A` | The value of attribute `A` | — |
| `//*/@A` | Attribute `A` of any element | — |
| `//E[@A2="t"]/@A1` | `A1` where `A2` is `t` | — |
| `//E1[@id="I1"] \| //E2[@id="I2"]` | Either element | `E1#I1, E2#I2` |
| `//E[@id="I1" or @id="I2"]` | `<E>` with either id | `E#I1, E#I2` |
## 🆔 Id, Name, Class & Language
| XPath | Description | CSS |
|-------------------------------------------------------------------|-----------------------------------------------------------|-------------------------|
| `//*[@id="I"]` | The element with id `I` | `#I` |
| `//E[@id="I"]` | `<E>` with id `I` | `E#I` |
| `//*[@name="N"]` | Any element named `N` | `[name='N']` |
| `//*[@name="N"][@value="v"]` | Named `N` with value `v` | `[name='N'][value='v']` |
| `//*[contains(concat(" ", normalize-space(@class), " "), " C ")]` | Element with class `C` — the padding avoids matching `C2` | `.C` |
| `//E[contains(concat(" ", normalize-space(@class), " "), " C ")]` | `<E>` with class `C` | `E.C` |
| `//E[@lang="L" or starts-with(@lang, concat("L","-"))]` | `<E>` in language `L` or a subcode | `E[lang\|='L']` |
## 📝 Text & Links
| XPath | Description | CSS |
|-----------------------------------|------------------------------------------------------|--------------------|
| `//*[.="t"]` | Element whose whole text is exactly `t` | — |
| `//E[contains(text(),"t")]` | `<E>` with a text node containing `t` | — |
| `//E[contains(.,"t")]` | `<E>` whose text, descendants included, contains `t` | — |
| `//E[normalize-space()="t"]` | Exact text, ignoring surrounding whitespace | — |
| `//a` | Every link | `a` |
| `//a[.="t"]` | The link labelled exactly `t` | — |
| `//a[@href="url"]` | The link pointing at `url` | `a[href='url']` |
| `//a[.="t"]/@href` | The URL behind the label `t` | — |
| `//a[starts-with(@href,"https")]` | Secure links only | `a[href^='https']` |
## 👪 Parents & Children
| XPath | Description | CSS |
|------------------------------------------------------------------|--------------------------------------------------------|-------------------------|
| `//E/*[1]` | First child of `<E>` | `E > *:first-child` |
| `//E/*[last()]` | Last child of `<E>` | `E > *:last-child` |
| `//E[1]` | Each `<E>` that is the first of its type in its parent | `E:first-of-type` |
| `//E[last()]` | Last `<E>` of its type | `E:last-of-type` |
| `//E[2]` | Second `<E>` of its type | `E:nth-of-type(2)` |
| `//*[2][name()="E"]` | Second child, if it is an `<E>` | `E:nth-child(2)` |
| `//E[last()-1]` | Second-to-last `<E>` of its type | `E:nth-last-of-type(2)` |
| `//E/..` | The parent of `<E>` | — |
| `//div/form/parent::*` | Every element that has a form child | `div:has(> form)` |
| `//*[@id="I"]//E` | `<E>` anywhere under id `I` | `#I E` |
| `//E[count(*)=0]` | `<E>` with no element children | `E:empty` |
| `//E[count(*)=1]` | `<E>` with exactly one child | — |
| `//E[count(preceding-sibling::*)+count(following-sibling::*)=0]` | `<E>` that is an only child | `E:only-child` |
| `//E[count(../E)=1]` | `<E>` with no `<E>` siblings | `E:only-of-type` |
| `//E1[E2 and not(*[not(self::E2)])]` | `<E1>` whose only children are `<E2>` | — |
| `//E[position() mod N = M + 1]` | Every Nth element, offset by M | `E:nth-child(Nn+M)` |
## 👥 Siblings
| XPath | Description | CSS |
|---------------------------------------------|--------------------------------------|---------------|
| `//E2/following-sibling::E1` | Every `<E1>` after a sibling `<E2>` | `E2 ~ E1` |
| `//E2/following-sibling::*[1][name()="E1"]` | `<E1>` immediately after `<E2>` | `E2 + E1` |
| `//E2/following-sibling::*[2][name()="E1"]` | `<E1>` two siblings after `<E2>` | `E2 + * + E1` |
| `//E/following-sibling::*[1]` | Whatever follows `<E>` directly | `E + *` |
| `//E2/preceding-sibling::E1` | Every `<E1>` before a sibling `<E2>` | — |
| `//E2/preceding-sibling::*[1][name()="E1"]` | `<E1>` immediately before `<E2>` | — |
| `//E/preceding-sibling::*[1]` | Whatever precedes `<E>` directly | — |
## 📊 Tables & Form State
| XPath | Description | CSS |
|------------------------------------------------|------------------------------------------|--------------------------------------|
| `//*[@id="T"]//tr[3]//td[2]` | Third row, second cell | `#T tr:nth-child(3) td:nth-child(2)` |
| `//td[preceding-sibling::td="t"]` | The cell right after the one reading `t` | — |
| `//td[preceding-sibling::td[contains(.,"t")]]` | Cells following one containing `t` | `td:has(~ td)` |
| `//table[count(tr)=1 and count(tr/td)=2]` | Tables with one row and two columns | — |
| `//table/tr[last()]` | The last row of a table | `table tr:last-child` |
| `//E[@disabled]` | A disabled control | `E:disabled` |
| `//E[not(@disabled)]` | A control that is not disabled | `E:enabled` |
| `//*[@checked]` | A checked checkbox or radio | `:checked` |
## 🧮 Functions
### Strings
| Function | Returns |
|--------------------------------------|----------------------------------------------------------|
| `contains(haystack, needle)` | True when the first string contains the second |
| `starts-with(haystack, needle)` | True when it starts with it |
| `concat(s1, s2, …)` | The strings joined together |
| `normalize-space(s)` | The string with runs of whitespace collapsed and trimmed |
| `string-length(s)` | The number of characters |
| `substring(s, start [, length])` | A slice — **1-based** |
| `substring-before(haystack, needle)` | Everything before the first occurrence |
| `substring-after(haystack, needle)` | Everything after it |
| `translate(s, from, to)` | Characters mapped one by one — the 1.0 way to fold case |
### Numbers, logic & nodes
| Function | Returns |
|---------------------------------------------|-----------------------------------------------------------|
| `count(node-set)` | How many nodes matched |
| `sum(node-set)` | The sum of their numeric values |
| `position()` | The index of the context node within its step |
| `last()` | The size of the context — so `[last()]` is the final node |
| `name([node-set])` | The qualified name of the first node |
| `namespace-uri([node-set])` | Its namespace URI |
| `lang(s)` | True when the node is in that language |
| `not(expr)` | The negation |
| `true()` / `false()` | Boolean constants |
| `boolean(expr)` / `string(o)` / `number(o)` | Type conversions |
| `ceiling(n)` / `floor(n)` / `round(n)` | Rounding |
> 💡 Case-insensitive matching in XPath 1.0:
> `//E[contains(translate(@A,"ABCDEFGHIJKLMNOPQRSTUVWXYZ","abcdefghijklmnopqrstuvwxyz"), "t")]`
## 🧭 Axes
| Axis | Short form | Selects |
|----------------------|-------------|-----------------------------------------------------------------|
| `child` | *(default)* | The direct children of the context node |
| `descendant` | `//` | Every node below the context node |
| `descendant-or-self` | `//` | The context node and everything below it |
| `parent` | `..` | The single parent node |
| `ancestor` | — | Parent, grandparent, up to the root |
| `ancestor-or-self` | — | The same, including the context node |
| `following` | — | Everything after the node in document order, except descendants |
| `following-sibling` | — | Later siblings under the same parent |
| `preceding` | — | Everything before it, except its ancestors |
| `preceding-sibling` | — | Earlier siblings under the same parent |
| `attribute` | `@` | The attributes of the node |
| `self` | `.` | The context node itself |
```xpath
//td[.="Total"]/ancestor::table (: the table a cell belongs to :)
//h2[.="Specs"]/following-sibling::ul[1] (: the list after a heading :)
//input[@name="q"]/ancestor-or-self::form (: the form wrapping a field :)
```
## 🧠 Worked Examples
```xpath
//hr[@class="edge" and position()=1]
(: the first hr of class 'edge' under each parent :)
//table[parent::div[@class="pad"] and not(@id)]//a
(: links in unnamed tables inside a padded div :)
/html/body/div/*[preceding-sibling::h4]
(: whatever comes after an h4 :)
//tr/td[font[@class="head" and text()="TRACK"]]
(: cells whose font child says TRACK :)
//*[count(*)=3]
(: elements with exactly three children :)
//rdf:Seq/rdf:li/em:id
(: namespaced elements — the prefixes must be bound by the host tool :)
//var | //acronym
(: a union of two node sets :)
```
## 🛠 Running XPath
```javascript
// browser devtools console
$x('//a[contains(@href, "docs")]')
// plain DOM API
document.evaluate('//h1', document, null, XPathResult.ANY_TYPE, null).iterateNext()
```
```bash
# from the shell, over XML or HTML
xmllint --xpath '//book/title/text()' catalog.xml
xmllint --html --xpath '//a/@href' page.html 2>/dev/null
```
```python
from lxml import html
tree = html.parse('page.html')
tree.xpath('//div[@class="card"]/a/@href')
```
```python
# Selenium and Playwright both accept XPath locators
driver.find_element(By.XPATH, '//button[normalize-space()="Save"]')
page.locator('xpath=//button[normalize-space()="Save"]').click()
```
## ⚠️ Notes
- **Indexes start at 1.** `[0]` never matches anything.
- **`//E[1]` is per-parent**, `(//E)[1]` is document-wide — the single most common XPath bug.
- **Namespaced documents** need prefixes registered with the host library; in XPath 1.0 a common workaround is `//*[local-name()="title"]`.
- **`//` is expensive** on large documents; anchor the path when you can.
- **Prefer text matching with `normalize-space()`** — HTML is full of stray whitespace and line breaks.
## 📚 Resources
- [MDN — XPath](https://developer.mozilla.org/en-US/docs/Web/XML/XPath)
- [MDN — XPath functions](https://developer.mozilla.org/en-US/docs/Web/XML/XPath/Reference/Functions)
- [XPath 1.0 specification](https://www.w3.org/TR/xpath-10/)
- [XPath cheat sheet (alephzarro)](http://xpath.alephzarro.com/content/cheatsheet.html)
Credit to the original author: https://gist.github.com/jmaccabee/550a0b9fcfdc7e6b170cd34c6ec7bc56
+142 -65
View File
@@ -1,67 +1,135 @@
Zellij is a **terminal workspace and multiplexer** with panes, tabs, layouts, and plugin support.
**Zellij** is a terminal workspace and multiplexer with panes, tabs, layouts, session persistence and WebAssembly plugins. It is modal: a `Ctrl` chord enters a mode, then single keys act inside it.
> ℹ️ Press `Ctrl+p` for **pane** mode, `Ctrl+t` for **tab** mode, `Ctrl+n` for **resize**, `Ctrl+h` for **move**, `Ctrl+s` for **search/scroll**, `Ctrl+o` for **session**. `Esc` or `Ctrl+c` returns to normal mode.
## 📦 Installation
```bash
# Install via Cargo
# from crates.io
cargo install --locked zellij
# Try without installing
# from a package manager
brew install zellij
# try it without installing anything
bash <(curl -L https://zellij.dev/launch)
```
## 🚀 Starting & Managing Sessions
## 🚀 Sessions
```bash
# Start a new Zellij session
# start a session
zellij
# Start with a specific session name
zellij --session mysession
# start a named session
zellij --session <name>
# Attach to an existing session
zellij attach mysession
# attach to a session
zellij attach <name>
# List all sessions
# attach, creating the session if it does not exist
zellij attach --create <name>
# list sessions
zellij list-sessions
# Kill a session
zellij kill-session mysession
# kill a session (it stays resurrectable)
zellij kill-session <name>
# kill every session
zellij kill-all-sessions
# delete a session for good
zellij delete-session <name>
```
## 🪟 Pane Management
## 🪟 Pane Mode — `Ctrl+p`
| Action | Command / Shortcut |
|--------|--------------------|
| Split vertically | `Ctrl + p` → `v` |
| Split horizontally | `Ctrl + p` → `h` |
| Close pane | `Ctrl + p` → `x` |
| Move focus | `Ctrl + p` → Arrow keys |
| Resize pane | `Ctrl + p` → `r` then arrows |
| Toggle floating pane | `Ctrl + p` → `f` |
| Toggle stacked panes | `Ctrl + p` → `s` |
| Key | Action |
|-----------------|-------------------------------------------------|
| `n` | New pane |
| `d` | Split down (new pane below) |
| `r` | Split right (new pane to the right) |
| `x` | Close the focused pane |
| `f` | Toggle fullscreen for the focused pane |
| `w` | Toggle a floating pane |
| `e` | Embed a floating pane, or float an embedded one |
| `c` | Rename the pane |
| `z` | Toggle the pane frames |
| `←↑↓→` / `hjkl` | Move the focus |
## 📑 Tab Management
## 📑 Tab Mode — `Ctrl+t`
| Action | Command / Shortcut |
|--------|--------------------|
| New tab | `Ctrl + p` → `t` |
| Close tab | `Ctrl + p` → `q` |
| Rename tab | `Ctrl + p` → `n` |
| Move between tabs | `Ctrl + p` → Left/Right arrows |
| Key | Action |
|--------------|---------------------------------------------|
| `n` | New tab |
| `x` | Close the tab |
| `r` | Rename the tab |
| `s` | Toggle sync — type into every pane at once |
| `b` | Break the focused pane out into its own tab |
| `[` / `]` | Move the pane to the previous / next tab |
| `←→` / `1…9` | Switch tabs |
## 🛠️ Layouts
## 📐 Resize & Move
| Mode | Keys | Action |
|-------------------|-----------------|---------------------------------|
| Resize — `Ctrl+n` | `←↑↓→` / `hjkl` | Grow the pane in that direction |
| Resize — `Ctrl+n` | `+` / `-` | Grow or shrink the pane |
| Move — `Ctrl+h` | `←↑↓→` / `hjkl` | Move the pane in the layout |
## 🔍 Search & Scrollback — `Ctrl+s`
| Key | Action |
|-----------------|---------------------------------------|
| `s` | Search the scrollback |
| `e` | Open the scrollback in your `$EDITOR` |
| `PgUp` / `PgDn` | Scroll a page at a time |
| `n` / `p` | Next / previous search hit |
| `c` | Clear the search |
## 🗄 Session Mode — `Ctrl+o`
| Key | Action |
|-----|---------------------------------------------|
| `d` | Detach — the session keeps running |
| `w` | Session manager (switch, resurrect, delete) |
| `p` | Plugin manager |
| `c` | Configuration |
> 💡 `Ctrl+g` locks the interface so every key goes straight to the program in the pane (useful when the app wants `Ctrl+p`). Press `Ctrl+g` again to unlock. `Ctrl+q` quits Zellij.
## ⚡ Shortcuts Without a Mode
| Keys | Action |
|-------------------------|---------------------------|
| `Alt+n` | New pane |
| `Alt+←↑↓→` / `Alt+hjkl` | Move the focus |
| `Alt+=` / `Alt+-` | Resize the focused pane |
| `Alt+[` / `Alt+]` | Cycle through the layouts |
| `Alt+f` | Toggle a floating pane |
| `Alt+i` / `Alt+o` | Move the tab left / right |
## 🛠 Layouts
Layouts are KDL files that describe the panes and tabs a session starts with.
```bash
# Start with a predefined layout
zellij --layout path/to/layout.kdl
# start with a layout
zellij --layout <path>/layout.kdl
# Example layout file (KDL format)
# dump the built-in default to start from
zellij setup --dump-layout default > layout.kdl
```
```kdl
layout {
tab {
tab name="editor" {
pane split_direction="vertical" {
pane
pane
pane command="cargo" {
args "watch" "-x" "test"
}
}
}
}
@@ -69,38 +137,47 @@ layout {
## ⚙️ Configuration
- Config file: `~/.config/zellij/config.kdl`
- Common options:
```kdl
keybinds {
normal {
bind "Ctrl g" { SwitchToMode "locked"; }
}
}
```
## 🔌 Plugins
- Zellij supports **WebAssembly plugins**.
- Example usage:
```bash
zellij --plugin path/to/plugin.wasm
```
## 🧭 Useful Flags
The config file is `~/.config/zellij/config.kdl`.
```bash
--session <name> # Name the session
--layout <file> # Use a layout file
--help # Show help
--version # Show version
# write the default configuration out to edit it
zellij setup --dump-config > ~/.config/zellij/config.kdl
# check where zellij looks for its files
zellij setup --check
```
## 🗂️ Quick Reference
```kdl
theme "gruvbox-dark"
default_shell "fish"
pane_frames false
- **Session** → `zellij`, `attach`, `list-sessions`, `kill-session`
- **Pane** → split, move, resize, float, stack
- **Tab** → new, close, rename, switch
- **Layout** → `--layout file.kdl`
- **Config** → `~/.config/zellij/config.kdl`
- **Plugins** → `.wasm` modules
keybinds {
normal {
bind "Ctrl g" { SwitchToMode "locked"; }
}
}
```
## 🔌 Running Commands & Plugins
```bash
# open a new pane running a command
zellij run -- htop
# open a file in a new pane, in your editor
zellij edit <file>
# drive a running session from a script
zellij action new-tab --name deploy
zellij action write-chars "make deploy"
# load a WebAssembly plugin
zellij --plugin <path>/plugin.wasm
```
## 📚 Resources
- [Documentation](https://zellij.dev/documentation/)
- [Layouts reference](https://zellij.dev/documentation/layouts)
- [Configuration reference](https://zellij.dev/documentation/configuration)
+199 -47
View File
@@ -1,72 +1,224 @@
## 📦 ZFS Pool Basics
**OpenZFS** combines the volume manager and the filesystem. A **pool** (`zpool`) is built from **vdevs**, and vdevs are built from disks; **datasets** (`zfs`) are the filesystems and volumes that live inside a pool.
> ⚠️ Redundancy lives at the **vdev** level, not the pool level. Lose one vdev and you lose the whole pool, however healthy the others are.
## 📦 Pool Basics
```bash
# Create a new pool
zpool create mypool /dev/sdX
# create a pool from a single disk (no redundancy)
zpool create mypool /dev/disk/by-id/<disk>
# List all pools
# a two-disk mirror, 4K-aligned
zpool create -o ashift=12 mypool mirror /dev/disk/by-id/<disk1> /dev/disk/by-id/<disk2>
# raidz2 across six disks
zpool create -o ashift=12 tank raidz2 <disk1> <disk2> <disk3> <disk4> <disk5> <disk6>
# mount the pool somewhere other than /<poolname>
zpool create -m /srv/data mypool <disk>
# add another vdev (this widens the pool, it does not add redundancy)
zpool add mypool mirror <disk3> <disk4>
# add a hot spare, a read cache (L2ARC) and a separate intent log
zpool add mypool spare <disk>
zpool add mypool cache <ssd>
zpool add mypool log mirror <ssd1> <ssd2>
# destroy a pool — irreversible
zpool destroy mypool
```
## 🔍 Inspecting
```bash
# pools, their size and health
zpool list
# Show detailed pool status
# per-vdev breakdown
zpool list -v
# layout, errors and any running scrub or resilver
zpool status mypool
# Add a new vdev to an existing pool
zpool add mypool /dev/sdY
# only the pools that need attention
zpool status -x
# Destroy a pool (⚠️ irreversible!)
zpool destroy mypool
# live throughput, refreshed every 5 seconds
zpool iostat -v 5
# Export (unmount) a pool
zpool export mypool
# every property of the pool
zpool get all mypool
# Import (mount) a pool
zpool import mypool
# Scrub a pool (check for errors)
zpool scrub mypool
# Check scrub status
zpool status mypool
# what has been done to this pool, ever
zpool history mypool
```
## 🛠️ Vdev Types & RAID Equivalents
## 🛠 Vdev Types
| ZFS Vdev Type | Description | RAID Equivalent | Min Disks | Fault Tolerance |
|-----------------------|--------------------------------------------------|----------------------|-----------|------------------|
| `stripe` | Data striped across disks, no redundancy | RAID 0 | 1 | 0 |
| `mirror` | Identical copies on multiple disks | RAID 1 | 2+ | 1 per mirror |
| `raidz1` | Single parity | RAID 5 | 3+ | 1 |
| `raidz2` | Double parity | RAID 6 | 4+ | 2 |
| `raidz3` | Triple parity | RAID 7 (uncommon) | 5+ | 3 |
| Vdev type | Description | RAID equivalent | Min. disks | Survives |
|-----------|-------------------------------------|---------------------|------------|--------------------------------|
| `stripe` | Data spread across disks, no parity | RAID 0 | 1 | nothing |
| `mirror` | A full copy on every disk | RAID 1 | 2 | all but one disk |
| `raidz1` | Single parity | RAID 5 | 3 | 1 disk |
| `raidz2` | Double parity | RAID 6 | 4 | 2 disks |
| `raidz3` | Triple parity | — (no common level) | 5 | 3 disks |
| `dRAID` | Distributed parity with spare space | — | 6+ | as configured, rebuilds faster |
> ⚠️ ZFS pools are built from one or more vdevs. If any vdev fails, the entire pool fails—even if other vdevs are healthy.
Support vdevs: `cache` (L2ARC read cache), `log` (SLOG, for synchronous writes), `special` (metadata and small blocks), `spare` (hot spare).
## 📁 Dataset & Filesystem Management
## 🩺 Health & Maintenance
```bash
# Create a dataset
zfs create mypool/mydataset
# verify every block against its checksum
zpool scrub mypool
# List datasets
zfs list
# stop or pause a running scrub
zpool scrub -s mypool
zpool scrub -p mypool
# Set quota on a dataset
zfs set quota=10G mypool/mydataset
# forget errors that have been dealt with
zpool clear mypool
# Snapshot a dataset
zfs snapshot mypool/mydataset@snap1
# swap a failing disk for a new one
zpool replace mypool <old-disk> <new-disk>
# Roll back to a snapshot
zfs rollback mypool/mydataset@snap1
# turn a single disk into a mirror, or break one up
zpool attach mypool <existing-disk> <new-disk>
zpool detach mypool <disk>
# Send snapshot to another system
zfs send mypool/mydataset@snap1 | ssh user@host zfs receive backup/mirror
# take a disk out of service and bring it back
zpool offline mypool <disk>
zpool online mypool <disk>
# tell the SSDs which blocks are free
zpool trim mypool
# enable the on-disk features of a newer OpenZFS release
zpool upgrade mypool
```
## 🧮 Tips & Best Practices
> 💡 Scrub monthly on consumer disks, weekly if the data matters. `zpool status` reports what the last scrub found.
- ✅ Always use whole disks (not partitions) for ZFS.
- ✅ Use ECC RAM if possible—ZFS relies heavily on memory integrity.
- ✅ Avoid mixing vdev types in a single pool.
- ✅ Use `zpool scrub` regularly to detect and fix silent corruption.
- ✅ Remember: redundancy is at the vdev level, not the pool level.
## 📤 Import & Export
```bash
# release a pool so it can be moved to another machine
zpool export mypool
# list the pools that could be imported
zpool import
# import by name, preferring the stable device names
zpool import -d /dev/disk/by-id mypool
# import every pool it can find
zpool import -a
# import a pool that was not exported cleanly
zpool import -f mypool
# import under a different name, or under an alternate root
zpool import mypool newname
zpool import -R /mnt mypool
```
## ⚙️ Pool Properties
```bash
# trim automatically as blocks are freed
zpool set autotrim=on mypool
# grow the pool when the underlying disks grow
zpool set autoexpand=on mypool
# replace a failed disk with a spare automatically
zpool set autoreplace=on mypool
# what to do when the pool faults: wait | continue | panic
zpool set failmode=continue mypool
# read one property
zpool get health,capacity,fragmentation mypool
```
## 📁 Datasets & Filesystems
```bash
# create a dataset
zfs create mypool/data
# create a nested dataset, making the parents as needed
zfs create -p mypool/data/projects/web
# create a block device (zvol) for a VM or iSCSI target
zfs create -V 50G mypool/vm-disk
# list datasets, snapshots and volumes
zfs list
zfs list -t all -r mypool
# common properties
zfs set compression=lz4 mypool/data
zfs set quota=100G mypool/data
zfs set reservation=10G mypool/data
zfs set atime=off mypool/data
zfs set recordsize=1M mypool/media
# read properties, including where they were inherited from
zfs get -r compression mypool
zfs get all mypool/data
# mount, unmount, and destroy
zfs mount mypool/data
zfs unmount mypool/data
zfs destroy -r mypool/data
```
## 📸 Snapshots & Replication
```bash
# take a snapshot (instant, and free until the data diverges)
zfs snapshot mypool/data@2026-08-22
# snapshot a dataset and everything under it
zfs snapshot -r mypool/data@nightly
# list snapshots and what they cost
zfs list -t snapshot -o name,used,refer
# go back in time — discards everything written since
zfs rollback mypool/data@2026-08-22
# read a single file out of a snapshot instead
ls /mypool/data/.zfs/snapshot/2026-08-22/
# a writable copy of a snapshot
zfs clone mypool/data@2026-08-22 mypool/data-copy
# send a full snapshot to another machine
zfs send mypool/data@snap1 | ssh user@host zfs receive backup/data
# send only what changed since the previous snapshot
zfs send -i mypool/data@snap1 mypool/data@snap2 | ssh user@host zfs receive backup/data
# delete a snapshot
zfs destroy mypool/data@snap1
```
## 🧠 Tips & Best Practices
- Address disks by `/dev/disk/by-id/...`, never `/dev/sdX` — kernel names move between boots.
- Set `ashift=12` at creation time for any modern drive; it cannot be changed afterwards.
- Give ZFS whole disks, not partitions, so it can manage the write cache itself.
- Keep pools below ~80% full — performance falls off a cliff above that, and fragmentation is permanent.
- Use ECC RAM where you can: ZFS trusts what is in memory.
- Do not mix vdev types or widths in one pool; the pool is only as good as its weakest vdev.
- A snapshot is not a backup — replicate it somewhere else with `zfs send`.
## 📚 Resources
- [OpenZFS documentation](https://openzfs.github.io/openzfs-docs/)
- [`zpool` manual page](https://openzfs.github.io/openzfs-docs/man/master/8/zpool.8.html)
- [`zfs` manual page](https://openzfs.github.io/openzfs-docs/man/master/8/zfs.8.html)
- [Workload tuning guide](https://openzfs.github.io/openzfs-docs/Performance%20and%20Tuning/Workload%20Tuning.html)