mirror of
https://github.com/sharevb/it-tools.git
synced 2026-11-20 21:23:08 +03:00
Restructure and enhance cheat sheets for improved readability (#566)
* docs(cheatsheets): expand and align the chmod, git, docker and swarm memos Unify the four cheat sheets on one layout: short intro, emoji section headings, grouped command blocks with aligned inline comments, reference tables and a resources section. Placeholders are consistently <angled>. - chmod: ls -l breakdown, octal and symbolic tables, special bits, umask, recursive/selective changes, recipes, related commands and gotchas; fix the directory access matrix (r alone does not grant file access) and the --preserve-root default - git: config and aliases, staging, commit, branching, merge/rebase with conflict flow, remotes, stashing, inspection, cherry-pick, tags, undo with a reset-mode table, cleaning, bisect, submodules and worktrees - docker: contexts and events, build flags, registries, run/exec/inspect, volumes and mounts, networks, the full compose workflow, cleanup and one-liners - swarm: required firewall ports, node and manager operations, live service updates, job modes, placement expressions, publish modes, health checks, secret and config rotation, task states and a troubleshooting table * docs(cheatsheets): put descriptions above commands so each line copies clean Trailing inline comments meant selecting a single command also picked up the padding and its description. Move every description onto its own line above the command it documents, keeping one code block per topic. - each command now stands alone on its line: triple-click copies exactly the command, nothing else - no comment column to re-pad when a longer command is added later, so adding a line no longer reindents the whole block - code lines drop from a median of 78 columns to ~22 (p90 42), so blocks no longer scroll sideways on narrow screens - wrap the three remaining long commands over several lines - reword the descriptions that only made sense trailing a command * docs(cheatsheets): rework the tmux, screen, zellij, nano, jq, zpool, CSS and JSONPath memos Bring eight more cheat sheets onto the same layout as the Docker and Git ones: short intro, emoji section headings, one code block per topic with the description above each command, reference tables, resources at the end. - tmux: session/window/pane/copy-mode tables, scripting with send-keys and capture-pane, a starting .tmux.conf - screen: shared and detached attach modes, split regions, copy mode, logging and monitoring, nested-screen prefix - zellij: correct the pane-mode bindings (d/r split, not v/h), add tab, resize, move, search and session modes, layouts, zellij run/edit/action - nano: add the command-line options and .nanorc settings the sheet was missing, drop the stray H1 and broken headings - jq: flags, core syntax, object/array/string tables, conditionals, variables, reduce, and runnable recipes; drop the stale AWS AMI note and the HTML pipe entities - zpool: vdev and support-vdev types, import/export, properties, dataset and snapshot/replication commands, tuning notes - CSS selectors: split one 60-row table into basic, combinator, attribute, structural, state, form, functional and pseudo-element groups; add :is()/:where()/:has() and a specificity table - JSONPath: RFC 9535 functions, worked example document, filter operators * docs(cheatsheets): rework the Dockerfile, Compose and kubectl memos - Dockerfile: instruction reference table, BuildKit cache/secret/bind mounts and heredocs, COPY --chown/--chmod/--from, HEALTHCHECK, STOPSIGNAL, a CMD vs ENTRYPOINT table, build commands and a worked multi-stage example - Compose: drop the obsolete version: framing, merge the two duplicated "Compose vs Swarm" sections into one table, correct the claim that file-based secrets are Swarm-only, and add depends_on conditions, profiles, watch mode, include/extends/anchors, env interpolation and the full CLI section - kubectl: replace the removed rolling-update and the retired Heapster section, add contexts, custom-columns and JSONPath output, rollouts, debug containers, port-forward, secrets, labels, node maintenance and one-liners; tag the code blocks so they highlight * docs(cheatsheets): rework the sed, regex and common-regex memos - sed: how the pattern space works, an addressing table, the full command reference including hold-space and branching, multi-line recipes, and a warning about BSD/macOS -i; merge the duplicated deletion and addressing sections - regex: add flags, lazy quantifiers, named groups and backreferences, Unicode property escapes, replacement tokens, the JavaScript methods that expose groups, and a gotchas section (stateful /g, catastrophic backtracking, escaping user input) - common-regex: rewrite the patterns that markdown escaping had mangled into unusable text, group them by subject, and add UUID, semver, CIDR, MAC, ISO 8601, duration, log-line and CSV patterns Every pattern in common-regex now compiles, and the ones people copy most (IPv4, UUID, semver, E.164, ISO dates, password rules, slug, duration, MAC) were checked against matching and non-matching input. * docs(cheatsheets): rework the XPath, conventional-commits and RJ45 memos - XPath: give every table a CSS-equivalent column, fix the broken "only children of type" expression, flag which functions are 2.0+ and give the 1.0 workarounds for ends-with and case folding, add the axes short forms, worked examples, and how to run XPath from devtools, xmllint, lxml, Selenium and Playwright - conventional commits: types table with the release effect each one has, description do/don't table, footer reference, both ways to mark a breaking change, a version-bump table, and husky v9 hook syntax - RJ45: keep the colour swatches and the pinout image, add what each pin carries on 10/100 vs Gigabit, the PoE standards table (correcting the claim that the blue pair is the PoE pair), gigabit crossover, Cat8, length and solid-vs-stranded rules, and a fault-finding table * docs(cheatsheets): rework the nginx and PowerShell memos - nginx: the whole sheet was a single 218-line code block; split it into sections with a service/CLI block, a file-layout table, location matching priority, root vs alias, reverse proxy and WebSocket headers, load-balancing methods, TLS, rate limiting, caching and gzip, a variable reference and a 502/504 troubleshooting table - PowerShell: add discovery (Get-Help/Get-Command/Get-Member), functions and param blocks, try/catch and error preferences, Invoke-RestMethod and JSON/CSV conversion, services, jobs, remoting, modules and execution policy, PowerShell 7 operators, an alias table, and repair the mangled escape-character rows * docs(cheatsheets): rework the vim and bash memos Both sheets were a single unbroken code block with ASCII banners, so the whole page rendered as one <pre> with nothing to scan. Rebuild them as sections with key/action tables and runnable blocks. - vim: explain the count + operator + motion grammar, add the text-object table (ciw, ci", da(), registers and why "0p exists, the :g and :v global commands, folds, completion, diff mode, undo by time, a :set reference and a starter vimrc - bash: parameter expansion, arrays including associative ones, the test operator tables, redirection and process substitution, job control and traps, set -euo pipefail with its caveat, shellcheck, and an ANSI colour table in place of sixty lines of colour variables The vim sheet dropped roughly a hundred lines of one person's leader-key mappings and plugin bindings (NERDTree, Taglist, sparkup, PHP DOC), which only worked with their dotfiles; a note now explains that leader mappings come from your own vimrc.
This commit is contained in:
1 parent
a5607e04ad
commit
5bf94bc567
25 files changed
+6206
-4556
No files matched your search
File diff suppressed because it is too large.
Load diff
+197
-71
@@ -1,119 +1,245 @@
|
||||
The `chmod` command changes file and directory permissions in Unix/Linux.
|
||||
**`chmod`** (*change mode*) sets who may read, write, and execute a file or directory on Unix-like systems. The mode can be written **symbolically** (`u+x`) or **numerically** (`755`) — both reach the same result.
|
||||
|
||||
## 🧑 User Classes
|
||||
```bash
|
||||
chmod [options] <mode> <file>...
|
||||
```
|
||||
|
||||
| Symbol | Meaning |
|
||||
|--------|----------------|
|
||||
| `u` | User (owner) |
|
||||
| `g` | Group |
|
||||
| `o` | Others |
|
||||
| `a` | All (u+g+o) |
|
||||
> 💡 Inspect before and after with `ls -l <file>`, or get the octal value directly with `stat -c '%a %n' <file>`.
|
||||
|
||||
## 🔐 Permission Types
|
||||
## 🗂 Reading `ls -l` Output
|
||||
|
||||
| Symbol | Meaning | Numeric |
|
||||
|--------|----------|---------|
|
||||
| `r` | Read | 4 |
|
||||
| `w` | Write | 2 |
|
||||
| `x` | Execute | 1 |
|
||||
```text
|
||||
- rwx r-x r-- → 754
|
||||
│ │ │ │
|
||||
│ │ │ └── others : r-- = 4
|
||||
│ │ └─────── group : r-x = 5
|
||||
│ └──────────── user : rwx = 7
|
||||
└─────────────── type : - file · d directory · l symlink · c/b device
|
||||
```
|
||||
|
||||
## 🧑 User Classes (*who*)
|
||||
|
||||
| Symbol | Class | Applies to |
|
||||
|--------|--------|-------------------------------------------------|
|
||||
| `u` | User | The file's owner |
|
||||
| `g` | Group | Members of the file's group |
|
||||
| `o` | Others | Everyone else |
|
||||
| `a` | All | `u`, `g` and `o` at once (implied when omitted) |
|
||||
|
||||
## 🔐 Permission Bits (*what*)
|
||||
|
||||
| Symbol | Permission | Octal | On a file | On a directory |
|
||||
|--------|------------|-------|---------------------|-------------------------------------------|
|
||||
| `r` | Read | `4` | Read the contents | List entry names (`ls`) |
|
||||
| `w` | Write | `2` | Modify the contents | Create, rename and delete entries |
|
||||
| `x` | Execute | `1` | Run as a program | Enter it (`cd`) and reach entries by name |
|
||||
| `-` | None | `0` | — | — |
|
||||
|
||||
## 📝 Symbolic Mode
|
||||
|
||||
Use `+`, `-`, or `=` to modify permissions:
|
||||
| Operator | Effect |
|
||||
|----------|-------------------------------------------------------------------|
|
||||
| `+` | Add the listed permissions, leave the rest untouched |
|
||||
| `-` | Remove the listed permissions, leave the rest untouched |
|
||||
| `=` | Set exactly these permissions, clearing the others for that class |
|
||||
|
||||
```bash
|
||||
chmod u+x file # Add execute to user
|
||||
chmod go-w file # Remove write from group and others
|
||||
chmod a=r file # Set read-only for all
|
||||
# add execute for the owner
|
||||
chmod u+x <file>
|
||||
|
||||
# remove write from the group
|
||||
chmod g-w <file>
|
||||
|
||||
# cut off group and others entirely
|
||||
chmod go-rwx <file>
|
||||
|
||||
# everyone gets read only, nothing else
|
||||
chmod a=r <file>
|
||||
|
||||
# several clauses, comma separated
|
||||
chmod u=rwx,go=rx <file>
|
||||
|
||||
# 'a' is implied, then filtered by the umask
|
||||
chmod +x <file>
|
||||
|
||||
# regular and special bits in one call
|
||||
chmod u+x,g+s <file>
|
||||
|
||||
# copy the mode of another file
|
||||
chmod --reference=<ref> <file>
|
||||
```
|
||||
|
||||
> 💡 Use `X` (capital) instead of `x` to add execute **only** to directories and to files that already have an execute bit — the safe way to fix a tree: `chmod -R u=rwX,go=rX <dir>`.
|
||||
|
||||
## 🔢 Numeric (Octal) Mode
|
||||
|
||||
Combine values for user, group, and others:
|
||||
Each digit is the sum of `r` (4), `w` (2) and `x` (1):
|
||||
|
||||
| Mode | Permissions | Meaning |
|
||||
|------|-------------|---------------|
|
||||
| 777 | rwxrwxrwx | Full access |
|
||||
| 755 | rwxr-xr-x | Common for scripts |
|
||||
| 644 | rw-r--r-- | Common for text files |
|
||||
| Octal | Sum | Bits |
|
||||
|-------|-------|-------|
|
||||
| `7` | 4+2+1 | `rwx` |
|
||||
| `6` | 4+2 | `rw-` |
|
||||
| `5` | 4+1 | `r-x` |
|
||||
| `4` | 4 | `r--` |
|
||||
| `3` | 2+1 | `-wx` |
|
||||
| `2` | 2 | `-w-` |
|
||||
| `1` | 1 | `--x` |
|
||||
| `0` | 0 | `---` |
|
||||
|
||||
Three digits map to **user, group, others**. An optional fourth *leading* digit sets the special bits (`chmod 4755`, see below).
|
||||
|
||||
### Common modes
|
||||
|
||||
| Mode | Symbolic | Typical use |
|
||||
|-------|-------------|-------------------------------------------------------------|
|
||||
| `777` | `rwxrwxrwx` | Full access for everyone — ⚠️ almost never the right answer |
|
||||
| `755` | `rwxr-xr-x` | Scripts, binaries and directories |
|
||||
| `750` | `rwxr-x---` | Executable for the owner and their group only |
|
||||
| `700` | `rwx------` | Private directory (`~/.ssh`, `~/.gnupg`) |
|
||||
| `664` | `rw-rw-r--` | Files shared read-write inside a group |
|
||||
| `644` | `rw-r--r--` | Regular files, web assets, configs |
|
||||
| `640` | `rw-r-----` | Config with secrets, readable by a service group |
|
||||
| `600` | `rw-------` | Private files (`~/.ssh/id_ed25519`, `.env`) |
|
||||
| `400` | `r--------` | Read-only key material |
|
||||
|
||||
```bash
|
||||
chmod 755 script.sh
|
||||
|
||||
chmod 644 notes.txt
|
||||
|
||||
chmod 600 ~/.ssh/id_ed25519
|
||||
```
|
||||
|
||||
## 🧪 Special Permission Flags
|
||||
## 🧪 Special Permission Bits
|
||||
|
||||
These flags go beyond basic read/write/execute and control execution context and deletion behavior.
|
||||
These go beyond read/write/execute and control the execution context and deletion rules.
|
||||
|
||||
### 🔸 `setuid` (Set User ID on Execution)
|
||||
|
||||
- **Octal prefix**: `4`
|
||||
- **Applies to**: Executable files
|
||||
- **Effect**: When executed, the process runs with the file owner's privileges (not the user's).
|
||||
- **Use case**: Programs like `passwd` need elevated privileges to modify system files.
|
||||
| Bit | Octal | Symbolic | Applies to | Effect | Shown in `ls -l` as |
|
||||
|----------|-------|----------|--------------------------|----------------------------------------------------------------------|-------------------------|
|
||||
| `setuid` | `4` | `u+s` | Executable files | Runs with the **file owner's** privileges | `s` in the user field |
|
||||
| `setgid` | `2` | `g+s` | Executables, directories | Runs as the file's group / new entries inherit the directory's group | `s` in the group field |
|
||||
| `sticky` | `1` | `+t` | Directories | Only the entry's owner (or root) may delete or rename it | `t` in the others field |
|
||||
|
||||
```bash
|
||||
# setuid → -rwsr-xr-x
|
||||
chmod 4755 /usr/bin/somebinary
|
||||
# -rwsr-xr-x → 's' replaces 'x' in user field
|
||||
|
||||
# setgid → drwxrwsr-x: one group for the whole tree
|
||||
chmod 2775 /srv/shared
|
||||
|
||||
# sticky → drwxrwxrwt: a shared drop box, like /tmp
|
||||
chmod 1777 /srv/tmp
|
||||
|
||||
# strip every special bit again
|
||||
chmod u-s,g-s,o-t <file>
|
||||
```
|
||||
|
||||
> ⚠️ Security risk if misused — avoid setting on scripts or user-controlled binaries.
|
||||
> ⚠️ `setuid` is a privilege-escalation vector — never set it on scripts or on binaries a user can replace. Linux ignores it on interpreted scripts anyway.
|
||||
>
|
||||
> ℹ️ A capital `S` or `T` in `ls -l` (`-rwSr--r--`) means the special bit is set while the matching `x` bit is **not** — usually a mistake.
|
||||
|
||||
### 🔸 `setgid` (Set Group ID)
|
||||
## 📂 Directory Access Matrix
|
||||
|
||||
- **Octal prefix**: `2`
|
||||
- **Applies to**:
|
||||
- **Files**: Runs with the file group’s privileges.
|
||||
- **Directories**: New files inherit the directory’s group.
|
||||
- **Use case**: Shared project folders where all files should belong to the same group.
|
||||
`x` on a directory means *traverse*, not *execute* — without it nothing below the directory can be reached, no matter how permissive the files themselves are.
|
||||
|
||||
| Mode | List names (`ls`) | Details (`ls -l`, `stat`) | Enter (`cd`) | Open a known file | Create / delete entries |
|
||||
|-------|-------------------|---------------------------|--------------|-------------------|-------------------------|
|
||||
| `r--` | ✅ | ❌ | ❌ | ❌ | ❌ |
|
||||
| `--x` | ❌ | ✅ (by exact name) | ✅ | ✅ | ❌ |
|
||||
| `r-x` | ✅ | ✅ | ✅ | ✅ | ❌ |
|
||||
| `rwx` | ✅ | ✅ | ✅ | ✅ | ✅ |
|
||||
| `-w-` | ❌ | ❌ | ❌ | ❌ | ❌ (write needs `x`) |
|
||||
|
||||
## 🔁 Recursive & Selective Changes
|
||||
|
||||
```bash
|
||||
chmod 2755 /shared/folder
|
||||
# drwxr-sr-x → 's' in group field
|
||||
# blunt: also marks every file executable
|
||||
chmod -R 755 <dir>
|
||||
|
||||
# preferred: X only touches dirs + real executables
|
||||
chmod -R u=rwX,go=rX <dir>
|
||||
|
||||
# directories only
|
||||
find <dir> -type d -exec chmod 755 {} +
|
||||
|
||||
# files only
|
||||
find <dir> -type f -exec chmod 644 {} +
|
||||
|
||||
# make every shell script executable
|
||||
find <dir> -type f -name '*.sh' -exec chmod +x {} +
|
||||
```
|
||||
|
||||
> 📁 On directories, this ensures group consistency for collaborative environments.
|
||||
## ⚙️ Useful Options
|
||||
|
||||
### 🔸 `sticky` Bit
|
||||
| Option | Description |
|
||||
|-----------------------------|-----------------------------------------------------------------|
|
||||
| `-R`, `--recursive` | Apply to directories and everything inside them |
|
||||
| `-v`, `--verbose` | Print a line for every file processed |
|
||||
| `-c`, `--changes` | Print a line only when a mode actually changed |
|
||||
| `-f`, `--silent`, `--quiet` | Suppress most error messages |
|
||||
| `--reference=<file>` | Reuse the mode of another file instead of spelling it out |
|
||||
| `--preserve-root` | Refuse to run recursively on `/` — not the default, unlike `rm` |
|
||||
| `-h` (BSD/macOS) | Act on the symlink itself instead of its target |
|
||||
|
||||
- **Octal prefix**: `1`
|
||||
- **Applies to**: Directories only
|
||||
- **Effect**: Only the file owner (or root) can delete or rename files inside.
|
||||
- **Use case**: `/tmp` directory — prevents users from deleting each other’s files.
|
||||
## 🎭 `umask` — Default Permissions for New Files
|
||||
|
||||
New files start from `666` and new directories from `777`; the umask bits are removed from that base.
|
||||
|
||||
```bash
|
||||
chmod 1777 /tmp
|
||||
# drwxrwxrwt → 't' in others field
|
||||
# show the current mask, e.g. 0022
|
||||
umask
|
||||
|
||||
# show it symbolically, e.g. u=rwx,g=rx,o=rx
|
||||
umask -S
|
||||
|
||||
# stricter default for this shell session
|
||||
umask 027
|
||||
```
|
||||
|
||||
> 🧷 Think of it as a “delete lock” for shared spaces.
|
||||
| umask | New files | New directories |
|
||||
|-------|-----------|-----------------|
|
||||
| `022` | `644` | `755` |
|
||||
| `002` | `664` | `775` |
|
||||
| `027` | `640` | `750` |
|
||||
| `077` | `600` | `700` |
|
||||
|
||||
## 📂 Execute (`x`) Flag: Files vs Directories
|
||||
> 💡 Put `umask 027` in `~/.bashrc` (or a systemd unit's `UMask=`) to make the default stick.
|
||||
|
||||
| Context | Effect of `x` Permission |
|
||||
|-------------|----------------------------------------|
|
||||
| **File** | Allows execution as a program/script |
|
||||
| **Directory** | Allows entering the directory (`cd`) and accessing contents by name |
|
||||
## 🚑 Common Recipes
|
||||
|
||||
### 🔍 Directory Access Matrix
|
||||
| Goal | Command |
|
||||
|-------------------------------------|------------------------------------------------------------------------------|
|
||||
| Make a script runnable | `chmod +x deploy.sh` |
|
||||
| Lock down an SSH key pair | `chmod 700 ~/.ssh && chmod 600 ~/.ssh/id_* && chmod 644 ~/.ssh/*.pub` |
|
||||
| Serve web content read-only | `chmod -R u=rwX,go=rX /var/www/site` |
|
||||
| Shared team directory (one group) | `chmod 2775 /srv/shared` |
|
||||
| Public scratch directory | `chmod 1777 /srv/tmp` |
|
||||
| Clone the mode of a known-good file | `chmod --reference=good.conf new.conf` |
|
||||
| Undo an accidental `chmod -R 777 .` | `find . -type d -exec chmod 755 {} + && find . -type f -exec chmod 644 {} +` |
|
||||
|
||||
| Permission | Can List (`ls`) | Can Enter (`cd`) | Can Access Files |
|
||||
|------------|------------------|------------------|------------------|
|
||||
| `r` | ✅ | ❌ | ✅ (if name known) |
|
||||
| `x` | ❌ | ✅ | ✅ (if name known) |
|
||||
| `r+x` | ✅ | ✅ | ✅ |
|
||||
## 🧰 Related Commands
|
||||
|
||||
> 🧠 Without `x` on a directory, you can't `cd` into it — even if you can list its contents.
|
||||
| Command | Purpose |
|
||||
|-----------------------------------|--------------------------------------------------------------------|
|
||||
| `chown <user>:<group> <file>` | Change the owner and/or group |
|
||||
| `chgrp <group> <file>` | Change the group only |
|
||||
| `ls -l` / `ls -ld <dir>` | Show permissions (`-d` for the directory itself, not its contents) |
|
||||
| `stat -c '%a %A %U:%G %n' <file>` | Octal mode, symbolic mode, owner and group in one line |
|
||||
| `namei -l <path>` | Show the permissions of every component of a path |
|
||||
| `getfacl` / `setfacl` | Fine-grained ACLs when `u/g/o` is not expressive enough |
|
||||
| `id` / `groups` | Check which groups you actually belong to |
|
||||
|
||||
## 🔍 Quick Reference
|
||||
## ⚠️ Gotchas
|
||||
|
||||
| Command | Description |
|
||||
|---------------------|--------------------------------------|
|
||||
| `chmod +x file` | Make file executable |
|
||||
| `chmod -R 755 dir` | Recursively set permissions |
|
||||
| `chmod u=rwx file` | Set user to rwx only |
|
||||
| `chmod a-x file` | Remove execute from all |
|
||||
- **`chmod 777` is not a fix.** It grants every local user write access; the real problem is almost always ownership (`chown`) or a missing `x` on a parent directory.
|
||||
- **"Permission denied" on a file with `rw-`?** Every directory in the path needs `x`. Debug the whole chain with `namei -l /path/to/file`.
|
||||
- **Interpreted scripts need `r` too** — the kernel hands the file to the interpreter, which has to read it.
|
||||
- **Group changes need a new login** (or `newgrp <group>`) before the shell picks them up.
|
||||
- **FAT/NTFS/exFAT mounts ignore `chmod`** — set permissions with the `umask=`, `fmask=` and `dmask=` mount options instead.
|
||||
- **Removing `x` from a directory hides everything below it**, even world-readable files.
|
||||
|
||||
💡 Tip: Use `ls -l` to view current permissions.
|
||||
## 📚 Resources
|
||||
|
||||
- [GNU coreutils — `chmod` invocation](https://www.gnu.org/software/coreutils/manual/html_node/chmod-invocation.html)
|
||||
- [GNU coreutils — file permissions](https://www.gnu.org/software/coreutils/manual/html_node/File-permissions.html)
|
||||
- Man pages: `man 1 chmod` · `man 1 umask` · `man 1 chown` · `man 5 acl`
|
||||
@@ -1,174 +1,253 @@
|
||||
Patterns you reach for again and again, written for the **JavaScript/PCRE** flavour. Drop the surrounding anchors (`^`…`$`) when you want to *find* matches inside a longer text instead of validating a whole string.
|
||||
|
||||
### Commonly used Regex
|
||||
|
||||
#### 1\. Digits
|
||||
> ⚠️ Validation by regex is always an approximation. For emails, phone numbers, URLs and credit cards, a regex is a cheap first filter — the real check is a library, or sending the confirmation.
|
||||
|
||||
* [Whole Numbers](https://www.regexpal.com/?fam=104020) – `/^\d+$/`
|
||||
* [Decimal Numbers](https://www.regexpal.com/?fam=104021) – `/^\d*\.\d+$/`
|
||||
* [Whole + Decimal Numbers](https://www.regexpal.com/?fam=104022) – `/^\d*(\.\d+)?$/`
|
||||
* [Negative, Positive Whole + Decimal Numbers](https://www.regexpal.com/?fam=104023) – `/^-?\d*(\.\d+)?$/`
|
||||
* [Whole + Decimal + Fractions](https://www.regexpal.com/94462) – `/[-]?[0-9]+[,.]?[0-9]*([\/][0-9]+[,.]?[0-9]*)*/`
|
||||
## 🔢 Numbers
|
||||
|
||||
#### 2\. Alphanumeric Characters
|
||||
```regex
|
||||
# whole number
|
||||
^\d+$
|
||||
|
||||
* [Alphanumeric without space](https://www.regexpal.com/?fam=104024) – `/^[a-zA-Z0-9]*$/`
|
||||
* [Alphanumeric with space](https://www.regexpal.com/?fam=104025) – `/^[a-zA-Z0-9 ]*$/`
|
||||
# decimal number
|
||||
^\d*\.\d+$
|
||||
|
||||
#### 3\. Email
|
||||
# whole or decimal
|
||||
^\d*(\.\d+)?$
|
||||
|
||||
* [Common email Ids](https://www.regexpal.com/?fam=104026) – `/^([a-zA-Z0-9._%-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,6})*$/`
|
||||
* [Uncommon email ids](https://www.regexpal.com/?fam=104027) – `/^([a-z0-9_\.\+-]+)@([\da-z\.-]+)\.([a-z\.]{2,6})$/`
|
||||
# optional sign, whole or decimal
|
||||
^[+-]?\d*(\.\d+)?$
|
||||
|
||||
#### 4\. Password Strength
|
||||
# thousands separators: 1,234,567.89
|
||||
^\d{1,3}(,\d{3})*(\.\d+)?$
|
||||
|
||||
* [Complex](https://www.regexpal.com/?fam=104028): Should have 1 lowercase letter, 1 uppercase letter, 1 number, 1 special character and be at least 8 characters long
|
||||
# percentage, 0–100 with optional decimals
|
||||
^(100(\.0+)?|\d{1,2}(\.\d+)?)%?$
|
||||
|
||||
/(?=(.\*\[0-9\]))(?=.\*\[\\!@#$%^&\*()\\\\\[\\\]{}\\-\_+=~\`|:;"'<>,./?\])(?=.\*\[a-z\])(?=(.\*\[A-Z\]))(?=(.\*)).{8,}/
|
||||
# hexadecimal number
|
||||
^(0[xX])?[0-9a-fA-F]+$
|
||||
|
||||
* [Moderate](https://www.regexpal.com/?fam=104029): Should have 1 lowercase letter, 1 uppercase letter, 1 number, and be at least 8 characters long
|
||||
|
||||
/(?=(.\*\[0-9\]))((?=.\*\[A-Za-z0-9\])(?=.\*\[A-Z\])(?=.\*\[a-z\]))^.{8,}$/
|
||||
|
||||
#### 5\. Username
|
||||
|
||||
* [Alphanumeric string](https://www.regexpal.com/?fam=104030) that may include \_ and – having a length of 3 to 16 characters – `/^[a-z0-9_-]{3,16}$/`
|
||||
|
||||
#### 6\. URL
|
||||
|
||||
* Include [http(s) Protocol](https://www.regexpal.com/?fam=104034)
|
||||
|
||||
```
|
||||
/https?:\\/\\/(www\\.)?\[-a-zA-Z0-9@:%.\_\\+~#=\]{2,256}\\.\[a-z\]{2,6}\\b(\[-a-zA-Z0-9@:%\_\\+.~#()?&//=\]\*)/
|
||||
# currency amount: $1,234.56
|
||||
^\$?\d{1,3}(,?\d{3})*(\.\d{2})?$
|
||||
```
|
||||
|
||||
* [Protocol Optional](https://www.regexpal.com/?fam=104035)
|
||||
## 🔤 Text & Identifiers
|
||||
|
||||
```
|
||||
/(https?:\\/\\/)?(www\\.)?\[-a-zA-Z0-9@:%.\_\\+~#=\]{2,256}\\.\[a-z\]{2,6}\\b(\[-a-zA-Z0-9@:%\_\\+.~#?&//=\]\*)/
|
||||
```regex
|
||||
# alphanumeric, no spaces
|
||||
^[a-zA-Z0-9]+$
|
||||
|
||||
# alphanumeric with spaces
|
||||
^[a-zA-Z0-9 ]+$
|
||||
|
||||
# username: 3–16 chars, letters, digits, underscore, dash
|
||||
^[a-z0-9_-]{3,16}$
|
||||
|
||||
# slug: lowercase words joined by single dashes
|
||||
^[a-z0-9]+(?:-[a-z0-9]+)*$
|
||||
|
||||
# hex colour, 3 or 6 digits
|
||||
^#?([0-9a-fA-F]{3}|[0-9a-fA-F]{6})$
|
||||
|
||||
# UUID v4
|
||||
^[0-9a-f]{8}-[0-9a-f]{4}-4[0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}$
|
||||
|
||||
# semantic version (major.minor.patch with optional pre-release and build)
|
||||
^\d+\.\d+\.\d+(-[0-9A-Za-z.-]+)?(\+[0-9A-Za-z.-]+)?$
|
||||
|
||||
# leading and trailing whitespace
|
||||
^\s+|\s+$
|
||||
|
||||
# blank or whitespace-only line
|
||||
^\s*$
|
||||
```
|
||||
|
||||
#### 7\. IP Address
|
||||
## 📧 Email
|
||||
|
||||
* [IPv4 address](https://www.regexpal.com/?fam=104036)
|
||||
```
|
||||
/^(([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])\.){3}([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])$/
|
||||
```
|
||||
* [IPv6 address](https://www.regexpal.com/?fam=104037)
|
||||
```
|
||||
/(([0-9a-fA-F]{1,4}:){7,7}[0-9a-fA-F]{1,4}|([0-9a-fA-F]{1,4}:){1,7}:|([0-9a-fA-F]{1,4}:){1,6}:[0-9a-fA-F]{1,4}|([0-9a-fA-F]{1,4}:){1,5}(:[0-9a-fA-F]{1,4}){1,2}|([0-9a-fA-F]{1,4}:){1,4}(:[0-9a-fA-F]{1,4}){1,3}|([0-9a-fA-F]{1,4}:){1,3}(:[0-9a-fA-F]{1,4}){1,4}|([0-9a-fA-F]{1,4}:){1,2}(:[0-9a-fA-F]{1,4}){1,5}|[0-9a-fA-F]{1,4}:((:[0-9a-fA-F]{1,4}){1,6})|:((:[0-9a-fA-F]{1,4}){1,7}|:)|fe80:(:[0-9a-fA-F]{0,4}){0,4}%[0-9a-zA-Z]{1,}|::(ffff(:0{1,4}){0,1}:){0,1}((25[0-5]|(2[0-4]|1{0,1}[0-9]){0,1}[0-9])\.){3,3}(25[0-5]|(2[0-4]|1{0,1}[0-9]){0,1}[0-9])|([0-9a-fA-F]{1,4}:){1,4}:((25[0-5]|(2[0-4]|1{0,1}[0-9]){0,1}[0-9])\.){3,3}(25[0-5]|(2[0-4]|1{0,1}[0-9]){0,1}[0-9]))/
|
||||
```regex
|
||||
# pragmatic: good enough for a form field
|
||||
^[^\s@]+@[^\s@]+\.[^\s@]+$
|
||||
|
||||
# stricter on the allowed characters
|
||||
^[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}$
|
||||
|
||||
# find addresses inside a body of text
|
||||
[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}
|
||||
```
|
||||
|
||||
* Both [IPv4, IPv6 addresses](https://www.regexpal.com/?fam=104038)
|
||||
```
|
||||
/((^\s*((([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])\.){3}([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5]))\s*$)|(^\s*((([0-9A-Fa-f]{1,4}:){7}([0-9A-Fa-f]{1,4}|:))|(([0-9A-Fa-f]{1,4}:){6}(:[0-9A-Fa-f]{1,4}|((25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)(\.(25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)){3})|:))|(([0-9A-Fa-f]{1,4}:){5}(((:[0-9A-Fa-f]{1,4}){1,2})|:((25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)(\.(25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)){3})|:))|(([0-9A-Fa-f]{1,4}:){4}(((:[0-9A-Fa-f]{1,4}){1,3})|((:[0-9A-Fa-f]{1,4})?:((25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)(\.(25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)){3}))|:))|(([0-9A-Fa-f]{1,4}:){3}(((:[0-9A-Fa-f]{1,4}){1,4})|((:[0-9A-Fa-f]{1,4}){0,2}:((25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)(\.(25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)){3}))|:))|(([0-9A-Fa-f]{1,4}:){2}(((:[0-9A-Fa-f]{1,4}){1,5})|((:[0-9A-Fa-f]{1,4}){0,3}:((25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)(\.(25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)){3}))|:))|(([0-9A-Fa-f]{1,4}:){1}(((:[0-9A-Fa-f]{1,4}){1,6})|((:[0-9A-Fa-f]{1,4}){0,4}:((25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)(\.(25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)){3}))|:))|(:(((:[0-9A-Fa-f]{1,4}){1,7})|((:[0-9A-Fa-f]{1,4}){0,5}:((25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)(\.(25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)){3}))|:)))(%.+)?\s*$))/`
|
||||
## 🔗 URLs
|
||||
|
||||
```regex
|
||||
# http(s) URL
|
||||
^https?:\/\/[^\s/$.?#].[^\s]*$
|
||||
|
||||
# protocol optional
|
||||
^(https?:\/\/)?([\w-]+\.)+[a-z]{2,}(\/[^\s]*)?$
|
||||
|
||||
# domain name only
|
||||
^([a-zA-Z0-9-]+\.)+[a-zA-Z]{2,}$
|
||||
|
||||
# extract every link from text
|
||||
https?:\/\/[^\s<>"']+
|
||||
|
||||
# capture the parts: protocol, host, path
|
||||
^(https?):\/\/([^\/\s:]+)(?::(\d+))?(\/[^\s?#]*)?
|
||||
```
|
||||
|
||||
#### 8\. Dates
|
||||
## 🌐 Network
|
||||
|
||||
* Date Format [YYYY-MM-dd](https://www.regexpal.com/?fam=104039) using separator `-`
|
||||
`/([12]\d{3}-(0[1-9]|1[0-2])-(0[1-9]|[12]\d|3[01]))/`
|
||||
* Date Format [dd-MM-YYYY](https://regexr.com/?346hf) using separators `-` or `.` or `/`
|
||||
`/^(?:(?:31(\/|-|\.)(?:0?[13578]|1[02]))\1|(?:(?:29|30)(\/|-|\.)(?:0?[1,3-9]|1[0-2])\2))(?:(?:1[6-9]|[2-9]\d)?\d{2})$|^(?:29(\/|-|\.)0?2\3(?:(?:(?:1[6-9]|[2-9]\d)?(?:0[48]|[2468][048]|[13579][26])|(?:(?:16|[2468][048]|[3579][26])00))))$|^(?:0?[1-9]|1\d|2[0-8])(\/|-|\.)(?:(?:0?[1-9])|(?:1[0-2]))\4(?:(?:1[6-9]|[2-9]\d)?\d{2})$/`
|
||||
* Date Format [dd-mmm-YYYY](https://regexr.com/39tr1) using separators `-` or `.` or `/`
|
||||
`/^(?:(?:31(\/|-|\.)(?:0?[13578]|1[02]|(?:Jan|Mar|May|Jul|Aug|Oct|Dec)))\1|(?:(?:29|30)(\/|-|\.)(?:0?[1,3-9]|1[0-2]|(?:Jan|Mar|Apr|May|Jun|Jul|Aug|Sep|Oct|Nov|Dec))\2))(?:(?:1[6-9]|[2-9]\d)?\d{2})$|^(?:29(\/|-|\.)(?:0?2|(?:Feb))\3(?:(?:(?:1[6-9]|[2-9]\d)?(?:0[48]|[2468][048]|[13579][26])|(?:(?:16|[2468][048]|[3579][26])00))))$|^(?:0?[1-9]|1\d|2[0-8])(\/|-|\.)(?:(?:0?[1-9]|(?:Jan|Feb|Mar|Apr|May|Jun|Jul|Aug|Sep))|(?:1[0-2]|(?:Oct|Nov|Dec)))\4(?:(?:1[6-9]|[2-9]\d)?\d{2})$/`
|
||||
|
||||
#### 9\. Time
|
||||
```regex
|
||||
# IPv4 address
|
||||
^((25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)\.){3}(25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)$
|
||||
|
||||
* Time Format [HH:MM 12-hour](https://www.regexpal.com/?fam=104040), optional leading 0
|
||||
# IPv4 with CIDR prefix
|
||||
^((25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)\.){3}(25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)\/(3[0-2]|[12]?\d)$
|
||||
|
||||
`/^(0?[1-9]|1[0-2]):[0-5][0-9]$/`
|
||||
# IPv6, full or compressed
|
||||
^(([0-9a-fA-F]{1,4}:){7}[0-9a-fA-F]{1,4}|([0-9a-fA-F]{1,4}:){1,7}:|([0-9a-fA-F]{1,4}:){1,6}:[0-9a-fA-F]{1,4}|([0-9a-fA-F]{1,4}:){1,5}(:[0-9a-fA-F]{1,4}){1,2}|([0-9a-fA-F]{1,4}:){1,4}(:[0-9a-fA-F]{1,4}){1,3}|([0-9a-fA-F]{1,4}:){1,3}(:[0-9a-fA-F]{1,4}){1,4}|([0-9a-fA-F]{1,4}:){1,2}(:[0-9a-fA-F]{1,4}){1,5}|[0-9a-fA-F]{1,4}:(:[0-9a-fA-F]{1,4}){1,6}|:((:[0-9a-fA-F]{1,4}){1,7}|:))$
|
||||
|
||||
* Time Format HH:MM 12-hour, optional leading 0, **[Meridiems (AM/PM)](https://www.regexpal.com/?fam=104041)**
|
||||
# MAC address, colon- or dash-separated
|
||||
^([0-9A-Fa-f]{2}[:-]){5}[0-9A-Fa-f]{2}$
|
||||
|
||||
`/((1[0-2]|0?[1-9]):([0-5][0-9]) ?([AaPp][Mm]))/`
|
||||
|
||||
* Time Format [HH:MM 24-hour](https://www.regexpal.com/?fam=104042) with leading 0
|
||||
|
||||
`/^(0[0-9]|1[0-9]|2[0-3]):[0-5][0-9]$/`
|
||||
|
||||
* Time Format [HH:MM 24-hour, optional leading 0](https://www.regexpal.com/?fam=104043)
|
||||
|
||||
`/^([0-9]|0[0-9]|1[0-9]|2[0-3]):[0-5][0-9]$/`
|
||||
|
||||
* Time Format [HH:MM:SS 24-hour](https://www.regexpal.com/?fam=104044)
|
||||
|
||||
`/(?:[01]\d|2[0123]):(?:[012345]\d):(?:[012345]\d)/`
|
||||
|
||||
#### 10\. HTML Tags
|
||||
|
||||
* [Elements with Attributes](https://www.regexpal.com/95941) `/<\/?[\w\s]*>|<.+[\W]>/`
|
||||
|
||||
#### 11\. Javascript Handlers
|
||||
|
||||
* [Inline JS handler](https://www.regexpal.com/?fam=104055) `/\bon\w+=\S+(?=.*>)/`
|
||||
* [Inline JS handler with element](https://www.regexpal.com/94641) `/(?:<[^>]+\s)(on\S+)=["']?((?:.(?!["']?\s+(?:\S+)=|[>"']))+.)["']?/`
|
||||
|
||||
#### 12\. Slug
|
||||
|
||||
* [Slug](https://www.regexpal.com/?fam=104056) `/^[a-z0-9]+(?:-[a-z0-9]+)*$/`
|
||||
|
||||
#### 13\. Match Duplicates in a String
|
||||
|
||||
* [Search Duplicates](https://www.regexpal.com/?fam=104060) `/(\b\w+\b)(?=.*\b\1\b)/`
|
||||
|
||||
#### 14\. Phone Numbers
|
||||
|
||||
* [International Phone Numbers](https://www.regexpal.com/?fam=99127) – with optional country code/extension
|
||||
```
|
||||
`/* International Phone Numbers */`
|
||||
/^(?:(?:\(?(?:00|\+)([1-4]\d\d|[1-9]\d?)\)?)?[\-\.\ \\\/]?)?((?:\(?\d{1,}\)?[\-\.\ \\\/]?){0,})(?:[\-\.\ \\\/]?(?:``#|ext\.?|extension|x)[\-\.\ \\\/]?(\d+))?$/
|
||||
# port number, 0–65535
|
||||
^([0-9]{1,4}|[1-5][0-9]{4}|6[0-4][0-9]{3}|65[0-4][0-9]{2}|655[0-2][0-9]|6553[0-5])$
|
||||
```
|
||||
|
||||
_Note:_ Use regex for validating phone numbers **only** if you don’t have the choice to use a library. There are [several libraries](https://stackoverflow.com/a/15644461/4717533) that handle phone numbers more accurately and should be used instead.
|
||||
## 📅 Dates & Times
|
||||
|
||||
#### 15\. File Path
|
||||
```regex
|
||||
# ISO 8601 date: 2026-08-22
|
||||
^\d{4}-(0[1-9]|1[0-2])-(0[1-9]|[12]\d|3[01])$
|
||||
|
||||
* [File Path with Filename and extension](https://www.regexpal.com/?fam=104047)
|
||||
# ISO 8601 date and time with optional zone
|
||||
^\d{4}-\d{2}-\d{2}[T ]\d{2}:\d{2}(:\d{2}(\.\d+)?)?(Z|[+-]\d{2}:?\d{2})?$
|
||||
|
||||
```
|
||||
/((\\/|\\\\|\\/\\/|https?:\\\\\\\\|https?:\\/\\/)\[a-z0-9 \_@\\-^!#$%&+={}.\\/\\\\\\\[\\\]\]+)+\\.\[a-z\]+$/
|
||||
# day-first with - . or / as separator
|
||||
^(0?[1-9]|[12]\d|3[01])([-./])(0?[1-9]|1[0-2])\2\d{4}$
|
||||
|
||||
# 24-hour time, optional seconds
|
||||
^([01]\d|2[0-3]):[0-5]\d(:[0-5]\d)?$
|
||||
|
||||
# 12-hour time with meridiem
|
||||
^(0?[1-9]|1[0-2]):[0-5]\d\s?([AaPp][Mm])$
|
||||
|
||||
# duration in ISO 8601: P1DT2H30M
|
||||
^P(?!$)(\d+Y)?(\d+M)?(\d+D)?(T(?=\d)(\d+H)?(\d+M)?(\d+S)?)?$
|
||||
```
|
||||
|
||||
* File Path with optional Filename, extension
|
||||
> ℹ️ These check the *shape*, not the calendar — `2026-02-31` passes. Parse the value if the date has to be real.
|
||||
|
||||
```
|
||||
/^(.+)/(\[^/\]+)$/
|
||||
## 🔐 Password Rules
|
||||
|
||||
Each requirement is a lookahead; combine only the ones you actually need.
|
||||
|
||||
```regex
|
||||
# at least 8 characters, one lower, one upper, one digit
|
||||
^(?=.*[a-z])(?=.*[A-Z])(?=.*\d).{8,}$
|
||||
|
||||
# the same, plus one special character
|
||||
^(?=.*[a-z])(?=.*[A-Z])(?=.*\d)(?=.*[^\w\s]).{8,}$
|
||||
|
||||
# at least 12 characters, no other rules (the modern recommendation)
|
||||
^.{12,}$
|
||||
|
||||
# individual checks, to report which rule failed
|
||||
[a-z] # has a lower-case letter
|
||||
[A-Z] # has an upper-case letter
|
||||
\d # has a digit
|
||||
[^\w\s] # has a symbol
|
||||
```
|
||||
|
||||
* [File Name with extension](https://www.regexpal.com/?fam=104048) having 3 chars
|
||||
## ☎️ Phone Numbers
|
||||
|
||||
```
|
||||
/^\[\\w,\\s-\]+\\.\[A-Za-z\]{3}$/
|
||||
```regex
|
||||
# E.164, the format to store numbers in
|
||||
^\+[1-9]\d{1,14}$
|
||||
|
||||
# lenient international, allowing spaces, dashes, dots and parentheses
|
||||
^\+?[\d\s().-]{7,20}$
|
||||
|
||||
# North American number, with or without country code
|
||||
^(\+?1[-. ]?)?\(?\d{3}\)?[-. ]?\d{3}[-. ]?\d{4}$
|
||||
```
|
||||
|
||||
### Additional Regexes
|
||||
> 💡 Use a library (libphonenumber and its ports) for anything beyond a smoke test — numbering plans change, and regex cannot know which prefixes exist.
|
||||
|
||||
#### 1\. Zip codes
|
||||
## 📄 Files & Paths
|
||||
|
||||
There is **NO** single Regex that can handle all zip codes given that zip codes around the world do not follow a common pattern. Here is a [list](https://stackoverflow.com/a/7185241/4717533) that contains Regex specific to each country.
|
||||
```regex
|
||||
# file name with an extension
|
||||
^[\w,\s-]+\.[A-Za-z0-9]+$
|
||||
|
||||
#### 2\. Payment Validation
|
||||
# capture the name and the extension separately
|
||||
^(.+?)\.([A-Za-z0-9]+)$
|
||||
|
||||
Here is a [link](https://www.regular-expressions.info/creditcard.html) that contains regex for validating leading Credit cards like Visa, Mastercard and so on.
|
||||
# extension only
|
||||
\.[0-9a-z]+$
|
||||
|
||||
#### 3\. Identity Documents
|
||||
# absolute Unix path
|
||||
^(\/[^\/\0]+)+\/?$
|
||||
|
||||
* Social Security Number – [Ref](https://www.codeproject.com/Articles/651609/Validating-Social-Security-Numbers-through-Regular)
|
||||
# Windows path
|
||||
^[a-zA-Z]:\\(?:[^\\/:*?"<>|\r\n]+\\)*[^\\/:*?"<>|\r\n]*$
|
||||
|
||||
```
|
||||
/\* can use either hypen(-) or space( ) character as separator \*/
|
||||
|
||||
/^((?!219-09-9999|078-05-1120)(?!666|000|9\\d{2})\\d{3}-(?!00)\\d{2}-(?!0{4})\\d{4})|((?!219 09 9999|078 05 1120)(?!666|000|9\\d{2})\\d{3} (?!00)\\d{2} (?!0{4})\\d{4})|((?!219099999|078051120)(?!666|000|9\\d{2})\\d{3}(?!00)\\d{2}(?!0{4})\\d{4})$/
|
||||
# image file
|
||||
\.(jpe?g|png|gif|webp|avif|svg)$
|
||||
```
|
||||
|
||||
* Passport – `/^[A-PR-WY][1-9]\d\s?\d{4}[1-9]$/`
|
||||
## 🏷 Markup & Logs
|
||||
|
||||
#### References
|
||||
* [https://www.codeproject.com/validate-passport-number](https://www.codeproject.com/Questions/1046445/How-to-validate-passport-number-using-javascript)
|
||||
* [https://gist.github.com/](https://gist.github.com/nerdsrescueme/1237767)
|
||||
* [https://code.tutsplus.com/regular-expressions](https://code.tutsplus.com/tutorials/8-regular-expressions-you-should-know--net-6149)
|
||||
* [https://www.regular-expressions.info](https://www.regular-expressions.info/)
|
||||
* [http://www.regexlib.com/](http://www.regexlib.com/)
|
||||
* [https://projects.lukehaas.me/regexhub/](https://projects.lukehaas.me/regexhub/)
|
||||
* [http://wiki.zoolz.com/commonly-used-regular-expressions/](http://wiki.zoolz.com/commonly-used-regular-expressions/)
|
||||
* [https://www.smashingmagazine.com/advanced-regular-expressions/](https://www.smashingmagazine.com/2009/05/introduction-to-advanced-regular-expressions/)
|
||||
```regex
|
||||
# any HTML tag
|
||||
<\/?[a-zA-Z][\w-]*(\s[^>]*)?>
|
||||
|
||||
# HTML comment
|
||||
<!--[\s\S]*?-->
|
||||
|
||||
# inline JavaScript handler: onclick=, onload=, ...
|
||||
\bon\w+\s*=\s*["'][^"']*["']
|
||||
|
||||
# content between double quotes (non-greedy)
|
||||
"([^"\\]*(\\.[^"\\]*)*)"
|
||||
|
||||
# an @mention or #hashtag
|
||||
[@#][\w-]+
|
||||
|
||||
# common log line: IP, timestamp, method, path, status
|
||||
^(\S+) \S+ \S+ \[([^\]]+)\] "(\w+) ([^"]*)" (\d{3})
|
||||
```
|
||||
|
||||
## 🧰 Handy Tricks
|
||||
|
||||
```regex
|
||||
# a word repeated later in the same text
|
||||
(\b\w+\b)(?=.*\b\1\b)
|
||||
|
||||
# two consecutive identical words
|
||||
\b(\w+)\s+\1\b
|
||||
|
||||
# a line that does NOT contain a word
|
||||
^(?!.*forbidden).*$
|
||||
|
||||
# everything between two markers, non-greedy
|
||||
BEGIN([\s\S]*?)END
|
||||
|
||||
# strip ANSI colour codes
|
||||
\x1B\[[0-9;]*[A-Za-z]
|
||||
|
||||
# split CSV, respecting quoted fields
|
||||
("([^"]|"")*"|[^,]*)(,|$)
|
||||
```
|
||||
|
||||
## ⚠️ Notes
|
||||
|
||||
- **Postal codes have no universal pattern** — every country differs. Look up the one you need.
|
||||
- **Credit cards**: check the shape, then verify with the Luhn algorithm; brand prefixes are documented at [regular-expressions.info](https://www.regular-expressions.info/creditcard.html).
|
||||
- **National identifiers** (SSN, passport, tax numbers) carry checksums and legal edge cases a regex cannot express.
|
||||
- **Watch out for catastrophic backtracking** in patterns with nested quantifiers — test any user-facing regex against long, nearly-matching input.
|
||||
|
||||
## 📚 Resources
|
||||
|
||||
- [regex101 — test and explain](https://regex101.com)
|
||||
- [Regular-Expressions.info](https://www.regular-expressions.info/)
|
||||
- [RegExHub — common patterns](https://projects.lukehaas.me/regexhub/)
|
||||
- [MDN — regular expressions](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Guide/Regular_expressions)
|
||||
@@ -1,63 +1,147 @@
|
||||
| Selector | Example | Example description |
|
||||
|-----------------------|-------------------------|----------------------------------------------------------------------------------------------------------|
|
||||
| `.class` | `.intro` | Selects all elements with class="intro" |
|
||||
| `.class1.class2` | `.name1.name2` | Selects all elements with both name1 and name2 set within its class attribute |
|
||||
| `.class1 .class2` | `.name1 .name2` | Selects all elements with name2 that is a descendant of an element with name1 |
|
||||
| `#id` | `#firstname` | Selects the element with id="firstname" |
|
||||
| `*` | `*` | Selects all elements |
|
||||
| `element` | `p` | Selects all \<p\> elements |
|
||||
| `element.class` | `p.intro` | Selects all \<p\> elements with class="intro" |
|
||||
| `element,element` | `div, p` | Selects all \<div\> elements and all \<p\> elements |
|
||||
| `element element` | `div p` | Selects all \<p\> elements inside \<div\> elements |
|
||||
| `element>element` | `div > p` | Selects all \<p\> elements where the parent is a \<div\> element |
|
||||
| `element+element` | `div + p` | Selects the first \<p\> element that is placed immediately after \<div\> elements |
|
||||
| `element1~element2` | `p ~ ul` | Selects every \<ul\> element that is preceded by a \<p\> element |
|
||||
| `[attribute]` | `[target]` | Selects all elements with a target attribute |
|
||||
| `[attribute=value]` | `[target="_blank"]` | Selects all elements with target="_blank" |
|
||||
| `[attribute~=value]` | `[title~="flower"]` | Selects all elements with a title attribute containing the word "flower" |
|
||||
| `[attribute\|=value]` | `[lang\|="en"]` | Selects all elements with a lang attribute value equal to "en" or starting with "en-" |
|
||||
| `[attribute^=value]` | `a[href^="https"]` | Selects every \<a\> element whose href attribute value begins with "https" |
|
||||
| `[attribute$=value]` | `a[href$=".pdf"]` | Selects every \<a\> element whose href attribute value ends with ".pdf" |
|
||||
| `[attribute*=value]` | `a[href*="w3schools"]` | Selects every \<a\> element whose href attribute value contains the substring "w3schools" |
|
||||
| `:active` | `a:active` | Selects the active link |
|
||||
| `::after` | `p::after` | Insert something after the content of each \<p\> element |
|
||||
| `::before` | `p::before` | Insert something before the content of each \<p\> element |
|
||||
| `:checked` | `input:checked` | Selects every checked \<input\> element |
|
||||
| `:default` | `input:default` | Selects the default \<input\> element |
|
||||
| `:disabled` | `input:disabled` | Selects every disabled \<input\> element |
|
||||
| `:empty` | `p:empty` | Selects every \<p\> element that has no children (including text nodes) |
|
||||
| `:enabled` | `input:enabled` | Selects every enabled \<input\> element |
|
||||
| `:first-child` | `p:first-child` | Selects every \<p\> element that is the first child of its parent |
|
||||
| `::first-letter` | `p::first-letter` | Selects the first letter of every \<p\> element |
|
||||
| `::first-line` | `p::first-line` | Selects the first line of every \<p\> element |
|
||||
| `:first-of-type` | `p:first-of-type` | Selects every \<p\> element that is the first \<p\> element of its parent |
|
||||
| `:focus` | `input:focus` | Selects the input element which has focus |
|
||||
| `:fullscreen` | `:fullscreen` | Selects the element that is in full-screen mode |
|
||||
| `:has()` | `h2:has(+p)` | Selects h2 elements that are immediately followed by a p element, and applies the style to h2 |
|
||||
| `:hover` | `a:hover` | Selects links on mouse over |
|
||||
| `:in-range` | `input:in-range` | Selects input elements with a value within a specified range |
|
||||
| `:indeterminate` | `input:indeterminate` | Selects input elements that are in an indeterminate state |
|
||||
| `:invalid` | `input:invalid` | Selects all input elements with an invalid value |
|
||||
| `:lang()` | `p:lang(it)` | Selects every \<p\> element with a lang attribute equal to "it" (Italian) |
|
||||
| `:last-child` | `p:last-child` | Selects every \<p\> element that is the last child of its parent |
|
||||
| `:last-of-type` | `p:last-of-type` | Selects every \<p\> element that is the last \<p\> element of its parent |
|
||||
| `:link` | `a:link` | Selects all unvisited links |
|
||||
| `::marker` | `::marker` | Selects the markers of list items |
|
||||
| `:not()` | `:not(p)` | Selects every element that is not a \<p\> element |
|
||||
| `:nth-child()` | `p:nth-child(2)` | Selects every \<p\> element that is the second child of its parent |
|
||||
| `:nth-last-child()` | `p:nth-last-child(2)` | Selects every \<p\> element that is the second child of its parent, counting from the last child |
|
||||
| `:nth-last-of-type()` | `p:nth-last-of-type(2)` | Selects every \<p\> element that is the second \<p\> element of its parent, counting from the last child |
|
||||
| `:nth-of-type()` | `p:nth-of-type(2)` | Selects every \<p\> element that is the second \<p\> element of its parent |
|
||||
| `:only-of-type` | `p:only-of-type` | Selects every \<p\> element that is the only \<p\> element of its parent |
|
||||
| `:only-child` | `p:only-child` | Selects every \<p\> element that is the only child of its parent |
|
||||
| `:optional` | `input:optional` | Selects input elements with no "required" attribute |
|
||||
| `:out-of-range` | `input:out-of-range` | Selects input elements with a value outside a specified range |
|
||||
| `::placeholder` | `input::placeholder` | Selects input elements with the "placeholder" attribute specified |
|
||||
| `:read-only` | `input:read-only` | Selects input elements with the "readonly" attribute specified |
|
||||
| `:read-write` | `input:read-write` | Selects input elements with the "readonly" attribute NOT specified |
|
||||
| `:required` | `input:required` | Selects input elements with the "required" attribute specified |
|
||||
| `:root` | `:root` | Selects the document's root element |
|
||||
| `::selection` | `::selection` | Selects the portion of an element that is selected by a user |
|
||||
| `:target` | `#news:target` | Selects the current active #news element (clicked on a URL containing that anchor name) |
|
||||
| `:valid` | `input:valid` | Selects all input elements with a valid value |
|
||||
| `:visited` | `a:visited` | Selects all visited links |
|
||||
**CSS selectors** pick the elements a rule applies to. They are also what `document.querySelectorAll()`, Playwright, Puppeteer and most scrapers use, so the same syntax works far beyond stylesheets.
|
||||
|
||||
```css
|
||||
/* every <a> inside a .card that opens in a new tab */
|
||||
.card a[target="_blank"] { color: rebeccapurple; }
|
||||
```
|
||||
|
||||
## 🎯 Basic Selectors
|
||||
|
||||
| Selector | Example | Selects |
|
||||
|------------------|----------------|--------------------------------------------|
|
||||
| `*` | `*` | Every element |
|
||||
| `element` | `p` | Every `<p>` element |
|
||||
| `.class` | `.intro` | Every element with `class="intro"` |
|
||||
| `#id` | `#firstname` | The element with `id="firstname"` |
|
||||
| `.class1.class2` | `.name1.name2` | Elements carrying **both** classes |
|
||||
| `element.class` | `p.intro` | Every `<p>` with `class="intro"` |
|
||||
| `a, b` | `div, p` | Every `<div>` **and** every `<p>` (a list) |
|
||||
|
||||
## 🔗 Combinators
|
||||
|
||||
| Selector | Example | Selects |
|
||||
|----------|-----------|---------------------------------------------------------------------------|
|
||||
| `a b` | `div p` | Every `<p>` anywhere inside a `<div>` (descendant) |
|
||||
| `a > b` | `div > p` | Every `<p>` whose direct parent is a `<div>` (child) |
|
||||
| `a + b` | `div + p` | The `<p>` immediately after a `<div>` (adjacent sibling) |
|
||||
| `a ~ b` | `p ~ ul` | Every `<ul>` that follows a `<p>` under the same parent (general sibling) |
|
||||
|
||||
## 🏷 Attribute Selectors
|
||||
|
||||
| Selector | Example | Selects |
|
||||
|--------------------|---------------------|--------------------------------------------------------------------|
|
||||
| `[attr]` | `[target]` | Elements that have the attribute at all |
|
||||
| `[attr="value"]` | `[target="_blank"]` | Exact value |
|
||||
| `[attr~="value"]` | `[title~="flower"]` | Value is one of a space-separated list |
|
||||
| `[attr\|="value"]` | `[lang\|="en"]` | Value is `en` or starts with `en-` |
|
||||
| `[attr^="value"]` | `a[href^="https"]` | Value starts with |
|
||||
| `[attr$="value"]` | `a[href$=".pdf"]` | Value ends with |
|
||||
| `[attr*="value"]` | `a[href*="docs"]` | Value contains |
|
||||
| `[attr="value" i]` | `[type="TEXT" i]` | Case-insensitive match (add `s` instead to force case-sensitivity) |
|
||||
|
||||
## 🌳 Structural Pseudo-Classes
|
||||
|
||||
| Selector | Example | Selects |
|
||||
|------------------------|-------------------------|----------------------------------------------------------|
|
||||
| `:root` | `:root` | The document root — where custom properties usually live |
|
||||
| `:first-child` | `p:first-child` | A `<p>` that is the first child of its parent |
|
||||
| `:last-child` | `p:last-child` | A `<p>` that is the last child of its parent |
|
||||
| `:only-child` | `p:only-child` | A `<p>` that is the only child |
|
||||
| `:nth-child(n)` | `li:nth-child(2)` | The second child — also `odd`, `even`, `3n+1` |
|
||||
| `:nth-last-child(n)` | `li:nth-last-child(2)` | Counting from the end |
|
||||
| `:first-of-type` | `p:first-of-type` | The first `<p>` among its siblings |
|
||||
| `:last-of-type` | `p:last-of-type` | The last `<p>` among its siblings |
|
||||
| `:only-of-type` | `p:only-of-type` | The only `<p>` among its siblings |
|
||||
| `:nth-of-type(n)` | `p:nth-of-type(2)` | The second `<p>` among its siblings |
|
||||
| `:nth-last-of-type(n)` | `p:nth-last-of-type(2)` | Same, counting from the end |
|
||||
| `:empty` | `p:empty` | No children at all, text included |
|
||||
|
||||
## 🖱 State & Interaction
|
||||
|
||||
| Selector | Example | Selects |
|
||||
|------------------|------------------------|------------------------------------------------------------------|
|
||||
| `:link` | `a:link` | Unvisited links |
|
||||
| `:visited` | `a:visited` | Visited links |
|
||||
| `:hover` | `a:hover` | The element under the pointer |
|
||||
| `:active` | `a:active` | The element being clicked |
|
||||
| `:focus` | `input:focus` | The focused element |
|
||||
| `:focus-visible` | `button:focus-visible` | Focused *and* the browser thinks a ring belongs there (keyboard) |
|
||||
| `:focus-within` | `form:focus-within` | An element containing the focused element |
|
||||
| `:target` | `#news:target` | The element the URL fragment points at |
|
||||
| `:fullscreen` | `:fullscreen` | The element displayed full-screen |
|
||||
|
||||
## 📝 Form Pseudo-Classes
|
||||
|
||||
| Selector | Example | Selects |
|
||||
|----------------------|---------------------------|-------------------------------------------------|
|
||||
| `:checked` | `input:checked` | Checked checkboxes, radios and selected options |
|
||||
| `:default` | `input:default` | The default control in a form |
|
||||
| `:disabled` | `input:disabled` | Disabled controls |
|
||||
| `:enabled` | `input:enabled` | Controls that are not disabled |
|
||||
| `:required` | `input:required` | Controls carrying `required` |
|
||||
| `:optional` | `input:optional` | Controls without `required` |
|
||||
| `:valid` | `input:valid` | Controls whose value passes validation |
|
||||
| `:invalid` | `input:invalid` | Controls whose value fails validation |
|
||||
| `:user-valid` | `input:user-valid` | Valid, but only after the user has interacted |
|
||||
| `:user-invalid` | `input:user-invalid` | Invalid, but only after the user has interacted |
|
||||
| `:in-range` | `input:in-range` | Value inside `min`/`max` |
|
||||
| `:out-of-range` | `input:out-of-range` | Value outside `min`/`max` |
|
||||
| `:indeterminate` | `input:indeterminate` | Neither checked nor unchecked |
|
||||
| `:read-only` | `input:read-only` | Not editable by the user |
|
||||
| `:read-write` | `input:read-write` | Editable by the user |
|
||||
| `:placeholder-shown` | `input:placeholder-shown` | The placeholder is currently visible |
|
||||
|
||||
## 🧩 Functional Pseudo-Classes
|
||||
|
||||
| Selector | Example | Selects |
|
||||
|-------------------------|--------------------------------|-------------------------------------------------------------------------|
|
||||
| `:is()` | `:is(h1, h2, h3) > code` | Any of the listed selectors — takes the highest specificity of the list |
|
||||
| `:where()` | `:where(h1, h2) a` | Same matching as `:is()`, but contributes **zero** specificity |
|
||||
| `:not()` | `li:not(.done, .hidden)` | Elements that match none of the listed selectors |
|
||||
| `:has()` | `article:has(> img)` | A parent that contains a match — the "parent selector" |
|
||||
| `:lang()` | `p:lang(it)` | Elements in a given language |
|
||||
| `:nth-child(An+B of S)` | `li:nth-child(2n of .visible)` | Counts only the siblings matching `S` |
|
||||
|
||||
## ✨ Pseudo-Elements
|
||||
|
||||
Pseudo-elements use `::` (double colon) and target a *part* of an element rather than the element itself.
|
||||
|
||||
| Selector | Example | Targets |
|
||||
|--------------------------|-------------------------------|-------------------------------------------------|
|
||||
| `::before` | `p::before` | Generated content in front of the content |
|
||||
| `::after` | `p::after` | Generated content after the content |
|
||||
| `::first-letter` | `p::first-letter` | The first letter |
|
||||
| `::first-line` | `p::first-line` | The first rendered line |
|
||||
| `::marker` | `li::marker` | The bullet or number of a list item |
|
||||
| `::selection` | `::selection` | The part the user has selected |
|
||||
| `::placeholder` | `input::placeholder` | The placeholder text |
|
||||
| `::backdrop` | `dialog::backdrop` | The layer behind a modal or full-screen element |
|
||||
| `::file-selector-button` | `input::file-selector-button` | The button of a file input |
|
||||
|
||||
> ℹ️ `::before` and `::after` need a `content` property — even `content: ""` — before they render.
|
||||
|
||||
## 📊 Specificity
|
||||
|
||||
Specificity is compared as three numbers, **ID – class – type**, highest wins; a later rule only wins against an equal score.
|
||||
|
||||
| Selector kind | Weight | Example |
|
||||
|--------------------------------|----------------------------|-----------------------------|
|
||||
| Inline `style` attribute | overrides everything below | `style="…"` |
|
||||
| ID | `1-0-0` | `#header` |
|
||||
| Class, attribute, pseudo-class | `0-1-0` | `.card`, `[href]`, `:hover` |
|
||||
| Element, pseudo-element | `0-0-1` | `p`, `::before` |
|
||||
| `*`, combinators, `:where()` | `0-0-0` | `*`, `>`, `:where(.a)` |
|
||||
|
||||
```css
|
||||
#nav .item a /* 1-1-1 */
|
||||
.menu .item a /* 0-2-1 — loses to the line above */
|
||||
:where(#nav) a /* 0-0-1 — :where() zeroes out what is inside it */
|
||||
```
|
||||
|
||||
> 💡 Reach for `:where()` in library or reset styles so consumers can override them without an `!important` arms race.
|
||||
|
||||
## 📚 Resources
|
||||
|
||||
- [MDN — CSS selectors](https://developer.mozilla.org/en-US/docs/Web/CSS/CSS_selectors)
|
||||
- [MDN — specificity](https://developer.mozilla.org/en-US/docs/Web/CSS/Specificity)
|
||||
- [Selectors Level 4 specification](https://www.w3.org/TR/selectors-4/)
|
||||
@@ -1,463 +1,495 @@
|
||||
Docker Compose is a powerful tool for defining and running multi-container Docker applications. It uses a YAML file (`compose.yaml`) to configure application services, networks, volumes, and more. This file allows developers to declaratively describe infrastructure and dependencies, making it easier to manage complex environments.
|
||||
**Docker Compose** describes a multi-container application in one YAML file and runs it with a single command. The file declares *services* (containers), plus the *networks*, *volumes*, *secrets* and *configs* they use.
|
||||
|
||||
Whether you're building a local dev stack or deploying to production, Compose simplifies orchestration and keeps your configuration readable and version-controlled.
|
||||
> ℹ️ The `version:` key at the top is obsolete — the Compose Spec ignores it. Start the file with `services:`.
|
||||
|
||||
## 📁 File Name
|
||||
```yaml
|
||||
compose.yaml
|
||||
## 📁 File Names & Precedence
|
||||
|
||||
| File | Role |
|
||||
|-------------------------|--------------------------------------------------------------|
|
||||
| `compose.yaml` | The preferred name (`compose.yml` also works) |
|
||||
| `docker-compose.yaml` | Legacy name, still supported |
|
||||
| `compose.override.yaml` | Merged on top of the base file automatically |
|
||||
| `-f a.yaml -f b.yaml` | Explicit list; later files override earlier ones |
|
||||
| `.env` | Variables for interpolation, read from the project directory |
|
||||
|
||||
```bash
|
||||
# check what Compose actually resolved, after merges and interpolation
|
||||
docker compose config
|
||||
|
||||
# use a specific set of files
|
||||
docker compose -f compose.yaml -f compose.prod.yaml up -d
|
||||
```
|
||||
|
||||
### ✅ YAML Formatting Rules
|
||||
### YAML rules that bite
|
||||
|
||||
- Use **2 spaces** for indentation (not tabs)
|
||||
- Keys and values are **case-sensitive**
|
||||
- Lists use `-` for each item
|
||||
- Strings with special characters should be quoted
|
||||
- Environment variables can be defined inline or via `.env` files
|
||||
- Two spaces per indent level, never tabs
|
||||
- Keys and values are case-sensitive
|
||||
- Lists use `-`; quote any string containing `:`, `#`, `{`, `}` or a leading `*`
|
||||
- `yes`/`no`/`on`/`off` are booleans — quote them if you mean the words
|
||||
|
||||
## 🧱 Basic Structure
|
||||
|
||||
```yaml
|
||||
services:
|
||||
service_name:
|
||||
image: image_name:tag
|
||||
build:
|
||||
context: .
|
||||
dockerfile: Dockerfile
|
||||
ports:
|
||||
- "host_port:container_port"
|
||||
volumes:
|
||||
- ./host_path:/container_path
|
||||
environment:
|
||||
- VAR_NAME=value
|
||||
depends_on:
|
||||
- other_service
|
||||
networks:
|
||||
- custom_network
|
||||
networks:
|
||||
custom_network:
|
||||
driver: bridge
|
||||
volumes:
|
||||
custom_volume:
|
||||
```
|
||||
|
||||
## ⚙️ Services
|
||||
|
||||
Each service defines a container.
|
||||
|
||||
### Common Service Options
|
||||
## 🧱 Minimal Structure
|
||||
|
||||
```yaml
|
||||
services:
|
||||
web:
|
||||
image: nginx:latest
|
||||
build:
|
||||
context: ./app
|
||||
dockerfile: Dockerfile
|
||||
command: ["nginx", "-g", "daemon off;"]
|
||||
container_name: custom_name
|
||||
image: nginx:1.27
|
||||
ports:
|
||||
- "8080:80"
|
||||
depends_on:
|
||||
- api
|
||||
|
||||
api:
|
||||
build: .
|
||||
environment:
|
||||
DATABASE_URL: postgres://db:5432/app
|
||||
networks:
|
||||
- backend
|
||||
|
||||
networks:
|
||||
backend:
|
||||
|
||||
volumes:
|
||||
db-data:
|
||||
```
|
||||
|
||||
## ⚙️ Service Options
|
||||
|
||||
```yaml
|
||||
services:
|
||||
app:
|
||||
# where the image comes from
|
||||
image: myapp:1.2
|
||||
build: .
|
||||
pull_policy: always
|
||||
|
||||
# identity and lifecycle
|
||||
container_name: myapp
|
||||
hostname: app
|
||||
restart: unless-stopped
|
||||
init: true
|
||||
stop_grace_period: 30s
|
||||
|
||||
# what it runs
|
||||
entrypoint: ["/entrypoint.sh"]
|
||||
command: ["node", "server.js"]
|
||||
working_dir: /app
|
||||
user: "1000:1000"
|
||||
|
||||
# configuration
|
||||
environment:
|
||||
NODE_ENV: production
|
||||
API_KEY: ${API_KEY}
|
||||
env_file:
|
||||
- .env
|
||||
- .env.production
|
||||
|
||||
# connectivity
|
||||
ports:
|
||||
- "8080:80"
|
||||
expose:
|
||||
- "80"
|
||||
environment:
|
||||
- DEBUG=true
|
||||
env_file:
|
||||
- .env
|
||||
volumes:
|
||||
- ./data:/data
|
||||
restart: always
|
||||
depends_on:
|
||||
- db
|
||||
- "9000"
|
||||
networks:
|
||||
- frontend
|
||||
extra_hosts:
|
||||
- "host.docker.internal:host-gateway"
|
||||
dns:
|
||||
- 1.1.1.1
|
||||
|
||||
# storage
|
||||
volumes:
|
||||
- db-data:/var/lib/data
|
||||
- ./src:/app/src:ro
|
||||
tmpfs:
|
||||
- /tmp
|
||||
|
||||
# ordering and health
|
||||
depends_on:
|
||||
db:
|
||||
condition: service_healthy
|
||||
healthcheck:
|
||||
test: ["CMD", "curl", "-f", "http://localhost"]
|
||||
test: ["CMD", "curl", "-f", "http://localhost/health"]
|
||||
interval: 30s
|
||||
timeout: 10s
|
||||
retries: 5
|
||||
timeout: 5s
|
||||
retries: 3
|
||||
start_period: 20s
|
||||
|
||||
# host access and limits
|
||||
cap_add:
|
||||
- SYS_PTRACE
|
||||
devices:
|
||||
- /dev/dri:/dev/dri
|
||||
ulimits:
|
||||
nofile: 65535
|
||||
deploy:
|
||||
resources:
|
||||
limits:
|
||||
cpus: "1.5"
|
||||
memory: 512M
|
||||
|
||||
# bookkeeping
|
||||
labels:
|
||||
com.example.team: platform
|
||||
logging:
|
||||
driver: json-file
|
||||
options:
|
||||
max-size: "10m"
|
||||
max-file: "3"
|
||||
```
|
||||
|
||||
## 🏗️ Build Options
|
||||
## 🏗 Build Options
|
||||
|
||||
```yaml
|
||||
build:
|
||||
context: ./dir
|
||||
dockerfile: Dockerfile
|
||||
args:
|
||||
build_arg: value
|
||||
target: build-stage
|
||||
services:
|
||||
app:
|
||||
build:
|
||||
context: .
|
||||
dockerfile: docker/Dockerfile
|
||||
target: production
|
||||
args:
|
||||
NODE_VERSION: "22"
|
||||
cache_from:
|
||||
- myapp:cache
|
||||
secrets:
|
||||
- npmrc
|
||||
platforms:
|
||||
- linux/amd64
|
||||
- linux/arm64
|
||||
image: myapp:1.2 # the name given to the built image
|
||||
```
|
||||
|
||||
- **`build:`** Tells Compose how to build the image.
|
||||
- `context:` is the directory containing the Dockerfile and source code.
|
||||
- `dockerfile:` lets you specify a custom Dockerfile name or path.
|
||||
|
||||
## 📦 Volumes
|
||||
|
||||
```yaml
|
||||
services:
|
||||
db:
|
||||
volumes:
|
||||
# named volume — managed by Docker, survives 'down'
|
||||
- db-data:/var/lib/postgresql/data
|
||||
# bind mount — a host path, read-only
|
||||
- ./config:/etc/app:ro
|
||||
# anonymous volume — keeps node_modules out of the bind mount above
|
||||
- /app/node_modules
|
||||
# long syntax
|
||||
- type: bind
|
||||
source: ./src
|
||||
target: /app/src
|
||||
read_only: true
|
||||
|
||||
volumes:
|
||||
data_volume:
|
||||
driver: local
|
||||
db-data:
|
||||
shared:
|
||||
external: true # created outside Compose
|
||||
nfs-data:
|
||||
driver_opts:
|
||||
type: none
|
||||
device: /path/on/host
|
||||
o: bind
|
||||
type: nfs
|
||||
o: addr=10.0.0.10,rw
|
||||
device: ":/exports/data"
|
||||
```
|
||||
|
||||
### Mounting Volumes
|
||||
|
||||
```yaml
|
||||
volumes:
|
||||
- data_volume:/app/data
|
||||
- ./local:/container/path
|
||||
```
|
||||
|
||||
- **`volumes:`** Mounts host directories or named volumes into the container.
|
||||
- `./src:/app/src` mounts the local `src` folder into the container at `/app/src`.
|
||||
|
||||
## 🌐 Networks
|
||||
|
||||
```yaml
|
||||
services:
|
||||
web:
|
||||
networks:
|
||||
- frontend
|
||||
api:
|
||||
networks:
|
||||
frontend:
|
||||
aliases:
|
||||
- api.internal
|
||||
backend:
|
||||
|
||||
networks:
|
||||
frontend:
|
||||
driver: bridge
|
||||
backend:
|
||||
driver: overlay
|
||||
internal: true # no outbound access
|
||||
existing:
|
||||
external: true
|
||||
name: some-other-network
|
||||
```
|
||||
|
||||
- **`networks:`** Connects the service to one or more custom networks. Enables service discovery and isolation.
|
||||
> 💡 Compose creates a default network per project and every service joins it, so containers already reach each other by service name. Declare networks when you want to *separate* things.
|
||||
|
||||
### Assigning Networks to Services
|
||||
## 🔀 Ports
|
||||
|
||||
```yaml
|
||||
services:
|
||||
app:
|
||||
networks:
|
||||
- frontend
|
||||
- backend
|
||||
```
|
||||
|
||||
## 🌐 Ports
|
||||
|
||||
```yaml
|
||||
web:
|
||||
ports:
|
||||
- "3000:3000"
|
||||
```
|
||||
- **`ports:`** Maps container ports to host ports. Format is `"host:container"`. Useful for exposing services to your local machine.
|
||||
|
||||
## 🔐 Secrets (Docker Swarm only)
|
||||
|
||||
```yaml
|
||||
secrets:
|
||||
db_password:
|
||||
file: ./db_password.txt
|
||||
|
||||
services:
|
||||
db:
|
||||
secrets:
|
||||
- db_password
|
||||
```
|
||||
|
||||
## 🔑 Configs (Docker Swarm only)
|
||||
|
||||
```yaml
|
||||
configs:
|
||||
my_config:
|
||||
file: ./config.txt
|
||||
|
||||
services:
|
||||
app:
|
||||
configs:
|
||||
- source: my_config
|
||||
target: /etc/config.txt
|
||||
```
|
||||
|
||||
## 🧪 Healthcheck
|
||||
|
||||
```yaml
|
||||
healthcheck:
|
||||
test: ["CMD", "curl", "-f", "http://localhost"]
|
||||
interval: 30s
|
||||
timeout: 10s
|
||||
retries: 3
|
||||
start_period: 5s
|
||||
```
|
||||
|
||||
- **`healthcheck:`** Defines how Docker checks if the container is healthy.
|
||||
- `test:` is the command to run
|
||||
- `interval:` how often to run the check
|
||||
- `timeout:` how long to wait for a response
|
||||
- `retries:` how many failures before marking unhealthy
|
||||
|
||||
## 🔄 Restart Policies
|
||||
|
||||
```yaml
|
||||
restart: no # Never restart
|
||||
restart: always # Always restart
|
||||
restart: on-failure # Restart on failure
|
||||
restart: unless-stopped
|
||||
- "8080:80" # host:container
|
||||
- "127.0.0.1:8080:80" # bind to one interface only
|
||||
- "8080-8090:80-90" # a range
|
||||
- "80" # random host port
|
||||
- target: 80 # long syntax
|
||||
published: "8080"
|
||||
protocol: tcp
|
||||
mode: host
|
||||
```
|
||||
|
||||
## 🧬 Environment Variables
|
||||
|
||||
```yaml
|
||||
environment:
|
||||
- VAR1=value1
|
||||
- VAR2=value2
|
||||
env_file:
|
||||
- .env
|
||||
services:
|
||||
app:
|
||||
environment:
|
||||
# map form (preferred)
|
||||
LOG_LEVEL: debug
|
||||
# take the value from the shell or .env
|
||||
API_KEY: ${API_KEY}
|
||||
# with a default, and a hard requirement
|
||||
PORT: ${PORT:-3000}
|
||||
DB_URL: ${DB_URL:?DB_URL must be set}
|
||||
env_file:
|
||||
- path: .env.production
|
||||
required: false
|
||||
```
|
||||
|
||||
- **`environment:`** Sets environment variables inside the container. Useful for configuration.
|
||||
- **`env_file:`** Loads environment variables from a file. Keeps secrets and config separate from the Compose file.
|
||||
|
||||
## Command
|
||||
|
||||
```yaml
|
||||
command: npm start
|
||||
```
|
||||
- **`command:`** Overrides the default command defined in the Dockerfile. Useful for customizing container behavior.
|
||||
|
||||
## Dependencies
|
||||
|
||||
```yaml
|
||||
depends_on:
|
||||
- db
|
||||
```
|
||||
- **`depends_on:`** Specifies service startup order. In Compose, this does not wait for the service to be "ready"—just started.
|
||||
|
||||
## 🧹 Clean Up
|
||||
|
||||
```bash
|
||||
docker compose down # Stop and remove containers, networks, volumes
|
||||
docker compose down -v # Also remove named volumes
|
||||
# variables come from the shell, .env, and --env-file, in that order of precedence
|
||||
API_KEY=abc docker compose up -d
|
||||
docker compose --env-file .env.staging up -d
|
||||
```
|
||||
|
||||
## 🚀 Commands
|
||||
## 🩺 Healthchecks & Startup Order
|
||||
|
||||
| Command | Description |
|
||||
|--------|-------------|
|
||||
| `docker compose up` | Start services |
|
||||
| `docker compose up -d` | Start in detached mode |
|
||||
| `docker compose down` | Stop and remove services |
|
||||
| `docker compose build` | Build images |
|
||||
| `docker compose ps` | List containers |
|
||||
| `docker compose logs` | View logs |
|
||||
| `docker compose exec <service> <cmd>` | Run command in container |
|
||||
| `docker compose config` | Validate and view config |
|
||||
|
||||
|
||||
## 🧠 Mounting GPU / iGPU
|
||||
|
||||
Docker Compose supports GPU access via the `device_requests` field (Compose v3.8+ and Docker 19.03+).
|
||||
|
||||
### ✅ NVIDIA GPU Example
|
||||
`depends_on` on its own only waits for the container to *start*. Wait for it to be **healthy** instead:
|
||||
|
||||
```yaml
|
||||
services:
|
||||
gpu-app:
|
||||
image: nvidia/cuda:11.0-base
|
||||
db:
|
||||
image: postgres:16
|
||||
healthcheck:
|
||||
test: ["CMD-SHELL", "pg_isready -U postgres"]
|
||||
interval: 10s
|
||||
timeout: 5s
|
||||
retries: 5
|
||||
start_period: 30s
|
||||
|
||||
api:
|
||||
image: myapi
|
||||
depends_on:
|
||||
db:
|
||||
condition: service_healthy # or service_started, service_completed_successfully
|
||||
```
|
||||
|
||||
## 🔄 Restart Policies
|
||||
|
||||
| Policy | Behaviour |
|
||||
|------------------|---------------------------------------------------------------|
|
||||
| `no` | Never restart (the default) |
|
||||
| `on-failure` | Restart only on a non-zero exit — `on-failure:5` to cap tries |
|
||||
| `always` | Always restart, including after a daemon restart |
|
||||
| `unless-stopped` | Like `always`, but stays stopped if you stopped it yourself |
|
||||
|
||||
## 🧩 Profiles
|
||||
|
||||
Profiles keep optional services out of the way until you ask for them.
|
||||
|
||||
```yaml
|
||||
services:
|
||||
web:
|
||||
image: nginx # no profile: always started
|
||||
|
||||
debug:
|
||||
image: busybox
|
||||
command: top
|
||||
profiles: [debug]
|
||||
|
||||
seed:
|
||||
image: myapp
|
||||
command: npm run seed
|
||||
profiles: [tools]
|
||||
```
|
||||
|
||||
```bash
|
||||
# start the default services plus one profile
|
||||
docker compose --profile debug up -d
|
||||
|
||||
# several at once
|
||||
docker compose --profile debug --profile tools up -d
|
||||
|
||||
# COMPOSE_PROFILES works too
|
||||
COMPOSE_PROFILES=debug,tools docker compose up -d
|
||||
```
|
||||
|
||||
## 👀 Watch Mode
|
||||
|
||||
`docker compose watch` syncs or rebuilds automatically as you edit — a dev loop without bind-mount surprises.
|
||||
|
||||
```yaml
|
||||
services:
|
||||
web:
|
||||
build: .
|
||||
develop:
|
||||
watch:
|
||||
- action: sync
|
||||
path: ./src
|
||||
target: /app/src
|
||||
- action: rebuild
|
||||
path: package.json
|
||||
- action: sync+restart
|
||||
path: ./config
|
||||
target: /etc/app
|
||||
```
|
||||
|
||||
```bash
|
||||
docker compose watch
|
||||
```
|
||||
|
||||
## ♻️ Reuse: include, extends & anchors
|
||||
|
||||
```yaml
|
||||
# pull in another compose file as if it were written here
|
||||
include:
|
||||
- path: ./monitoring/compose.yaml
|
||||
|
||||
services:
|
||||
# inherit another service's definition
|
||||
worker:
|
||||
extends:
|
||||
file: common.yaml
|
||||
service: base-app
|
||||
command: ["node", "worker.js"]
|
||||
|
||||
# YAML anchors for repeated blocks
|
||||
x-logging: &default-logging
|
||||
driver: json-file
|
||||
options:
|
||||
max-size: "10m"
|
||||
|
||||
services:
|
||||
api:
|
||||
logging: *default-logging
|
||||
web:
|
||||
logging: *default-logging
|
||||
```
|
||||
|
||||
## 🔐 Secrets & Configs
|
||||
|
||||
File-based secrets work in plain Compose: each one is mounted read-only at `/run/secrets/<name>`. External secrets and configs require Swarm.
|
||||
|
||||
```yaml
|
||||
services:
|
||||
db:
|
||||
image: postgres:16
|
||||
environment:
|
||||
POSTGRES_PASSWORD_FILE: /run/secrets/db_password
|
||||
secrets:
|
||||
- db_password
|
||||
configs:
|
||||
- source: pg_conf
|
||||
target: /etc/postgresql/postgresql.conf
|
||||
|
||||
secrets:
|
||||
db_password:
|
||||
file: ./db_password.txt
|
||||
api_token:
|
||||
external: true # Swarm only
|
||||
|
||||
configs:
|
||||
pg_conf:
|
||||
file: ./postgresql.conf
|
||||
```
|
||||
|
||||
## 🖥 GPU & Device Access
|
||||
|
||||
```yaml
|
||||
services:
|
||||
# NVIDIA — needs the NVIDIA Container Toolkit on the host
|
||||
cuda-app:
|
||||
image: nvidia/cuda:12.4.1-base-ubuntu22.04
|
||||
deploy:
|
||||
resources:
|
||||
reservations:
|
||||
devices:
|
||||
- driver: nvidia
|
||||
count: all
|
||||
count: all # or: device_ids: ["0"]
|
||||
capabilities: [gpu]
|
||||
```
|
||||
|
||||
### ✅ Intel iGPU (via VAAPI or OpenCL)
|
||||
|
||||
```yaml
|
||||
services:
|
||||
# Intel iGPU — VAAPI/OpenCL through the render device
|
||||
igpu-app:
|
||||
image: intel/openvino
|
||||
devices:
|
||||
- /dev/dri:/dev/dri
|
||||
group_add:
|
||||
- video
|
||||
```
|
||||
|
||||
> 🔧 Make sure your host has the necessary drivers and runtime (e.g., NVIDIA Container Toolkit or Intel Media SDK).
|
||||
|
||||
## 🔄 Compose vs Swarm YAML Differences
|
||||
|
||||
| Feature | Docker Compose (`compose.yaml`) | Docker Swarm (`stack.yml`) |
|
||||
|--------------------|----------------------------------------|-----------------------------|
|
||||
| `restart` | ✅ Supported | ❌ Not supported |
|
||||
| `depends_on` | ✅ Supported | ❌ Not supported |
|
||||
| `deploy` | ❌ Ignored | ✅ Required for replicas |
|
||||
| `build` | ✅ Supported | ❌ Not supported |
|
||||
| `volumes` (bind) | ✅ Supported | ✅ Supported |
|
||||
| `configs`/`secrets`| ❌ Ignored | ✅ Supported |
|
||||
| `healthcheck` | ✅ Supported | ✅ Supported |
|
||||
|
||||
> 🧠 **Tip:** Use `compose.yaml` for local development and `stack.yml` for Swarm deployments.
|
||||
|
||||
## 🧬 Profiles (Compose v3.9+)
|
||||
|
||||
Profiles allow conditional inclusion of services based on the active profile. This is useful for separating dev/test/staging environments.
|
||||
|
||||
### ✅ Defining Profiles
|
||||
|
||||
```yaml
|
||||
services:
|
||||
web:
|
||||
image: nginx
|
||||
profiles:
|
||||
- default
|
||||
|
||||
debug:
|
||||
image: busybox
|
||||
command: top
|
||||
profiles:
|
||||
- debug
|
||||
```
|
||||
|
||||
### ✅ Activating Profiles
|
||||
## 🚀 Everyday Commands
|
||||
|
||||
```bash
|
||||
docker compose --profile debug up
|
||||
# start everything in the background
|
||||
docker compose up -d
|
||||
|
||||
# rebuild the images first
|
||||
docker compose up -d --build
|
||||
|
||||
# start one service and what it depends on
|
||||
docker compose up -d <service>
|
||||
|
||||
# recreate containers even if nothing changed
|
||||
docker compose up -d --force-recreate
|
||||
|
||||
# stop and remove containers and networks
|
||||
docker compose down
|
||||
|
||||
# also remove the named volumes and local images
|
||||
docker compose down -v --rmi local
|
||||
|
||||
# what is running, including health
|
||||
docker compose ps
|
||||
|
||||
# follow the logs
|
||||
docker compose logs -f
|
||||
docker compose logs -f --tail 100 <service>
|
||||
|
||||
# a shell inside a running service
|
||||
docker compose exec <service> sh
|
||||
|
||||
# a one-off container for a task
|
||||
docker compose run --rm <service> npm test
|
||||
|
||||
# rebuild, pull, restart
|
||||
docker compose build --no-cache
|
||||
docker compose pull
|
||||
docker compose restart <service>
|
||||
|
||||
# scale a stateless service
|
||||
docker compose up -d --scale worker=3
|
||||
|
||||
# validate and print the effective configuration
|
||||
docker compose config
|
||||
docker compose config --services
|
||||
|
||||
# processes and resource usage
|
||||
docker compose top
|
||||
docker compose stats
|
||||
```
|
||||
|
||||
### ✅ Notes
|
||||
## ⚔️ Compose vs Swarm Stacks
|
||||
|
||||
- Services without a `profiles` key are always included.
|
||||
- Multiple profiles can be activated simultaneously.
|
||||
- Useful for feature toggles, optional services, or environment-specific setups.
|
||||
Both read a Compose file, but `docker stack deploy` honours a different subset of it.
|
||||
|
||||
## ⚔️ YAML Differences: Docker Compose vs Docker Swarm Mode
|
||||
| Key | `docker compose` | `docker stack deploy` |
|
||||
|----------------------------------|-----------------------------|-------------------------------------------------|
|
||||
| `build:` | ✅ builds locally | ❌ ignored — push the image to a registry first |
|
||||
| `restart:` | ✅ | ❌ use `deploy.restart_policy` |
|
||||
| `depends_on:` | ✅ | ❌ ignored — rely on health checks and retries |
|
||||
| `profiles:` | ✅ | ❌ |
|
||||
| `develop.watch:` | ✅ | ❌ |
|
||||
| `deploy.replicas/placement` | ❌ ignored | ✅ this is how you scale |
|
||||
| `configs:` / external `secrets:` | ❌ external ones need Swarm | ✅ |
|
||||
| `healthcheck:` | ✅ | ✅ also drives rescheduling |
|
||||
| `volumes:` / `networks:` | ✅ | ✅ (overlay networks in Swarm) |
|
||||
|
||||
Docker Compose and Docker Swarm both use YAML files to define services, but they serve different purposes and support different features. Compose is optimized for local development and testing, while Swarm is designed for production-grade orchestration across clusters.
|
||||
|
||||
### 🧭 Purpose
|
||||
|
||||
| Mode | Use Case |
|
||||
|-------------|-----------------------------------|
|
||||
| Compose | Local development, testing |
|
||||
| Swarm | Cluster deployment, scaling |
|
||||
|
||||
### 🧩 Key Differences in YAML Structure
|
||||
|
||||
| Feature | Compose (`compose.yaml`) | Swarm (`stack.yml`) |
|
||||
|----------------------|-------------------------------|----------------------|
|
||||
| `build:` | ✅ Supported | ❌ Ignored |
|
||||
| `restart:` | ✅ Supported | ❌ Ignored |
|
||||
| `depends_on:` | ✅ Supported | ❌ Ignored |
|
||||
| `deploy:` | ❌ Ignored | ✅ Required for scaling, placement |
|
||||
| `configs:` | ❌ Ignored | ✅ Supported |
|
||||
| `secrets:` | ❌ Ignored | ✅ Supported |
|
||||
| `healthcheck:` | ✅ Supported | ✅ Supported |
|
||||
| `volumes:` (bind) | ✅ Supported | ✅ Supported |
|
||||
| `networks:` | ✅ Supported | ✅ Supported |
|
||||
| `profiles:` | ✅ Supported (v3.9+) | ❌ Not supported |
|
||||
|
||||
### 🔧 Compose-Only Features
|
||||
|
||||
These features are useful for local development but are ignored in Swarm:
|
||||
|
||||
#### `build:`
|
||||
```yaml
|
||||
services:
|
||||
app:
|
||||
build:
|
||||
context: .
|
||||
dockerfile: Dockerfile
|
||||
```
|
||||
- Compose builds images locally.
|
||||
- Swarm requires pre-built images pushed to a registry.
|
||||
|
||||
#### `restart:`
|
||||
```yaml
|
||||
restart: unless-stopped
|
||||
```
|
||||
- Compose uses this to auto-restart containers.
|
||||
- Swarm uses `deploy.restart_policy` instead.
|
||||
|
||||
#### `depends_on:`
|
||||
```yaml
|
||||
depends_on:
|
||||
- db
|
||||
```
|
||||
- Compose starts services in order.
|
||||
- Swarm ignores this; use healthchecks and wait-for-it scripts.
|
||||
|
||||
### 🛡️ Swarm-Only Features
|
||||
|
||||
These features are exclusive to Swarm and ignored by Compose:
|
||||
|
||||
#### `deploy:`
|
||||
```yaml
|
||||
services:
|
||||
app:
|
||||
deploy:
|
||||
replicas: 3
|
||||
placement:
|
||||
constraints:
|
||||
- node.role == manager
|
||||
restart_policy:
|
||||
condition: on-failure
|
||||
```
|
||||
- Controls scaling, placement, and restart behavior in a cluster.
|
||||
|
||||
#### `configs:` and `secrets:`
|
||||
```yaml
|
||||
configs:
|
||||
app_config:
|
||||
file: ./config.yml
|
||||
|
||||
secrets:
|
||||
db_password:
|
||||
file: ./password.txt
|
||||
```
|
||||
- Used to securely distribute configuration and secrets across nodes.
|
||||
|
||||
#### `placement:` (inside `deploy`)
|
||||
```yaml
|
||||
placement:
|
||||
constraints:
|
||||
- node.labels.env == production
|
||||
```
|
||||
- Assigns services to specific nodes based on labels.
|
||||
|
||||
### 🧪 Healthchecks (Supported in Both)
|
||||
|
||||
```yaml
|
||||
healthcheck:
|
||||
test: ["CMD", "curl", "-f", "http://localhost"]
|
||||
interval: 30s
|
||||
timeout: 10s
|
||||
retries: 3
|
||||
```
|
||||
- Works in both Compose and Swarm.
|
||||
- In Swarm, health status can influence service rescheduling.
|
||||
|
||||
### 📦 Volume Differences
|
||||
|
||||
| Type | Compose | Swarm |
|
||||
|-------------|---------|-------|
|
||||
| Bind mount | ✅ | ✅ |
|
||||
| Named volume| ✅ | ✅ |
|
||||
| External volume | ✅ | ✅ |
|
||||
| Volume driver options | ✅ | ✅ |
|
||||
|
||||
Swarm requires external volumes to be pre-created on all nodes.
|
||||
|
||||
### 🧠 Summary
|
||||
|
||||
| Feature Category | Compose | Swarm |
|
||||
|----------------------|---------|-------|
|
||||
| Local builds | ✅ | ❌ |
|
||||
| Cluster scaling | ❌ | ✅ |
|
||||
| Secrets/configs | ❌ | ✅ |
|
||||
| Profiles | ✅ | ❌ |
|
||||
| Restart policies | ✅ | ✅ (via `deploy`) |
|
||||
| Service dependencies | ✅ | ❌ |
|
||||
|
||||
> 🧭 **Tip:** Use `compose.yaml` for development and `stack.yml` for Swarm. You can split your configuration into multiple files or use tools like `kompose` to convert Kubernetes manifests.
|
||||
> 🧭 Keep `compose.yaml` for local development and a separate `stack.yaml` for Swarm, or put the Swarm-only bits in an override file.
|
||||
|
||||
## 📚 Resources
|
||||
|
||||
- [Compose File Reference](https://docs.docker.com/compose/compose-file/)
|
||||
- [Docker CLI Reference](https://docs.docker.com/engine/reference/commandline/compose/)
|
||||
- [NVIDIA GPU Support](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/install-guide.html)
|
||||
- [Intel GPU Support](https://github.com/intel/media-driver)
|
||||
- [Compose file reference](https://docs.docker.com/reference/compose-file/)
|
||||
- [`docker compose` CLI reference](https://docs.docker.com/reference/cli/docker/compose/)
|
||||
- [Compose watch](https://docs.docker.com/compose/how-tos/file-watch/)
|
||||
- [Environment variables and interpolation](https://docs.docker.com/compose/how-tos/environment-variables/)
|
||||
- [NVIDIA Container Toolkit](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/install-guide.html)
|
||||
+413
-356
@@ -1,482 +1,539 @@
|
||||
Docker provides the ability to package and run an application in a loosely isolated environment called a container. The isolation and security allows you to run many containers simultaneously on a given host. Containers are lightweight and contain everything needed to run the application, so you do not need to rely on what is currently installed on the host. You can easily share containers while you work, and be sure that everyone you share with gets the same container that works in the same way.
|
||||
**Docker** packages an application and everything it needs into a *container* — a lightweight, isolated process that runs the same way on any host. Images are the immutable blueprints; containers are running instances of them.
|
||||
|
||||
## Installation & Resources
|
||||
- **Image** — a read-only template built from a `Dockerfile`
|
||||
- **Container** — a running (or stopped) instance of an image
|
||||
- **Volume** — storage that outlives the container that wrote it
|
||||
- **Network** — a virtual network where containers reach each other by name
|
||||
- **Registry** — where images are pushed and pulled (Docker Hub, GHCR, ECR, …)
|
||||
|
||||
- Docker Desktop (Mac, Linux, Windows): https://docs.docker.com/desktop
|
||||
- Example projects using Docker: https://github.com/docker/awesome-compose
|
||||
- Official documentation: https://docs.docker.com
|
||||
> 💡 Every subcommand has its own help: `docker run --help`, `docker compose up --help`.
|
||||
|
||||
## General Commands
|
||||
## ⚡ Most-Used Commands
|
||||
|
||||
| Command | What it does |
|
||||
|----------------------------------|--------------------------------------------|
|
||||
| `docker ps -a` | List all containers, running or not |
|
||||
| `docker images` | List local images |
|
||||
| `docker run -it --rm <image> sh` | Throwaway shell in a fresh container |
|
||||
| `docker exec -it <container> sh` | Shell inside a *running* container |
|
||||
| `docker logs -f <container>` | Follow a container's output |
|
||||
| `docker build -t <name>:<tag> .` | Build an image from the local `Dockerfile` |
|
||||
| `docker compose up -d --build` | Rebuild and start a whole stack |
|
||||
| `docker system df` | See what is eating your disk |
|
||||
|
||||
## 🧭 General
|
||||
|
||||
Start the Docker daemon:
|
||||
```bash
|
||||
# start the Docker daemon (usually done by the service manager)
|
||||
dockerd
|
||||
```
|
||||
|
||||
Get help with Docker (works on all subcommands, e.g. `docker run --help`):
|
||||
```bash
|
||||
# top-level help; works on every subcommand
|
||||
docker --help
|
||||
```
|
||||
|
||||
Display system-wide information:
|
||||
```bash
|
||||
# system-wide information: storage driver, resources, warnings
|
||||
docker info
|
||||
```
|
||||
|
||||
Show Docker version (client and server):
|
||||
```bash
|
||||
# client and server versions
|
||||
docker version
|
||||
|
||||
# list contexts (local socket, remote hosts, ...)
|
||||
docker context ls
|
||||
|
||||
# target another daemon with the same commands
|
||||
docker context use <ctx>
|
||||
|
||||
# live stream of daemon events
|
||||
docker events
|
||||
```
|
||||
|
||||
## Images
|
||||
## 📦 Images
|
||||
|
||||
Docker images are a lightweight, standalone, executable package of software that includes everything needed to run an application: code, runtime, system tools, system libraries, and settings.
|
||||
An image is a stack of read-only layers: code, runtime, libraries and settings baked together.
|
||||
|
||||
Build an image from a Dockerfile in the current directory:
|
||||
```bash
|
||||
docker build -t <image_name> .
|
||||
```
|
||||
# build from the Dockerfile in this directory
|
||||
docker build -t <image> .
|
||||
|
||||
Build an image without using the cache:
|
||||
```bash
|
||||
docker build -t <image_name> . --no-cache
|
||||
```
|
||||
# build and tag, e.g. myapp:1.2
|
||||
docker build -t <image>:<tag> .
|
||||
|
||||
Build with a specific tag (e.g. `myapp:v1.2`):
|
||||
```bash
|
||||
docker build -t <image_name>:<tag> .
|
||||
```
|
||||
# rebuild every layer from scratch
|
||||
docker build -t <image> . --no-cache
|
||||
|
||||
List local images:
|
||||
```bash
|
||||
# use a Dockerfile somewhere else
|
||||
docker build -f <path>/Dockerfile -t <image> .
|
||||
|
||||
# pass a build argument
|
||||
docker build --build-arg KEY=value -t <image> .
|
||||
|
||||
# stop at a stage of a multi-stage build
|
||||
docker build --target <stage> -t <image> .
|
||||
|
||||
# build for another architecture
|
||||
docker build --platform linux/amd64 -t <image> .
|
||||
|
||||
# list local images
|
||||
docker images
|
||||
```
|
||||
|
||||
Tag an image for pushing to a registry:
|
||||
```bash
|
||||
docker tag <image> <username>/<image>:<tag>
|
||||
```
|
||||
# include intermediate layers
|
||||
docker images -a
|
||||
|
||||
Delete an image:
|
||||
```bash
|
||||
docker rmi <image_name>
|
||||
```
|
||||
# add a name for a registry
|
||||
docker tag <image> <user>/<image>:<tag>
|
||||
|
||||
Remove dangling (unused) images:
|
||||
```bash
|
||||
# full metadata as JSON
|
||||
docker inspect <image>
|
||||
|
||||
# layers, sizes and the commands behind them
|
||||
docker history <image>
|
||||
|
||||
# delete an image
|
||||
docker rmi <image>
|
||||
|
||||
# force-delete an image containers still reference
|
||||
docker rmi -f <image>
|
||||
|
||||
# remove dangling images
|
||||
docker image prune
|
||||
```
|
||||
|
||||
Remove all unused images, not just dangling:
|
||||
```bash
|
||||
# remove every image no container uses
|
||||
docker image prune -a
|
||||
|
||||
# export an image to a tarball
|
||||
docker save -o <file>.tar <image>
|
||||
|
||||
# import an image from a tarball
|
||||
docker load -i <file>.tar
|
||||
```
|
||||
|
||||
Show the layers of an image:
|
||||
```bash
|
||||
docker history <image_name>
|
||||
```
|
||||
|
||||
## Docker Hub
|
||||
|
||||
Docker Hub is a service for finding and sharing container images. Learn more at https://hub.docker.com
|
||||
|
||||
Log in to Docker Hub:
|
||||
## 🐳 Registries & Docker Hub
|
||||
|
||||
[Docker Hub](https://hub.docker.com) is the default public registry; the same commands work against any other one.
|
||||
|
||||
```bash
|
||||
# log in to Docker Hub
|
||||
docker login -u <username>
|
||||
```
|
||||
|
||||
Log out:
|
||||
```bash
|
||||
# log in to a private registry (GHCR, ECR, ...)
|
||||
docker login <registry-url>
|
||||
|
||||
# drop the stored credentials
|
||||
docker logout
|
||||
|
||||
# search Docker Hub from the terminal
|
||||
docker search <term>
|
||||
|
||||
# pull the :latest tag
|
||||
docker pull <image>
|
||||
|
||||
# pull a specific tag
|
||||
docker pull <image>:<tag>
|
||||
|
||||
# pull for another architecture
|
||||
docker pull --platform linux/arm64 <image>
|
||||
|
||||
# publish an image
|
||||
docker push <user>/<image>:<tag>
|
||||
|
||||
# architectures available for a tag
|
||||
docker manifest inspect <image>:<tag>
|
||||
```
|
||||
|
||||
Search Docker Hub for an image:
|
||||
## 🚢 Running Containers
|
||||
|
||||
```bash
|
||||
docker search <image_name>
|
||||
# create and start a container
|
||||
docker run <image>
|
||||
|
||||
# give it a stable name
|
||||
docker run --name <container> <image>
|
||||
|
||||
# detached: run in the background
|
||||
docker run -d <image>
|
||||
|
||||
# interactive shell (sh on minimal images)
|
||||
docker run -it <image> bash
|
||||
|
||||
# delete the container as soon as it exits
|
||||
docker run --rm <image>
|
||||
|
||||
# publish a port, e.g. -p 8080:80
|
||||
docker run -p <host>:<container> <image>
|
||||
|
||||
# publish every EXPOSEd port on random ports
|
||||
docker run -P <image>
|
||||
|
||||
# set an environment variable
|
||||
docker run -e KEY=value <image>
|
||||
|
||||
# load environment variables from a file
|
||||
docker run --env-file ./.env <image>
|
||||
|
||||
# mount a named volume
|
||||
docker run -v <volume>:/data <image>
|
||||
|
||||
# bind-mount the current directory
|
||||
docker run -v $(pwd):/app <image>
|
||||
|
||||
# bind-mount the current directory read-only
|
||||
docker run -v $(pwd):/app:ro <image>
|
||||
|
||||
# set the working directory
|
||||
docker run -w /app <image> <command>
|
||||
|
||||
# run as your own UID/GID, not root
|
||||
docker run -u $(id -u):$(id -g) <image>
|
||||
|
||||
# attach to a user-defined network
|
||||
docker run --network <network> <image>
|
||||
|
||||
# restart policy, see the flags table below
|
||||
docker run --restart unless-stopped <image>
|
||||
|
||||
# cap the resources it may use
|
||||
docker run --memory 512m --cpus 1.5 <image>
|
||||
|
||||
# override the image entrypoint
|
||||
docker run --entrypoint <cmd> <image>
|
||||
|
||||
# create without starting
|
||||
docker create --name <container> <image>
|
||||
```
|
||||
|
||||
Pull an image from Docker Hub:
|
||||
```bash
|
||||
docker pull <image_name>
|
||||
```
|
||||
|
||||
Pull a specific tagged version:
|
||||
```bash
|
||||
docker pull <image_name>:<tag>
|
||||
```
|
||||
|
||||
Publish an image to Docker Hub:
|
||||
```bash
|
||||
docker push <username>/<image_name>
|
||||
```
|
||||
|
||||
## Containers
|
||||
|
||||
A container is a runtime instance of a Docker image. A container will always run the same, regardless of the infrastructure. Containers isolate software from its environment and ensure it works uniformly across development, staging, and production.
|
||||
|
||||
### Running containers
|
||||
|
||||
Create and run a container from an image:
|
||||
```bash
|
||||
docker run <image_name>
|
||||
```
|
||||
|
||||
Run with a custom name:
|
||||
```bash
|
||||
docker run --name <container_name> <image_name>
|
||||
```
|
||||
|
||||
Run in the background (detached):
|
||||
```bash
|
||||
docker run -d <image_name>
|
||||
```
|
||||
|
||||
Run interactively with a shell:
|
||||
```bash
|
||||
docker run -it <image_name> bash
|
||||
```
|
||||
|
||||
Auto-remove the container when it exits:
|
||||
```bash
|
||||
docker run --rm <image_name>
|
||||
```
|
||||
|
||||
Publish a container's port to the host:
|
||||
```bash
|
||||
docker run -p <host_port>:<container_port> <image_name>
|
||||
```
|
||||
|
||||
Set an environment variable:
|
||||
```bash
|
||||
docker run -e KEY=value <image_name>
|
||||
```
|
||||
|
||||
Load environment variables from a file:
|
||||
```bash
|
||||
docker run --env-file ./env.list <image_name>
|
||||
```
|
||||
|
||||
Mount a named volume:
|
||||
```bash
|
||||
docker run -v <volume_name>:/path/in/container <image_name>
|
||||
```
|
||||
|
||||
Bind-mount the current directory:
|
||||
```bash
|
||||
docker run -v $(pwd):/app <image_name>
|
||||
```
|
||||
|
||||
Set a restart policy (options: `no`, `on-failure`, `always`, `unless-stopped`):
|
||||
```bash
|
||||
docker run --restart unless-stopped <image_name>
|
||||
```
|
||||
|
||||
Attach to a specific network:
|
||||
```bash
|
||||
docker run --network <network_name> <image_name>
|
||||
```
|
||||
|
||||
### Managing containers
|
||||
|
||||
List currently running containers:
|
||||
## 🎛 Managing Containers
|
||||
|
||||
```bash
|
||||
# running containers
|
||||
docker ps
|
||||
```
|
||||
|
||||
List all containers (running and stopped):
|
||||
```bash
|
||||
# every container, running or exited
|
||||
docker ps -a
|
||||
```
|
||||
|
||||
Start a stopped container:
|
||||
```bash
|
||||
docker start <container_name>
|
||||
```
|
||||
# IDs only — handy for scripting
|
||||
docker ps -q
|
||||
|
||||
Stop a running container:
|
||||
```bash
|
||||
docker stop <container_name>
|
||||
```
|
||||
# filter by status, name, label, ancestor, ...
|
||||
docker ps --filter "status=exited"
|
||||
|
||||
Restart a container:
|
||||
```bash
|
||||
docker restart <container_name>
|
||||
```
|
||||
# start a stopped container
|
||||
docker start <container>
|
||||
|
||||
Remove a stopped container:
|
||||
```bash
|
||||
docker rm <container_name>
|
||||
```
|
||||
# graceful stop (SIGTERM, then SIGKILL)
|
||||
docker stop <container>
|
||||
|
||||
Force-remove a running container:
|
||||
```bash
|
||||
docker rm -f <container_name>
|
||||
```
|
||||
# stop and start again
|
||||
docker restart <container>
|
||||
|
||||
Remove all stopped containers:
|
||||
```bash
|
||||
# immediate SIGKILL
|
||||
docker kill <container>
|
||||
|
||||
# freeze all processes in the container
|
||||
docker pause <container>
|
||||
|
||||
# resume them
|
||||
docker unpause <container>
|
||||
|
||||
# remove a stopped container
|
||||
docker rm <container>
|
||||
|
||||
# stop and remove in one go
|
||||
docker rm -f <container>
|
||||
|
||||
# rename a container
|
||||
docker rename <old> <new>
|
||||
|
||||
# change resource limits or restart policy live
|
||||
docker update --restart=always <container>
|
||||
|
||||
# remove every stopped container
|
||||
docker container prune
|
||||
|
||||
# block until it exits, then print its exit code
|
||||
docker wait <container>
|
||||
```
|
||||
|
||||
Rename a container:
|
||||
```bash
|
||||
docker rename <old_name> <new_name>
|
||||
```
|
||||
## 🔍 Inspecting & Debugging
|
||||
|
||||
### Inspecting and interacting
|
||||
|
||||
Open a shell inside a running container:
|
||||
```bash
|
||||
docker exec -it <container_name> bash
|
||||
```
|
||||
# shell inside a running container
|
||||
docker exec -it <container> bash
|
||||
|
||||
Use `sh` for minimal images (e.g. alpine) that don't ship bash:
|
||||
```bash
|
||||
docker exec -it <container_name> sh
|
||||
```
|
||||
# for alpine/distroless-style images
|
||||
docker exec -it <container> sh
|
||||
|
||||
Fetch logs:
|
||||
```bash
|
||||
docker logs <container_name>
|
||||
```
|
||||
# get in as root to install debug tools
|
||||
docker exec -it -u root <container> sh
|
||||
|
||||
Follow logs (like `tail -f`):
|
||||
```bash
|
||||
docker logs -f <container_name>
|
||||
```
|
||||
# run a one-off command
|
||||
docker exec <container> <command>
|
||||
|
||||
Show only the last 100 lines:
|
||||
```bash
|
||||
docker logs --tail 100 <container_name>
|
||||
```
|
||||
# print the container's output
|
||||
docker logs <container>
|
||||
|
||||
Show low-level container details (JSON):
|
||||
```bash
|
||||
docker inspect <container_name>
|
||||
```
|
||||
# follow it, like tail -f
|
||||
docker logs -f <container>
|
||||
|
||||
Live resource usage stats for all containers:
|
||||
```bash
|
||||
# last 100 lines, with timestamps
|
||||
docker logs --tail 100 -t <container>
|
||||
|
||||
# only the last 10 minutes
|
||||
docker logs --since 10m <container>
|
||||
|
||||
# low-level details as JSON
|
||||
docker inspect <container>
|
||||
|
||||
# one field via a Go template
|
||||
docker inspect -f '{{.State.Status}}' <container>
|
||||
|
||||
# the container's IP address on the default bridge
|
||||
docker inspect -f '{{.NetworkSettings.IPAddress}}' <container>
|
||||
|
||||
# live CPU/memory/IO for all containers
|
||||
docker stats
|
||||
|
||||
# processes running inside
|
||||
docker top <container>
|
||||
|
||||
# published port mappings
|
||||
docker port <container>
|
||||
|
||||
# filesystem changes since it started
|
||||
docker diff <container>
|
||||
|
||||
# copy a file out of a container
|
||||
docker cp <container>:/path/file ./
|
||||
|
||||
# copy a file into a container
|
||||
docker cp ./file <container>:/path/
|
||||
|
||||
# snapshot a container as a new image
|
||||
docker commit <container> <image>:<tag>
|
||||
|
||||
# attach to the main process (Ctrl-P Ctrl-Q detaches)
|
||||
docker attach <container>
|
||||
```
|
||||
|
||||
Show running processes inside a container:
|
||||
## 💾 Volumes
|
||||
|
||||
Volumes keep data outside the container's writable layer, so it survives `docker rm` and image upgrades.
|
||||
|
||||
```bash
|
||||
docker top <container_name>
|
||||
```
|
||||
# create a named volume
|
||||
docker volume create <volume>
|
||||
|
||||
Copy a file out of a container:
|
||||
```bash
|
||||
docker cp <container_name>:/path/in/container ./
|
||||
```
|
||||
|
||||
Copy a file into a container:
|
||||
```bash
|
||||
docker cp ./local_file <container_name>:/path/
|
||||
```
|
||||
|
||||
Show filesystem changes since container start:
|
||||
```bash
|
||||
docker diff <container_name>
|
||||
```
|
||||
|
||||
## Volumes
|
||||
|
||||
Volumes persist data outside the container's writable layer, so data survives container removal.
|
||||
|
||||
Create a named volume:
|
||||
```bash
|
||||
docker volume create <volume_name>
|
||||
```
|
||||
|
||||
List volumes:
|
||||
```bash
|
||||
# list volumes
|
||||
docker volume ls
|
||||
```
|
||||
|
||||
Show details about a volume:
|
||||
```bash
|
||||
docker volume inspect <volume_name>
|
||||
```
|
||||
# driver, mount point, labels
|
||||
docker volume inspect <volume>
|
||||
|
||||
Remove a volume:
|
||||
```bash
|
||||
docker volume rm <volume_name>
|
||||
```
|
||||
# delete a volume (and its data)
|
||||
docker volume rm <volume>
|
||||
|
||||
Remove all unused volumes:
|
||||
```bash
|
||||
# delete every unused volume
|
||||
docker volume prune
|
||||
|
||||
# named volume, managed by Docker
|
||||
docker run -v <volume>:/data <image>
|
||||
|
||||
# bind mount from the host
|
||||
docker run -v $(pwd):/app <image>
|
||||
|
||||
# explicit bind-mount syntax
|
||||
docker run --mount type=bind,src=$(pwd),dst=/app <image>
|
||||
|
||||
# the same named volume, spelled out with --mount
|
||||
docker run --mount type=volume,src=<volume>,dst=/data <image>
|
||||
|
||||
# in-memory scratch space
|
||||
docker run --tmpfs /tmp <image>
|
||||
```
|
||||
|
||||
Attach a named volume to a container (managed by Docker):
|
||||
```bash
|
||||
docker run -v <volume_name>:/data <image_name>
|
||||
```
|
||||
|
||||
Bind-mount a host path into a container:
|
||||
```bash
|
||||
docker run -v $(pwd):/app <image_name>
|
||||
```
|
||||
|
||||
Equivalent bind mount using the `--mount` syntax:
|
||||
```bash
|
||||
docker run --mount type=bind,src=$(pwd),dst=/app <image_name>
|
||||
```
|
||||
|
||||
## Networks
|
||||
|
||||
Containers on the same network can reach each other by container name.
|
||||
|
||||
List networks:
|
||||
> 💡 Back up a volume: `docker run --rm -v <volume>:/data -v $(pwd):/backup alpine tar czf /backup/backup.tar.gz -C /data .`
|
||||
|
||||
## 🌐 Networks
|
||||
|
||||
Containers on the same user-defined network resolve each other by container name.
|
||||
|
||||
```bash
|
||||
# list networks
|
||||
docker network ls
|
||||
```
|
||||
|
||||
Create a network:
|
||||
```bash
|
||||
docker network create <network_name>
|
||||
```
|
||||
# create a bridge network
|
||||
docker network create <network>
|
||||
|
||||
Show network details:
|
||||
```bash
|
||||
docker network inspect <network_name>
|
||||
```
|
||||
# create a network with a fixed subnet
|
||||
docker network create --driver bridge --subnet 172.30.0.0/16 <network>
|
||||
|
||||
Attach a running container to a network:
|
||||
```bash
|
||||
docker network connect <network_name> <container_name>
|
||||
```
|
||||
# subnet and connected containers
|
||||
docker network inspect <network>
|
||||
|
||||
Detach a container from a network:
|
||||
```bash
|
||||
docker network disconnect <network_name> <container_name>
|
||||
```
|
||||
# attach a running container
|
||||
docker network connect <network> <container>
|
||||
|
||||
Remove a network:
|
||||
```bash
|
||||
docker network rm <network_name>
|
||||
```
|
||||
# detach it again
|
||||
docker network disconnect <network> <container>
|
||||
|
||||
Remove all unused networks:
|
||||
```bash
|
||||
# delete a network
|
||||
docker network rm <network>
|
||||
|
||||
# delete every unused network
|
||||
docker network prune
|
||||
|
||||
# share the host network stack (Linux)
|
||||
docker run --network host <image>
|
||||
|
||||
# no networking at all
|
||||
docker run --network none <image>
|
||||
```
|
||||
|
||||
## Docker Compose
|
||||
## 🧩 Docker Compose
|
||||
|
||||
Compose runs multi-container apps from a single `compose.yaml` (or `docker-compose.yml`) file.
|
||||
Compose describes a multi-container application in a single `compose.yaml` (or `docker-compose.yml`).
|
||||
|
||||
Start all services (foreground):
|
||||
```bash
|
||||
# start every service in the foreground
|
||||
docker compose up
|
||||
```
|
||||
|
||||
Start all services (detached):
|
||||
```bash
|
||||
# start detached
|
||||
docker compose up -d
|
||||
```
|
||||
|
||||
Rebuild images before starting:
|
||||
```bash
|
||||
docker compose up --build
|
||||
```
|
||||
# rebuild images first
|
||||
docker compose up -d --build
|
||||
|
||||
Stop and remove containers and networks:
|
||||
```bash
|
||||
# start one service and its dependencies
|
||||
docker compose up -d <service>
|
||||
|
||||
# stop and remove containers and networks
|
||||
docker compose down
|
||||
```
|
||||
|
||||
Also remove named volumes:
|
||||
```bash
|
||||
# stop, remove, and delete the named volumes too
|
||||
docker compose down -v
|
||||
```
|
||||
|
||||
List services in the project:
|
||||
```bash
|
||||
# status of the project's services
|
||||
docker compose ps
|
||||
```
|
||||
|
||||
Follow logs for all services:
|
||||
```bash
|
||||
# follow the logs of all services
|
||||
docker compose logs -f
|
||||
```
|
||||
|
||||
Follow logs for one service:
|
||||
```bash
|
||||
# follow the logs of a single service
|
||||
docker compose logs -f <service>
|
||||
```
|
||||
|
||||
Build (or rebuild) services:
|
||||
```bash
|
||||
# (re)build the service images
|
||||
docker compose build
|
||||
```
|
||||
|
||||
Pull service images:
|
||||
```bash
|
||||
# pull the service images
|
||||
docker compose pull
|
||||
```
|
||||
|
||||
Open a shell in a running service:
|
||||
```bash
|
||||
docker compose exec <service> bash
|
||||
```
|
||||
# shell inside a running service
|
||||
docker compose exec <service> sh
|
||||
|
||||
Run a one-off command in a new container:
|
||||
```bash
|
||||
docker compose run --rm <service> <command>
|
||||
```
|
||||
# one-off command in a new container
|
||||
docker compose run --rm <service> <cmd>
|
||||
|
||||
Restart a single service:
|
||||
```bash
|
||||
# restart one service
|
||||
docker compose restart <service>
|
||||
```
|
||||
|
||||
Validate and view the resolved compose file:
|
||||
```bash
|
||||
# stop the containers, keep them around
|
||||
docker compose stop
|
||||
|
||||
# start them again
|
||||
docker compose start
|
||||
|
||||
# validate and print the resolved configuration
|
||||
docker compose config
|
||||
|
||||
# processes running in each service
|
||||
docker compose top
|
||||
|
||||
# use a specific compose file
|
||||
docker compose -f <file> up -d
|
||||
|
||||
# include services behind a profile
|
||||
docker compose --profile <name> up -d
|
||||
|
||||
# rebuild/sync automatically on file changes
|
||||
docker compose watch
|
||||
```
|
||||
|
||||
## System & Cleanup
|
||||
## 🧹 System & Cleanup
|
||||
|
||||
Disk usage adds up fast. These commands reclaim space.
|
||||
Disk usage adds up fast — these are the commands that give the space back.
|
||||
|
||||
Show disk usage (images, containers, volumes, build cache):
|
||||
```bash
|
||||
# what images, containers, volumes and cache cost
|
||||
docker system df
|
||||
```
|
||||
|
||||
Remove stopped containers, dangling images, and unused networks:
|
||||
```bash
|
||||
# the same, itemised
|
||||
docker system df -v
|
||||
|
||||
# stopped containers, dangling images, unused networks
|
||||
docker system prune
|
||||
```
|
||||
|
||||
Also remove all unused images (not just dangling):
|
||||
```bash
|
||||
# the same, plus every image no container uses
|
||||
docker system prune -a
|
||||
```
|
||||
|
||||
Also remove unused volumes (destructive — double-check before running):
|
||||
```bash
|
||||
# the same, plus unused volumes ⚠️ destructive
|
||||
docker system prune -a --volumes
|
||||
```
|
||||
|
||||
Clean the build cache:
|
||||
```bash
|
||||
# clear the build cache
|
||||
docker builder prune
|
||||
|
||||
# images unused for more than a week
|
||||
docker image prune -a --filter "until=168h"
|
||||
```
|
||||
|
||||
## Quick Reference: Common Flags
|
||||
> ⚠️ `--volumes` deletes data no running container is using. Check `docker volume ls` first.
|
||||
|
||||
| Flag | Meaning |
|
||||
|------|---------|
|
||||
| `-d` | Detached (run in background) |
|
||||
| `-it` | Interactive + TTY (for shells) |
|
||||
| `--rm` | Remove container when it exits |
|
||||
| `-p host:container` | Publish a port |
|
||||
| `-v src:dst` | Mount volume or bind mount |
|
||||
| `-e KEY=value` | Environment variable |
|
||||
| `--name` | Assign a container name |
|
||||
| `--network` | Attach to a network |
|
||||
| `--restart` | Restart policy (`no`, `on-failure`, `always`, `unless-stopped`) |
|
||||
## 💡 Handy One-Liners
|
||||
|
||||
```bash
|
||||
# stop every running container
|
||||
docker stop $(docker ps -q)
|
||||
|
||||
# remove every container
|
||||
docker rm -f $(docker ps -aq)
|
||||
|
||||
# remove dangling images
|
||||
docker rmi $(docker images -qf dangling=true)
|
||||
|
||||
# shell into the most recent container
|
||||
docker exec -it $(docker ps -ql) sh
|
||||
|
||||
# disposable dev environment
|
||||
docker run --rm -it -v $(pwd):/app -w /app node:22 sh
|
||||
|
||||
# follow logs by partial name
|
||||
docker logs -f $(docker ps -qf name=<partial>)
|
||||
|
||||
# list every mount of a container
|
||||
docker inspect \
|
||||
-f '{{range .Mounts}}{{.Source}} -> {{.Destination}}{{println}}{{end}}' <container>
|
||||
```
|
||||
|
||||
## 🛠 Common Flags Reference
|
||||
|
||||
| Flag | Meaning |
|
||||
|-------------------------------|--------------------------------------------------|
|
||||
| `-d`, `--detach` | Run in the background |
|
||||
| `-it` | Interactive session with a TTY (shells) |
|
||||
| `--rm` | Remove the container when it exits |
|
||||
| `-p <host>:<container>` | Publish a port to the host |
|
||||
| `-v <src>:<dst>[:ro]` | Named volume or bind mount, optionally read-only |
|
||||
| `-e KEY=value` / `--env-file` | Environment variables |
|
||||
| `--name` | Assign a stable container name |
|
||||
| `--network` | Attach to a network |
|
||||
| `--restart` | `no`, `on-failure`, `always`, `unless-stopped` |
|
||||
| `-u <uid>:<gid>` | Run as a specific user |
|
||||
| `-w <dir>` | Working directory inside the container |
|
||||
| `--memory` / `--cpus` | Resource limits |
|
||||
| `--platform` | Target architecture, e.g. `linux/arm64` |
|
||||
|
||||
## 📚 Resources
|
||||
|
||||
- [Official documentation](https://docs.docker.com)
|
||||
- [Docker Desktop (Mac, Linux, Windows)](https://docs.docker.com/desktop)
|
||||
- [CLI reference](https://docs.docker.com/reference/cli/docker/)
|
||||
- [Dockerfile reference](https://docs.docker.com/reference/dockerfile/)
|
||||
- [Compose file reference](https://docs.docker.com/reference/compose-file/)
|
||||
- [Awesome Compose — example projects](https://github.com/docker/awesome-compose)
|
||||
@@ -1,309 +1,356 @@
|
||||
**Docker Swarm Mode** is Docker's native clustering and orchestration solution. It lets you manage a group of Docker nodes as a single virtual system, enabling high availability, load balancing, and simplified deployment of containerized applications.
|
||||
**Docker Swarm mode** is Docker's native clustering and orchestration solution. It turns a group of Docker hosts into a single virtual system with high availability, load balancing and a declarative deployment model — using the CLI you already know.
|
||||
|
||||
Key features:
|
||||
- Built-in orchestration with a declarative service model
|
||||
- Rolling updates and rollback
|
||||
- Self-healing (failed replicas are rescheduled automatically)
|
||||
- Secure node-to-node communication via mutual TLS
|
||||
- Built-in secrets and config distribution
|
||||
- Rolling updates and one-command rollback
|
||||
- Self-healing: failed replicas are rescheduled automatically
|
||||
- Secure node-to-node communication over mutual TLS, rotated automatically
|
||||
- Secrets and configs distributed straight from the cluster store
|
||||
|
||||
> ℹ️ Swarm does **not** auto-scale based on load. Replica counts are set manually with `docker service scale` or `--replicas`.
|
||||
> ℹ️ Swarm does **not** auto-scale on load. Replica counts are set manually with `--replicas` or `docker service scale`.
|
||||
|
||||
## 🧱 Concepts
|
||||
|
||||
| Term | What it is |
|
||||
|-------------|-------------------------------------------------------------------|
|
||||
| **Node** | A Docker engine that joined the swarm — a *manager* or a *worker* |
|
||||
| **Manager** | Keeps cluster state via Raft, schedules tasks, serves the API |
|
||||
| **Worker** | Runs tasks only; has no view of the cluster state |
|
||||
| **Service** | The declaration of what should run (image, replicas, ports, …) |
|
||||
| **Task** | One container slot of a service, scheduled onto a node |
|
||||
| **Stack** | A group of services deployed together from a Compose file |
|
||||
|
||||
## 📌 Swarm Initialization
|
||||
|
||||
Initialize a new swarm on the current node (becomes the first manager):
|
||||
|
||||
```bash
|
||||
# initialize a swarm; this node becomes the first manager
|
||||
docker swarm init
|
||||
```
|
||||
|
||||
If the host has multiple network interfaces, specify which one to advertise on:
|
||||
|
||||
```bash
|
||||
# pick the interface to advertise on multi-homed hosts
|
||||
docker swarm init --advertise-addr <manager-ip>
|
||||
```
|
||||
|
||||
Show the command another node needs to join as a worker:
|
||||
# address pool for the overlay networks
|
||||
docker swarm init --default-addr-pool 10.20.0.0/16
|
||||
|
||||
```bash
|
||||
# print the command a worker needs to join
|
||||
docker swarm join-token worker
|
||||
```
|
||||
|
||||
Show the command for joining as a manager:
|
||||
|
||||
```bash
|
||||
# print the command a manager needs to join
|
||||
docker swarm join-token manager
|
||||
```
|
||||
|
||||
Join an existing swarm (run on the joining node, using the output of the commands above):
|
||||
|
||||
```bash
|
||||
# run on the joining node
|
||||
docker swarm join --token <token> <manager-ip>:2377
|
||||
```
|
||||
|
||||
Rotate join tokens (invalidates the old ones):
|
||||
|
||||
```bash
|
||||
# rotate a token, invalidating the old one
|
||||
docker swarm join-token --rotate worker
|
||||
```
|
||||
|
||||
Leave the swarm (on a worker):
|
||||
|
||||
```bash
|
||||
# leave the swarm (on a worker)
|
||||
docker swarm leave
|
||||
```
|
||||
|
||||
Force leave (on a manager — last manager must use `--force`):
|
||||
|
||||
```bash
|
||||
# leave on a manager, or on the last node
|
||||
docker swarm leave --force
|
||||
|
||||
# keep fewer terminated tasks per service
|
||||
docker swarm update --task-history-limit 5
|
||||
|
||||
# lifetime of the node certificates
|
||||
docker swarm update --cert-expiry 720h
|
||||
|
||||
# rotate the swarm certificate authority
|
||||
docker swarm ca --rotate
|
||||
```
|
||||
|
||||
### 🔌 Ports that must be open between nodes
|
||||
|
||||
| Port | Protocol | Used for |
|
||||
|------------------------|-----------|--------------------------------------|
|
||||
| `2377` | TCP | Cluster management (managers only) |
|
||||
| `7946` | TCP + UDP | Node discovery and gossip |
|
||||
| `4789` | UDP | Overlay network data plane (VXLAN) |
|
||||
| IP protocol `50` (ESP) | — | Only with encrypted overlay networks |
|
||||
|
||||
## 👥 Node Management
|
||||
|
||||
List nodes in the swarm:
|
||||
|
||||
```bash
|
||||
# list the nodes of the swarm
|
||||
docker node ls
|
||||
|
||||
# filter by role, id, name, label, membership
|
||||
docker node ls --filter role=manager
|
||||
|
||||
# readable summary of one node
|
||||
docker node inspect <node> --pretty
|
||||
|
||||
# a single field via a Go template
|
||||
docker node inspect -f '{{.Status.State}}' <node>
|
||||
|
||||
# worker → manager
|
||||
docker node promote <node>
|
||||
|
||||
# manager → worker
|
||||
docker node demote <node>
|
||||
|
||||
# move tasks off and stop scheduling new ones
|
||||
docker node update --availability drain <node>
|
||||
|
||||
# keep running tasks, schedule nothing new
|
||||
docker node update --availability pause <node>
|
||||
|
||||
# bring a drained or paused node back
|
||||
docker node update --availability active <node>
|
||||
|
||||
# label a node for placement constraints
|
||||
docker node update --label-add env=production <node>
|
||||
|
||||
# remove a label again
|
||||
docker node update --label-rm env <node>
|
||||
|
||||
# tasks running on a node
|
||||
docker node ps <node>
|
||||
|
||||
# remove a node that already left
|
||||
docker node rm <node>
|
||||
|
||||
# remove an unreachable node
|
||||
docker node rm --force <node>
|
||||
```
|
||||
|
||||
Inspect a node:
|
||||
|
||||
```bash
|
||||
docker node inspect <node-name> --pretty
|
||||
```
|
||||
|
||||
Promote a worker to manager:
|
||||
|
||||
```bash
|
||||
docker node promote <node-name>
|
||||
```
|
||||
|
||||
Demote a manager to worker:
|
||||
|
||||
```bash
|
||||
docker node demote <node-name>
|
||||
```
|
||||
|
||||
Drain a node (reschedule its tasks elsewhere, prevent new scheduling):
|
||||
|
||||
```bash
|
||||
docker node update --availability drain <node-name>
|
||||
```
|
||||
|
||||
Pause a node (keep running tasks, prevent new ones):
|
||||
|
||||
```bash
|
||||
docker node update --availability pause <node-name>
|
||||
```
|
||||
|
||||
Activate a drained or paused node:
|
||||
|
||||
```bash
|
||||
docker node update --availability active <node-name>
|
||||
```
|
||||
|
||||
Add a label to a node (for use with placement constraints):
|
||||
|
||||
```bash
|
||||
docker node update --label-add env=production <node-name>
|
||||
```
|
||||
|
||||
Remove a node from the swarm (after it has left or is unreachable):
|
||||
|
||||
```bash
|
||||
docker node rm <node-name>
|
||||
```
|
||||
|
||||
## 🧠 Manager Nodes
|
||||
|
||||
Manager nodes orchestrate tasks, maintain cluster state via Raft consensus, and handle API requests. You can run multiple managers for HA, but only one is the **leader** at any time.
|
||||
|
||||
> ⚠️ **Use an odd number of managers** (3, 5, or 7). Raft tolerates `(N-1)/2` failures — 3 managers tolerate 1 failure, 5 tolerate 2. More than 7 hurts performance.
|
||||
|
||||
Show Swarm and Raft state for the local node:
|
||||
> 💡 Drain a node before maintenance (`--availability drain`), then set it back to `active`. Swarm reschedules the tasks for you.
|
||||
|
||||
## 🧠 Manager Nodes & Raft
|
||||
|
||||
Managers orchestrate tasks, maintain the cluster state through Raft consensus and serve the API. Several managers give you high availability, but only one is the **leader** at a time.
|
||||
|
||||
> ⚠️ **Use an odd number of managers** (3, 5 or 7). Raft tolerates `(N-1)/2` failures — 3 managers survive 1 loss, 5 survive 2. Beyond 7 the consensus overhead outweighs the benefit.
|
||||
|
||||
```bash
|
||||
# swarm and Raft state of the local node
|
||||
docker info
|
||||
```
|
||||
|
||||
Inspect the current node (includes manager status if applicable):
|
||||
|
||||
```bash
|
||||
# details of the current node, manager status included
|
||||
docker node inspect self --pretty
|
||||
```
|
||||
|
||||
Enable autolock (encrypts Raft logs; managers need a key to restart):
|
||||
|
||||
```bash
|
||||
# encrypt the Raft logs at rest
|
||||
docker swarm update --autolock=true
|
||||
```
|
||||
|
||||
Unlock a manager after restart:
|
||||
|
||||
```bash
|
||||
# unlock a manager after a restart
|
||||
docker swarm unlock
|
||||
```
|
||||
|
||||
Show or rotate the unlock key:
|
||||
|
||||
```bash
|
||||
# show the current unlock key
|
||||
docker swarm unlock-key
|
||||
|
||||
# rotate it
|
||||
docker swarm unlock-key --rotate
|
||||
```
|
||||
|
||||
## 📦 Service Management
|
||||
|
||||
Create a service:
|
||||
|
||||
```bash
|
||||
docker service create --name <service-name> <image>
|
||||
```
|
||||
# create a service
|
||||
docker service create --name <service> <image>
|
||||
|
||||
Create a service with a fixed number of replicas:
|
||||
# create a service with a fixed replica count
|
||||
docker service create --name <service> --replicas <n> <image>
|
||||
|
||||
```bash
|
||||
docker service create --name <service-name> --replicas <n> <image>
|
||||
```
|
||||
# create a global service: one task on every node
|
||||
docker service create --name <service> --mode global <image>
|
||||
|
||||
Create a global service (one task per node):
|
||||
|
||||
```bash
|
||||
docker service create --name <service-name> --mode global <image>
|
||||
```
|
||||
|
||||
List services:
|
||||
|
||||
```bash
|
||||
# list services
|
||||
docker service ls
|
||||
|
||||
# filter the list
|
||||
docker service ls --filter name=<service>
|
||||
|
||||
# readable service definition
|
||||
docker service inspect <service> --pretty
|
||||
|
||||
# tasks and the nodes they run on
|
||||
docker service ps <service>
|
||||
|
||||
# change the replica count
|
||||
docker service scale <service>=<n>
|
||||
|
||||
# scale several services at once
|
||||
docker service scale <svc-a>=3 <svc-b>=5
|
||||
|
||||
# roll out a new image
|
||||
docker service update --image <new-image> <service>
|
||||
|
||||
# redistribute tasks without changing anything
|
||||
docker service update --force <service>
|
||||
|
||||
# go back to the previous definition
|
||||
docker service rollback <service>
|
||||
|
||||
# follow the logs of every task
|
||||
docker service logs -f <service>
|
||||
|
||||
# recent output only
|
||||
docker service logs --tail 100 --since 10m <service>
|
||||
|
||||
# remove a service
|
||||
docker service rm <service>
|
||||
```
|
||||
|
||||
Inspect a service:
|
||||
### Changing a running service
|
||||
|
||||
```bash
|
||||
docker service inspect <service-name> --pretty
|
||||
# add or replace an environment variable
|
||||
docker service update --env-add KEY=value <service>
|
||||
|
||||
# remove one
|
||||
docker service update --env-rm KEY <service>
|
||||
|
||||
# publish another port
|
||||
docker service update --publish-add 8080:80 <service>
|
||||
|
||||
# stop publishing it
|
||||
docker service update --publish-rm 8080 <service>
|
||||
|
||||
# attach a volume
|
||||
docker service update --mount-add type=volume,src=data,dst=/data <service>
|
||||
|
||||
# detach it again, by target path
|
||||
docker service update --mount-rm /data <service>
|
||||
|
||||
# add a placement constraint
|
||||
docker service update --constraint-add 'node.labels.env==prod' <service>
|
||||
|
||||
# at most one task per node
|
||||
docker service update --replicas-max-per-node 1 <service>
|
||||
|
||||
# change the resource limits
|
||||
docker service update --limit-memory 512m --reserve-memory 256m <service>
|
||||
```
|
||||
|
||||
Scale a service:
|
||||
|
||||
```bash
|
||||
docker service scale <service-name>=<n>
|
||||
```
|
||||
|
||||
Update a service (e.g. change image):
|
||||
|
||||
```bash
|
||||
docker service update --image <new-image> <service-name>
|
||||
```
|
||||
|
||||
Roll back the last update:
|
||||
|
||||
```bash
|
||||
docker service rollback <service-name>
|
||||
```
|
||||
|
||||
Follow logs from all tasks of a service:
|
||||
|
||||
```bash
|
||||
docker service logs -f <service-name>
|
||||
```
|
||||
|
||||
Remove a service:
|
||||
|
||||
```bash
|
||||
docker service rm <service-name>
|
||||
```
|
||||
|
||||
## 🔁 Replicas
|
||||
|
||||
Replicas define how many instances of a service should run across the swarm. Swarm distributes them across available nodes and reschedules them if they fail.
|
||||
|
||||
Set replicas at create time:
|
||||
## 🔁 Replicas & Modes
|
||||
|
||||
Replicas define how many instances of a service run across the swarm. Swarm spreads them over the available nodes and reschedules them when a task or a node fails.
|
||||
|
||||
| Mode | Flag | Behaviour |
|
||||
|----------------|-------------------------------|---------------------------------------------------|
|
||||
| Replicated | `--mode replicated` (default) | Run exactly `--replicas` tasks, anywhere they fit |
|
||||
| Global | `--mode global` | Exactly one task per eligible node |
|
||||
| Replicated job | `--mode replicated-job` | Run `--replicas` tasks **to completion** |
|
||||
| Global job | `--mode global-job` | Run one task to completion on every node |
|
||||
|
||||
```bash
|
||||
# five tasks
|
||||
docker service create --replicas 5 --name myapp myimage
|
||||
```
|
||||
|
||||
Change replica count later:
|
||||
|
||||
```bash
|
||||
# scale the same service up to ten
|
||||
docker service scale myapp=10
|
||||
|
||||
# one per node
|
||||
docker service create --mode global --name agent myimage
|
||||
|
||||
# batch job: 20 tasks in total, 4 running at a time
|
||||
docker service create --mode replicated-job --replicas 20 \
|
||||
--max-concurrent 4 --name migrate myimage
|
||||
```
|
||||
|
||||
## 🐝 Task & Container Management
|
||||
## 🐝 Tasks & Containers
|
||||
|
||||
In Swarm, a **task** is a single container slot managed by Swarm. A task maps to a running container on some node.
|
||||
|
||||
List tasks of a service (shows which node each runs on):
|
||||
A **task** is a single container slot managed by Swarm; it maps to one container on one node and is never moved — a failed task is replaced by a new one.
|
||||
|
||||
```bash
|
||||
docker service ps <service-name>
|
||||
```
|
||||
# tasks of a service, with their node
|
||||
docker service ps <service>
|
||||
|
||||
List tasks running on a specific node:
|
||||
# full error messages — start debugging here
|
||||
docker service ps --no-trunc <service>
|
||||
|
||||
```bash
|
||||
docker node ps <node-name>
|
||||
```
|
||||
# hide the terminated tasks
|
||||
docker service ps -f "desired-state=running" <service>
|
||||
|
||||
List containers on the current node:
|
||||
# tasks on one node
|
||||
docker node ps <node>
|
||||
|
||||
```bash
|
||||
# containers on the current node
|
||||
docker container ls
|
||||
```
|
||||
|
||||
Inspect a container:
|
||||
|
||||
```bash
|
||||
# low-level details of one container
|
||||
docker container inspect <container-id>
|
||||
```
|
||||
|
||||
| Task state | Meaning |
|
||||
|--------------------------|-------------------------------------------------------------|
|
||||
| `NEW` / `PENDING` | Accepted, waiting for a node that satisfies the constraints |
|
||||
| `ASSIGNED` / `PREPARING` | Sent to a node, image being pulled |
|
||||
| `STARTING` / `RUNNING` | The container is starting or up |
|
||||
| `COMPLETE` | A job task finished successfully |
|
||||
| `FAILED` | The container exited with an error |
|
||||
| `SHUTDOWN` | Stopped on purpose (update, drain, scale down) |
|
||||
| `REJECTED` | The node refused the task (missing image, bad mount) |
|
||||
| `ORPHANED` | The node has been unreachable too long |
|
||||
|
||||
## 🎯 Placement
|
||||
|
||||
```bash
|
||||
docker service create --constraint 'node.role==worker' --name web nginx
|
||||
|
||||
docker service create --constraint 'node.labels.env==production' --name api myimage
|
||||
|
||||
docker service create --placement-pref 'spread=node.labels.zone' --name web nginx
|
||||
|
||||
docker service create --replicas-max-per-node 1 --name web nginx
|
||||
```
|
||||
|
||||
| Expression | Matches |
|
||||
|-------------------------------------------|---------------------------------------------------|
|
||||
| `node.role` | `manager` or `worker` |
|
||||
| `node.hostname` | The node's hostname |
|
||||
| `node.id` | The node's ID |
|
||||
| `node.labels.<key>` | A label set with `docker node update --label-add` |
|
||||
| `engine.labels.<key>` | A label set in the engine's `daemon.json` |
|
||||
| `node.platform.os` / `node.platform.arch` | `linux`, `windows` / `amd64`, `arm64` |
|
||||
|
||||
## 🌐 Networking
|
||||
|
||||
### What is an overlay network?
|
||||
|
||||
An overlay network is a virtual network that spans multiple Docker hosts. It lets containers running on different nodes communicate as if they were on the same Layer 2 network, with built-in DNS-based service discovery and encrypted traffic between nodes.
|
||||
|
||||
Create an overlay network:
|
||||
An **overlay network** is a virtual network spanning multiple Docker hosts. Containers on different nodes talk as if they were on the same L2 segment, with DNS-based service discovery built in.
|
||||
|
||||
```bash
|
||||
docker network create --driver overlay <network-name>
|
||||
```
|
||||
# create an overlay network
|
||||
docker network create --driver overlay <network>
|
||||
|
||||
Create an encrypted overlay network (encrypts data plane between nodes):
|
||||
# encrypt the data plane too
|
||||
docker network create --driver overlay --opt encrypted <network>
|
||||
|
||||
```bash
|
||||
docker network create --driver overlay --opt encrypted <network-name>
|
||||
```
|
||||
# let standalone containers join
|
||||
docker network create --driver overlay --attachable <network>
|
||||
|
||||
List networks:
|
||||
|
||||
```bash
|
||||
# list networks
|
||||
docker network ls
|
||||
|
||||
# attached services and containers
|
||||
docker network inspect <network>
|
||||
|
||||
# attach at creation time
|
||||
docker service create --name <service> --network <network> <image>
|
||||
|
||||
# attach an existing service
|
||||
docker service update --network-add <network> <service>
|
||||
|
||||
# detach it again
|
||||
docker service update --network-rm <network> <service>
|
||||
```
|
||||
|
||||
Inspect a network (shows attached services and containers):
|
||||
### Publishing ports
|
||||
|
||||
```bash
|
||||
docker network inspect <network-name>
|
||||
# routing mesh (ingress)
|
||||
docker service create --publish 8080:80 --name web nginx
|
||||
|
||||
# host port, no mesh
|
||||
docker service create --publish mode=host,target=80,published=8080 --name web nginx
|
||||
```
|
||||
|
||||
Attach a service to a network at create time:
|
||||
|
||||
```bash
|
||||
docker service create --name <service-name> --network <network-name> <image>
|
||||
```
|
||||
|
||||
Attach an existing service to an additional network:
|
||||
|
||||
```bash
|
||||
docker service update --network-add <network-name> <service-name>
|
||||
```
|
||||
| Mode | Behaviour |
|
||||
|---------------------|----------------------------------------------------------------------------|
|
||||
| `ingress` (default) | Every node accepts the port and load-balances to the tasks |
|
||||
| `host` | The port is published only on nodes running a task — no mesh, no extra hop |
|
||||
|
||||
## 🔄 Rolling Updates & Rollback
|
||||
|
||||
Configure update behavior at service create or update time:
|
||||
|
||||
```bash
|
||||
docker service create \
|
||||
--name web \
|
||||
@@ -313,164 +360,192 @@ docker service create \
|
||||
--update-failure-action rollback \
|
||||
--update-monitor 30s \
|
||||
nginx:1.25
|
||||
```
|
||||
|
||||
Trigger an update by changing the image:
|
||||
|
||||
```bash
|
||||
# trigger the rolling update
|
||||
docker service update --image nginx:1.26 web
|
||||
```
|
||||
|
||||
Roll back to the previous spec:
|
||||
|
||||
```bash
|
||||
# return to the previous definition
|
||||
docker service rollback web
|
||||
```
|
||||
|
||||
Update flags worth knowing:
|
||||
| Flag | Description |
|
||||
|---------------------------|------------------------------------------------------------------------------------------------------------------------------|
|
||||
| `--update-parallelism` | How many tasks are updated at once (`0` = all) |
|
||||
| `--update-delay` | Pause between batches |
|
||||
| `--update-failure-action` | `pause` (default), `continue` or `rollback` |
|
||||
| `--update-monitor` | How long a task is watched before it counts as healthy |
|
||||
| `--update-order` | `stop-first` (default) or `start-first` |
|
||||
| `--rollback-parallelism` | Same knobs for the rollback path (`--rollback-delay`, `--rollback-monitor`, `--rollback-failure-action`, `--rollback-order`) |
|
||||
|
||||
| Flag | Description |
|
||||
|------|-------------|
|
||||
| `--update-parallelism` | How many tasks to update at once |
|
||||
| `--update-delay` | Delay between batches |
|
||||
| `--update-failure-action` | `pause`, `continue`, or `rollback` on failure |
|
||||
| `--update-monitor` | Time to watch each task for failure |
|
||||
| `--update-order` | `stop-first` (default) or `start-first` |
|
||||
## 🩺 Health Checks
|
||||
|
||||
Swarm reschedules a task as soon as its health check fails, so an update never marks a broken image as healthy.
|
||||
|
||||
```bash
|
||||
docker service create \
|
||||
--name api \
|
||||
--health-cmd 'curl -f http://localhost:8080/health || exit 1' \
|
||||
--health-interval 10s \
|
||||
--health-timeout 3s \
|
||||
--health-retries 3 \
|
||||
--health-start-period 30s \
|
||||
myimage
|
||||
```
|
||||
|
||||
## 🔐 Secrets
|
||||
|
||||
Secrets are encrypted at rest in the Raft log and mounted into containers as in-memory files under `/run/secrets/`.
|
||||
|
||||
Create a secret from a file:
|
||||
Secrets are encrypted in the Raft log and mounted into the container as in-memory files under `/run/secrets/`.
|
||||
|
||||
```bash
|
||||
# create from a file
|
||||
docker secret create db_password ./db_password.txt
|
||||
```
|
||||
|
||||
Create a secret from stdin:
|
||||
|
||||
```bash
|
||||
# create from stdin
|
||||
echo "s3cr3t" | docker secret create db_password -
|
||||
```
|
||||
|
||||
List secrets:
|
||||
|
||||
```bash
|
||||
# list secrets
|
||||
docker secret ls
|
||||
```
|
||||
|
||||
Inspect a secret (does **not** reveal the value):
|
||||
|
||||
```bash
|
||||
# metadata only — never the value
|
||||
docker secret inspect db_password
|
||||
```
|
||||
|
||||
Attach a secret to a service:
|
||||
|
||||
```bash
|
||||
# attach a secret; it appears at /run/secrets/db_password
|
||||
docker service create --name db --secret db_password postgres:16
|
||||
|
||||
# ...or mount it at a custom path with a fixed mode
|
||||
docker service create --name db \
|
||||
--secret source=db_password,target=/run/secrets/pg_pw,mode=0400 postgres:16
|
||||
|
||||
# only possible when no service uses it
|
||||
docker secret rm db_password
|
||||
```
|
||||
|
||||
Remove a secret (only if no service uses it):
|
||||
Rotating a secret — create the new one, swap it, then drop the old one:
|
||||
|
||||
```bash
|
||||
echo "n3w-s3cr3t" | docker secret create db_password_v2 -
|
||||
|
||||
docker service update \
|
||||
--secret-rm db_password \
|
||||
--secret-add source=db_password_v2,target=db_password \
|
||||
db
|
||||
|
||||
docker secret rm db_password
|
||||
```
|
||||
|
||||
## 🔑 Configs
|
||||
|
||||
Configs are like secrets but **not encrypted at rest** — meant for non-sensitive config files (nginx.conf, etc.).
|
||||
|
||||
Create a config:
|
||||
Configs work like secrets but are **not encrypted at rest** — use them for non-sensitive files such as `nginx.conf`.
|
||||
|
||||
```bash
|
||||
# create a config
|
||||
docker config create my_nginx_conf ./nginx.conf
|
||||
```
|
||||
|
||||
List configs:
|
||||
|
||||
```bash
|
||||
# list configs
|
||||
docker config ls
|
||||
```
|
||||
|
||||
Attach a config to a service at a specific path:
|
||||
# metadata and the stored content
|
||||
docker config inspect my_nginx_conf
|
||||
|
||||
```bash
|
||||
# mount a config at a specific path
|
||||
docker service create \
|
||||
--name web \
|
||||
--config source=my_nginx_conf,target=/etc/nginx/nginx.conf \
|
||||
nginx
|
||||
```
|
||||
|
||||
Remove a config:
|
||||
# swap in a new version of the config
|
||||
docker service update \
|
||||
--config-rm my_nginx_conf \
|
||||
--config-add source=my_nginx_conf_v2,target=/etc/nginx/nginx.conf \
|
||||
web
|
||||
|
||||
```bash
|
||||
# only when no service uses it
|
||||
docker config rm my_nginx_conf
|
||||
```
|
||||
|
||||
## 📚 Stacks
|
||||
|
||||
A **stack** is a group of related services deployed together from a Compose file. This is the standard production workflow for Swarm.
|
||||
|
||||
Deploy (or update) a stack:
|
||||
A **stack** is a group of related services deployed together from a Compose file — the standard production workflow for Swarm.
|
||||
|
||||
```bash
|
||||
# deploy or update a stack
|
||||
docker stack deploy -c stack.yaml mystack
|
||||
```
|
||||
|
||||
List stacks:
|
||||
# pass your registry credentials along
|
||||
docker stack deploy -c stack.yaml --with-registry-auth mystack
|
||||
|
||||
```bash
|
||||
# remove services no longer in the file
|
||||
docker stack deploy -c stack.yaml --prune mystack
|
||||
|
||||
# list stacks
|
||||
docker stack ls
|
||||
```
|
||||
|
||||
List services in a stack:
|
||||
|
||||
```bash
|
||||
# services of a stack
|
||||
docker stack services mystack
|
||||
```
|
||||
|
||||
List tasks in a stack:
|
||||
|
||||
```bash
|
||||
# tasks of a stack
|
||||
docker stack ps mystack
|
||||
```
|
||||
|
||||
Remove a stack:
|
||||
# tasks of a stack, with full error messages
|
||||
docker stack ps --no-trunc mystack
|
||||
|
||||
```bash
|
||||
# print the merged, resolved file
|
||||
docker stack config -c stack.yaml
|
||||
|
||||
# remove the whole stack
|
||||
docker stack rm mystack
|
||||
```
|
||||
|
||||
Stacks honor the `deploy:` block in Compose files (replicas, placement, restart policy, update config). `build:`, top-level `restart:`, and `depends_on:` are ignored.
|
||||
Stacks honour the `deploy:` block of the Compose file (replicas, placement, resources, restart policy, update config). `build:`, top-level `restart:` and `depends_on:` are ignored.
|
||||
|
||||
## 🧯 Troubleshooting
|
||||
|
||||
```bash
|
||||
# the error column tells you why a task died
|
||||
docker service ps --no-trunc <service>
|
||||
|
||||
# application output of every task
|
||||
docker service logs --tail 200 <service>
|
||||
|
||||
# is a node Down or Unreachable?
|
||||
docker node ls
|
||||
|
||||
# what was actually deployed
|
||||
docker service inspect --pretty <service>
|
||||
|
||||
# live stream of orchestration events
|
||||
docker events --filter type=service
|
||||
```
|
||||
|
||||
| Symptom | Usual cause |
|
||||
|--------------------------------------|--------------------------------------------------------------------------------------|
|
||||
| Task stuck in `PENDING` | No node satisfies the constraints, ports, or resource reservations |
|
||||
| `no suitable node` | Placement constraint or `--replicas-max-per-node` cannot be met |
|
||||
| Task loops `FAILED` → `STARTING` | The container exits immediately — read `docker service logs` |
|
||||
| `REJECTED` with an image error | The image is missing on that node; use `--with-registry-auth` for private registries |
|
||||
| Service unreachable from other nodes | Ports `7946`/`4789` blocked, or the services are not on the same overlay network |
|
||||
|
||||
## 🛠 Common Flags Reference
|
||||
|
||||
### `docker service create` / `update`
|
||||
### `docker service create` / `docker service update`
|
||||
|
||||
| Flag | Description |
|
||||
|------|-------------|
|
||||
| `--replicas` | Number of service replicas |
|
||||
| `--mode` | `replicated` (default) or `global` |
|
||||
| `--publish` | Port mapping (`<host>:<container>`) |
|
||||
| `--mount` | Volume mounting |
|
||||
| `--network` | Attach to a network |
|
||||
| `--constraint` | Node placement rule (e.g. `node.role==worker`) |
|
||||
| `--placement-pref` | Spread tasks (e.g. `spread=node.labels.zone`) |
|
||||
| `--limit-cpu` / `--limit-memory` | Hard resource limits |
|
||||
| `--reserve-cpu` / `--reserve-memory` | Reserved resources |
|
||||
| `--secret` / `--config` | Attach a secret or config |
|
||||
| `--env` / `--env-file` | Environment variables |
|
||||
| `--restart-condition` | `none`, `on-failure`, or `any` |
|
||||
|
||||
### Update-specific flags
|
||||
|
||||
| Flag | Description |
|
||||
|------|-------------|
|
||||
| `--update-parallelism` | Tasks updated in parallel |
|
||||
| `--update-delay` | Delay between update batches |
|
||||
| `--update-failure-action` | `pause`, `continue`, `rollback` |
|
||||
| `--update-order` | `stop-first` or `start-first` |
|
||||
| Flag | Description |
|
||||
|--------------------------------------|------------------------------------------------------------------|
|
||||
| `--replicas` | Number of tasks for a replicated service |
|
||||
| `--mode` | `replicated` (default), `global`, `replicated-job`, `global-job` |
|
||||
| `--publish` | Port mapping (`<published>:<target>`, or `mode=host,...`) |
|
||||
| `--mount` | Attach a volume or bind mount |
|
||||
| `--network` | Attach to an overlay network |
|
||||
| `--constraint` | Placement rule, e.g. `node.role==worker` |
|
||||
| `--placement-pref` | Spread tasks, e.g. `spread=node.labels.zone` |
|
||||
| `--replicas-max-per-node` | Cap the tasks of this service per node |
|
||||
| `--limit-cpu` / `--limit-memory` | Hard resource limits |
|
||||
| `--reserve-cpu` / `--reserve-memory` | Resources reserved for scheduling |
|
||||
| `--secret` / `--config` | Attach a secret or a config |
|
||||
| `--env` / `--env-file` | Environment variables |
|
||||
| `--restart-condition` | `none`, `on-failure` or `any` (default) |
|
||||
| `--health-cmd` and friends | Override the image health check |
|
||||
| `--with-registry-auth` | Forward registry credentials to the nodes |
|
||||
|
||||
## 📄 Example: Production-style Web Service
|
||||
|
||||
@@ -482,11 +557,14 @@ docker service create \
|
||||
--network webnet \
|
||||
--constraint 'node.role==worker' \
|
||||
--constraint 'node.labels.env==production' \
|
||||
--placement-pref 'spread=node.labels.zone' \
|
||||
--update-parallelism 1 \
|
||||
--update-delay 10s \
|
||||
--update-failure-action rollback \
|
||||
--update-order start-first \
|
||||
--limit-memory 512m \
|
||||
--reserve-memory 256m \
|
||||
--restart-condition any \
|
||||
nginx:1.25
|
||||
```
|
||||
|
||||
@@ -496,4 +574,5 @@ docker service create \
|
||||
- [Swarm services](https://docs.docker.com/engine/swarm/services/)
|
||||
- [Manage secrets](https://docs.docker.com/engine/swarm/secrets/)
|
||||
- [Manage configs](https://docs.docker.com/engine/swarm/configs/)
|
||||
- [Stack file reference (Compose Spec `deploy:`)](https://docs.docker.com/compose/compose-file/deploy/)
|
||||
- [Stack file reference (Compose Spec `deploy:`)](https://docs.docker.com/reference/compose-file/deploy/)
|
||||
- [`docker service create` CLI reference](https://docs.docker.com/reference/cli/docker/service/create/)
|
||||
@@ -1,258 +1,277 @@
|
||||
A **Dockerfile** is a script composed of instructions to build a Docker image. Each instruction creates a layer in the image.
|
||||
A **Dockerfile** is the recipe for an image: each instruction runs in order and most of them add a layer. Layers are cached, so the order you write them in decides how fast your rebuilds are.
|
||||
|
||||
## 📦 Basic Structure
|
||||
|
||||
```Dockerfile
|
||||
# Comment
|
||||
INSTRUCTION arguments
|
||||
```
|
||||
|
||||
## 🚀 Core Instructions
|
||||
|
||||
### `FROM`
|
||||
Specifies the base image.
|
||||
|
||||
```Dockerfile
|
||||
FROM ubuntu:20.04
|
||||
```dockerfile
|
||||
# syntax=docker/dockerfile:1
|
||||
FROM node:22-alpine
|
||||
```
|
||||
|
||||
### `LABEL`
|
||||
Adds metadata to the image.
|
||||
|
||||
```Dockerfile
|
||||
LABEL maintainer="you@example.com"
|
||||
LABEL version="1.0" description="My App"
|
||||
```
|
||||
|
||||
### `ENV`
|
||||
Sets environment variables.
|
||||
|
||||
```Dockerfile
|
||||
ENV NODE_ENV=production
|
||||
ENV PATH="/app/bin:$PATH"
|
||||
```
|
||||
|
||||
### `RUN`
|
||||
Executes commands in a shell during build.
|
||||
|
||||
```Dockerfile
|
||||
RUN apt-get update && apt-get install -y curl
|
||||
RUN npm install
|
||||
```
|
||||
|
||||
Use `RUN ["executable", "param1", "param2"]` for JSON array form.
|
||||
|
||||
### `COPY`
|
||||
Copies files from host to image.
|
||||
|
||||
```Dockerfile
|
||||
COPY . /app
|
||||
COPY config.json /app/config.json
|
||||
```
|
||||
|
||||
### `ADD`
|
||||
Similar to `COPY`, but supports remote URLs and auto-extracts archives.
|
||||
|
||||
```Dockerfile
|
||||
ADD https://example.com/file.tar.gz /app/
|
||||
ADD archive.zip /app/
|
||||
```
|
||||
|
||||
### `CMD`
|
||||
Sets default command to run when container starts.
|
||||
|
||||
```Dockerfile
|
||||
CMD ["node", "server.js"] # Preferred exec form
|
||||
CMD node server.js # Shell form
|
||||
```
|
||||
|
||||
Only one `CMD` allowed; later ones override earlier.
|
||||
|
||||
### `ENTRYPOINT`
|
||||
Configures a container to run as an executable.
|
||||
|
||||
```Dockerfile
|
||||
ENTRYPOINT ["python", "app.py"]
|
||||
```
|
||||
|
||||
Use with `CMD` to pass default arguments.
|
||||
|
||||
### `WORKDIR`
|
||||
Sets the working directory for subsequent instructions.
|
||||
|
||||
```Dockerfile
|
||||
WORKDIR /app
|
||||
COPY package*.json ./
|
||||
RUN npm ci
|
||||
COPY . .
|
||||
CMD ["node", "server.js"]
|
||||
```
|
||||
|
||||
### `EXPOSE`
|
||||
Documents the port the container listens on.
|
||||
> 💡 Start the file with `# syntax=docker/dockerfile:1` to get the current BuildKit frontend — that is what enables heredocs, cache mounts and build secrets.
|
||||
|
||||
```Dockerfile
|
||||
EXPOSE 80
|
||||
EXPOSE 443
|
||||
## 📋 Instruction Reference
|
||||
|
||||
| Instruction | Purpose |
|
||||
|---------------|---------------------------------------------------------------|
|
||||
| `FROM` | The base image, and the start of a build stage |
|
||||
| `ARG` | A build-time variable (`--build-arg`) |
|
||||
| `ENV` | An environment variable, kept in the final image |
|
||||
| `WORKDIR` | The working directory for the instructions that follow |
|
||||
| `COPY` | Copy files from the context (or another stage) into the image |
|
||||
| `ADD` | Like `COPY`, but also fetches URLs and unpacks archives |
|
||||
| `RUN` | Execute a command at build time and commit the result |
|
||||
| `CMD` | Default command, easily overridden at `docker run` |
|
||||
| `ENTRYPOINT` | The executable the container always runs |
|
||||
| `EXPOSE` | Document the port the app listens on |
|
||||
| `USER` | The user the following instructions and the container run as |
|
||||
| `VOLUME` | Declare a path that should be a mount point |
|
||||
| `HEALTHCHECK` | How Docker decides the container is healthy |
|
||||
| `LABEL` | Image metadata |
|
||||
| `SHELL` | Change the shell used by the shell form of `RUN`/`CMD` |
|
||||
| `STOPSIGNAL` | The signal sent to stop the container |
|
||||
| `ONBUILD` | An instruction that runs when this image is used as a base |
|
||||
|
||||
## 🏗 Base Image & Metadata
|
||||
|
||||
```dockerfile
|
||||
# pin the base image — 'latest' makes builds irreproducible
|
||||
FROM node:22-alpine
|
||||
|
||||
# name a stage so later stages can copy from it
|
||||
FROM golang:1.23 AS build
|
||||
|
||||
# an ARG before the first FROM can parameterise the base image itself
|
||||
ARG NODE_VERSION=22
|
||||
FROM node:${NODE_VERSION}-alpine
|
||||
|
||||
# metadata; the OCI keys are the conventional ones
|
||||
LABEL org.opencontainers.image.source="https://github.com/acme/app"
|
||||
LABEL org.opencontainers.image.description="Acme API"
|
||||
```
|
||||
|
||||
Note: This does not publish the port.
|
||||
## 📂 Files & Working Directory
|
||||
|
||||
### `VOLUME`
|
||||
Creates a mount point for persistent or shared data.
|
||||
```dockerfile
|
||||
# always set an absolute working directory
|
||||
WORKDIR /app
|
||||
|
||||
```Dockerfile
|
||||
# copy from the build context
|
||||
COPY package*.json ./
|
||||
COPY . .
|
||||
|
||||
# copy and set ownership in one step, no extra layer for chown
|
||||
COPY --chown=node:node . .
|
||||
|
||||
# copy with explicit permissions
|
||||
COPY --chmod=755 entrypoint.sh /usr/local/bin/
|
||||
|
||||
# copy from another stage, or straight from another image
|
||||
COPY --from=build /src/app /usr/local/bin/app
|
||||
COPY --from=nginx:alpine /etc/nginx/nginx.conf /etc/nginx/nginx.conf
|
||||
|
||||
# ADD unpacks archives and fetches URLs — prefer COPY unless you need that
|
||||
ADD archive.tar.gz /opt/
|
||||
ADD https://example.com/file.tar.gz /tmp/
|
||||
|
||||
# declare a mount point for data that should not live in the image
|
||||
VOLUME ["/data"]
|
||||
```
|
||||
|
||||
### `USER`
|
||||
Sets the user to run subsequent instructions.
|
||||
## ⚙️ Build Steps
|
||||
|
||||
```Dockerfile
|
||||
```dockerfile
|
||||
# shell form: runs through /bin/sh -c
|
||||
RUN apt-get update && apt-get install -y --no-install-recommends curl \
|
||||
&& rm -rf /var/lib/apt/lists/*
|
||||
|
||||
# exec form: no shell, so no globbing or variable expansion
|
||||
RUN ["npm", "ci", "--omit=dev"]
|
||||
|
||||
# heredoc: a readable multi-line script (BuildKit)
|
||||
RUN <<EOF
|
||||
set -eux
|
||||
apk add --no-cache curl
|
||||
adduser -D appuser
|
||||
EOF
|
||||
|
||||
# cache mount: keep the package cache between builds
|
||||
RUN --mount=type=cache,target=/root/.npm npm ci
|
||||
|
||||
# secret mount: use a credential without baking it into a layer
|
||||
RUN --mount=type=secret,id=npmrc,target=/root/.npmrc npm ci
|
||||
|
||||
# bind mount: read from the context without copying it in
|
||||
RUN --mount=type=bind,source=package.json,target=package.json npm ci
|
||||
```
|
||||
|
||||
## 🚀 Runtime Configuration
|
||||
|
||||
```dockerfile
|
||||
# environment variables are visible to the running container
|
||||
ENV NODE_ENV=production
|
||||
ENV PATH="/app/bin:$PATH"
|
||||
|
||||
# document the port — this does not publish it, -p does
|
||||
EXPOSE 3000
|
||||
|
||||
# drop root as early as you can
|
||||
RUN adduser -D appuser
|
||||
USER appuser
|
||||
|
||||
# tell Docker how to check the app is alive
|
||||
HEALTHCHECK --interval=30s --timeout=3s --start-period=20s --retries=3 \
|
||||
CMD wget -qO- http://localhost:3000/health || exit 1
|
||||
|
||||
# the signal that stops the process cleanly
|
||||
STOPSIGNAL SIGTERM
|
||||
|
||||
# the command the container runs
|
||||
CMD ["node", "server.js"]
|
||||
```
|
||||
|
||||
### `ARG`
|
||||
Defines build-time variables.
|
||||
## ⚔️ CMD vs ENTRYPOINT
|
||||
|
||||
```Dockerfile
|
||||
ARG VERSION=1.0
|
||||
RUN echo $VERSION
|
||||
| | `CMD` | `ENTRYPOINT` |
|
||||
|--------------------------------|-------------------------------------------------|------------------------------------------|
|
||||
| Purpose | Default arguments, or the whole default command | The executable the image is built around |
|
||||
| `docker run <image> other-cmd` | Replaced entirely | Still runs; the argument is appended |
|
||||
| Override flag | — | `--entrypoint` |
|
||||
| Typical use | `CMD ["node", "server.js"]` | `ENTRYPOINT ["python", "app.py"]` |
|
||||
|
||||
```dockerfile
|
||||
# together: ENTRYPOINT is the binary, CMD holds the default arguments
|
||||
ENTRYPOINT ["python", "app.py"]
|
||||
CMD ["--port", "8000"]
|
||||
|
||||
# docker run image → python app.py --port 8000
|
||||
# docker run image --port 9000 → python app.py --port 9000
|
||||
```
|
||||
|
||||
Use `--build-arg VERSION=2.0` during `docker build`.
|
||||
> ⚠️ Prefer the **exec form** (`["cmd", "arg"]`). The shell form wraps your process in `/bin/sh -c`, which swallows `SIGTERM` and turns graceful shutdown into a ten-second kill.
|
||||
|
||||
### `ONBUILD`
|
||||
Triggers instructions when the image is used as a base.
|
||||
## 🧱 Multi-Stage Builds
|
||||
|
||||
```Dockerfile
|
||||
ONBUILD COPY . /app
|
||||
Several `FROM` instructions in one file: build with a fat toolchain, ship only the artifact.
|
||||
|
||||
```dockerfile
|
||||
# stage 1 — build
|
||||
FROM node:22-alpine AS build
|
||||
WORKDIR /app
|
||||
COPY package*.json ./
|
||||
RUN npm ci
|
||||
COPY . .
|
||||
RUN npm run build
|
||||
|
||||
# stage 2 — the image that actually ships
|
||||
FROM nginx:alpine
|
||||
COPY --from=build /app/dist /usr/share/nginx/html
|
||||
EXPOSE 80
|
||||
CMD ["nginx", "-g", "daemon off;"]
|
||||
```
|
||||
|
||||
```dockerfile
|
||||
# a compiled binary on a near-empty base
|
||||
FROM golang:1.23 AS build
|
||||
WORKDIR /src
|
||||
COPY . .
|
||||
RUN CGO_ENABLED=0 go build -o /out/app ./cmd/app
|
||||
|
||||
FROM gcr.io/distroless/static-debian12
|
||||
COPY --from=build /out/app /app
|
||||
USER nonroot:nonroot
|
||||
ENTRYPOINT ["/app"]
|
||||
```
|
||||
|
||||
```bash
|
||||
# build only up to a named stage — handy for a test or lint stage
|
||||
docker build --target build -t app:build .
|
||||
```
|
||||
|
||||
## 📁 .dockerignore
|
||||
|
||||
Everything in the build context is sent to the daemon and busts the cache. Exclude what the build does not need:
|
||||
|
||||
```plaintext
|
||||
.git
|
||||
node_modules
|
||||
dist
|
||||
*.log
|
||||
.env
|
||||
Dockerfile
|
||||
.dockerignore
|
||||
```
|
||||
|
||||
## 🧠 Best Practices
|
||||
|
||||
- Use minimal base images (e.g., `alpine`) for smaller size.
|
||||
- Combine `RUN` commands to reduce layers.
|
||||
- Use `.dockerignore` to exclude unnecessary files.
|
||||
- Prefer `COPY` over `ADD` unless you need archive extraction.
|
||||
- Use `ENTRYPOINT` for fixed commands and `CMD` for arguments.
|
||||
- Avoid hardcoding secrets or credentials.
|
||||
- **Order by how often things change**: base image, then dependency manifests, then `npm ci`/`pip install`, then the source. A source edit should not reinstall dependencies.
|
||||
- **Pin versions** — `node:22.11-alpine`, not `node:latest`.
|
||||
- **One `RUN` per logical step**, chaining installs and cleanup in the same layer (`rm -rf /var/lib/apt/lists/*`), or the deleted files still weigh on the image.
|
||||
- **Never bake secrets in.** A deleted file stays in the layer below; use `--mount=type=secret` or build arguments that never reach the final stage.
|
||||
- **Run as a non-root user** and keep the final stage as small as you can (`alpine`, `distroless`, `scratch`).
|
||||
- **Use `COPY`, not `ADD`,** unless you specifically want URL fetching or archive extraction.
|
||||
- **Combine with `--platform`** for multi-arch: `docker buildx build --platform linux/amd64,linux/arm64`.
|
||||
|
||||
## 🧪 Example Dockerfile
|
||||
|
||||
```Dockerfile
|
||||
FROM node:22-alpine
|
||||
|
||||
LABEL maintainer="guillaume@example.com"
|
||||
|
||||
WORKDIR /app
|
||||
|
||||
COPY package*.json ./
|
||||
RUN npm install
|
||||
|
||||
COPY . .
|
||||
|
||||
EXPOSE 3000
|
||||
|
||||
ENV NODE_ENV=production
|
||||
|
||||
CMD ["node", "index.js"]
|
||||
```
|
||||
|
||||
## 🛠️ Build & Run
|
||||
## 🛠 Building & Running
|
||||
|
||||
```bash
|
||||
# Build image
|
||||
# build from the Dockerfile in this directory
|
||||
docker build -t my-app .
|
||||
|
||||
# Run container
|
||||
# build without the cache
|
||||
docker build --no-cache -t my-app .
|
||||
|
||||
# a Dockerfile somewhere else
|
||||
docker build -f docker/Dockerfile.prod -t my-app .
|
||||
|
||||
# pass a build argument
|
||||
docker build --build-arg NODE_VERSION=20 -t my-app .
|
||||
|
||||
# pass a secret (never becomes a layer)
|
||||
docker build --secret id=npmrc,src=$HOME/.npmrc -t my-app .
|
||||
|
||||
# build for another architecture
|
||||
docker buildx build --platform linux/arm64 -t my-app .
|
||||
|
||||
# inspect the layers of the result
|
||||
docker history my-app
|
||||
|
||||
# run it
|
||||
docker run -p 3000:3000 my-app
|
||||
```
|
||||
|
||||
## 📁 .dockerignore Example
|
||||
## 📄 Full Example
|
||||
|
||||
```plaintext
|
||||
node_modules
|
||||
*.log
|
||||
Dockerfile
|
||||
.git
|
||||
```
|
||||
|
||||
## 🏗️ Multistage Builds
|
||||
|
||||
**Multistage builds** allow you to use multiple `FROM` statements in one Dockerfile to optimize image size and separate build dependencies from runtime.
|
||||
|
||||
### 🎯 Why Use Multistage Builds?
|
||||
|
||||
- Reduce final image size by excluding build tools and intermediate files.
|
||||
- Improve security by minimizing attack surface.
|
||||
- Keep Dockerfiles clean and maintainable.
|
||||
|
||||
### 🧱 Basic Syntax
|
||||
|
||||
```Dockerfile
|
||||
# Stage 1: Build
|
||||
FROM node:22-alpine AS builder
|
||||
```dockerfile
|
||||
# syntax=docker/dockerfile:1
|
||||
FROM node:22-alpine AS deps
|
||||
WORKDIR /app
|
||||
COPY package*.json ./
|
||||
RUN npm install
|
||||
RUN --mount=type=cache,target=/root/.npm npm ci
|
||||
|
||||
FROM node:22-alpine AS build
|
||||
WORKDIR /app
|
||||
COPY --from=deps /app/node_modules ./node_modules
|
||||
COPY . .
|
||||
RUN npm run build
|
||||
|
||||
# Stage 2: Production
|
||||
FROM nginx:alpine
|
||||
COPY --from=builder /app/dist /usr/share/nginx/html
|
||||
```
|
||||
|
||||
### 🏷️ Named Stages
|
||||
|
||||
You can name each stage using `AS <name>` and reference it later with `--from=<name>`.
|
||||
|
||||
```Dockerfile
|
||||
FROM golang:1.21 AS build
|
||||
WORKDIR /src
|
||||
COPY . .
|
||||
RUN go build -o myapp
|
||||
|
||||
FROM alpine:latest
|
||||
COPY --from=build /src/myapp /usr/local/bin/myapp
|
||||
ENTRYPOINT ["myapp"]
|
||||
```
|
||||
|
||||
### 📦 Copying Artifacts
|
||||
|
||||
Use `COPY --from=<stage>` to copy files from one stage to another.
|
||||
|
||||
```Dockerfile
|
||||
COPY --from=builder /app/output /app/output
|
||||
```
|
||||
|
||||
You can copy:
|
||||
- Files
|
||||
- Directories
|
||||
- Binaries
|
||||
- Configs
|
||||
|
||||
### 🧼 Clean Final Image
|
||||
|
||||
Multistage builds help you avoid bloated images:
|
||||
|
||||
```Dockerfile
|
||||
# Without multistage: includes compilers, source code, etc.
|
||||
# With multistage: only includes runtime essentials
|
||||
```
|
||||
|
||||
### 🧪 Real-World Example: React App
|
||||
|
||||
```Dockerfile
|
||||
# Build stage
|
||||
FROM node:22 AS build
|
||||
FROM node:22-alpine
|
||||
LABEL org.opencontainers.image.source="https://github.com/acme/app"
|
||||
ENV NODE_ENV=production
|
||||
WORKDIR /app
|
||||
COPY package*.json ./
|
||||
RUN npm install
|
||||
COPY . .
|
||||
RUN npm run build
|
||||
COPY --from=deps --chown=node:node /app/node_modules ./node_modules
|
||||
COPY --from=build --chown=node:node /app/dist ./dist
|
||||
USER node
|
||||
EXPOSE 3000
|
||||
HEALTHCHECK --interval=30s CMD wget -qO- http://localhost:3000/health || exit 1
|
||||
CMD ["node", "dist/server.js"]
|
||||
```
|
||||
|
||||
# Serve stage
|
||||
FROM nginx:alpine
|
||||
COPY --from=build /app/build /usr/share/nginx/html
|
||||
EXPOSE 80
|
||||
CMD ["nginx", "-g", "daemon off;"]
|
||||
```
|
||||
## 📚 Resources
|
||||
|
||||
- [Dockerfile reference](https://docs.docker.com/reference/dockerfile/)
|
||||
- [Building best practices](https://docs.docker.com/build/building/best-practices/)
|
||||
- [BuildKit mounts (cache, secret, bind)](https://docs.docker.com/build/cache/optimize/)
|
||||
- [Multi-stage builds](https://docs.docker.com/build/building/multi-stage/)
|
||||
@@ -1,247 +1,576 @@
|
||||
## Configuration
|
||||
**Git** is a distributed version control system: every clone is a full repository with its own history, branches and tags. This is a quick reference for the commands that come up daily — placeholders are written as `<file>`, `<branch>`, `<commit>` and `<url>`.
|
||||
|
||||
Set the global config
|
||||
```shell
|
||||
git config --global user.name "[name]"
|
||||
git config --global user.email "[email]"
|
||||
> 💡 Every command below has its own help (`git rebase --help`), and the destructive ones (`clean`, `push`, `rm`, `mv`) take `-n` / `--dry-run` to preview what would happen.
|
||||
|
||||
## ⚙️ Configuration
|
||||
|
||||
```bash
|
||||
# identity used on every commit
|
||||
git config --global user.name "<name>"
|
||||
git config --global user.email "<email>"
|
||||
|
||||
# name of the branch created by git init
|
||||
git config --global init.defaultBranch main
|
||||
|
||||
# editor for messages, rebases, ...
|
||||
git config --global core.editor "code --wait"
|
||||
|
||||
# rebase instead of merging on pull
|
||||
git config --global pull.rebase true
|
||||
|
||||
# plain 'git push' works on new branches
|
||||
git config --global push.autoSetupRemote true
|
||||
|
||||
# every setting and the file it came from
|
||||
git config --list --show-origin
|
||||
|
||||
# override for this repository only
|
||||
git config --local user.email "<work-email>"
|
||||
```
|
||||
|
||||
## Get started
|
||||
Handy aliases:
|
||||
|
||||
Create a git repository
|
||||
```shell
|
||||
```bash
|
||||
git config --global alias.st status
|
||||
|
||||
git config --global alias.co checkout
|
||||
|
||||
git config --global alias.lg "log --oneline --graph --decorate --all"
|
||||
|
||||
git config --global alias.last "log -1 --stat"
|
||||
```
|
||||
|
||||
## 🚀 Get Started
|
||||
|
||||
```bash
|
||||
# create a repository in the current directory
|
||||
git init
|
||||
|
||||
# clone a remote repository
|
||||
git clone <url>
|
||||
|
||||
# clone into a specific directory
|
||||
git clone <url> <directory>
|
||||
|
||||
# clone and check out one branch
|
||||
git clone --branch <branch> <url>
|
||||
|
||||
# shallow clone, latest commit only
|
||||
git clone --depth 1 <url>
|
||||
|
||||
# clone including submodules
|
||||
git clone --recurse-submodules <url>
|
||||
|
||||
# what is staged, changed, untracked
|
||||
git status
|
||||
|
||||
# the same, in short format
|
||||
git status -sb
|
||||
```
|
||||
|
||||
Pull a git repository
|
||||
```shell
|
||||
git pull [url]
|
||||
```
|
||||
## ➕ Staging
|
||||
|
||||
Clone an existing git repository
|
||||
```shell
|
||||
git clone [url]
|
||||
```
|
||||
```bash
|
||||
# stage a specific file
|
||||
git add <file>
|
||||
|
||||
## Staging
|
||||
|
||||
Stage specific files
|
||||
```shell
|
||||
git add [file]
|
||||
```
|
||||
|
||||
Stage all changes
|
||||
```shell
|
||||
# stage everything under the current directory
|
||||
git add .
|
||||
|
||||
# stage everything in the repository, deletions included
|
||||
git add -A
|
||||
|
||||
# interactively stage individual hunks
|
||||
git add -p <file>
|
||||
|
||||
# stage modifications and deletions, not new files
|
||||
git add -u
|
||||
|
||||
# unstage, keep the changes in the working tree
|
||||
git restore --staged <file>
|
||||
|
||||
# stop tracking a file, keep it on disk
|
||||
git rm --cached <file>
|
||||
|
||||
# rename and stage in one step
|
||||
git mv <old> <new>
|
||||
|
||||
# which .gitignore rule is hiding this file?
|
||||
git check-ignore -v <file>
|
||||
```
|
||||
|
||||
Interactively stage parts of a file (hunks)
|
||||
```shell
|
||||
git add -p [file]
|
||||
```
|
||||
## ✅ Commit
|
||||
|
||||
Unstage a file but keep changes
|
||||
```shell
|
||||
git restore --staged [file]
|
||||
```
|
||||
```bash
|
||||
# commit what is staged
|
||||
git commit -m "<message>"
|
||||
|
||||
## Commit
|
||||
# stage tracked changes and commit
|
||||
git commit -am "<message>"
|
||||
|
||||
Commit all tracked changes
|
||||
```shell
|
||||
git commit -am "[commit message]"
|
||||
```
|
||||
# rewrite the last commit (message + content)
|
||||
git commit --amend
|
||||
|
||||
Add new modifications to the last commit
|
||||
```shell
|
||||
# amend without changing the message
|
||||
git commit --amend --no-edit
|
||||
|
||||
# fixup commit, squashed later by --autosquash
|
||||
git commit --fixup <commit>
|
||||
|
||||
# add a Signed-off-by trailer
|
||||
git commit -s -m "<message>"
|
||||
|
||||
# empty commit, e.g. to trigger CI
|
||||
git commit --allow-empty -m "<message>"
|
||||
```
|
||||
|
||||
## Branches
|
||||
> ⚠️ `--amend` rewrites history. Only amend commits that have not been pushed to a shared branch.
|
||||
|
||||
Lists all local branches in your repository (use -a for local and remote branches)
|
||||
```shell
|
||||
## 🌿 Branches
|
||||
|
||||
```bash
|
||||
# list local branches
|
||||
git branch
|
||||
|
||||
# local and remote-tracking branches
|
||||
git branch -a
|
||||
|
||||
# last commit and upstream of each branch
|
||||
git branch -vv
|
||||
|
||||
# switch to an existing branch
|
||||
git switch <branch>
|
||||
|
||||
# create a branch and switch to it
|
||||
git switch -c <branch>
|
||||
|
||||
# jump back to the previous branch
|
||||
git switch -
|
||||
|
||||
# older equivalent of switch -c
|
||||
git checkout -b <branch>
|
||||
|
||||
# branch off a specific commit or tag
|
||||
git switch -c <branch> <commit>
|
||||
|
||||
# rename a branch
|
||||
git branch -m <old> <new>
|
||||
|
||||
# delete a merged branch
|
||||
git branch -d <branch>
|
||||
|
||||
# force-delete an unmerged branch
|
||||
git branch -D <branch>
|
||||
|
||||
# branches already merged into HEAD (safe to delete)
|
||||
git branch --merged
|
||||
```
|
||||
|
||||
Switch to an existing branch
|
||||
```shell
|
||||
git switch [branch name]
|
||||
## 🔀 Merging & Rebasing
|
||||
|
||||
```bash
|
||||
# merge a branch into the current one
|
||||
git merge <branch>
|
||||
|
||||
# always create a merge commit
|
||||
git merge --no-ff <branch>
|
||||
|
||||
# bring in the changes as one staged change set
|
||||
git merge --squash <branch>
|
||||
|
||||
# bail out of a conflicted merge
|
||||
git merge --abort
|
||||
|
||||
# replay the current branch on top of another
|
||||
git rebase <branch>
|
||||
|
||||
# interactively squash, reword, drop, reorder commits
|
||||
git rebase -i HEAD~5
|
||||
|
||||
# apply the --fixup commits automatically
|
||||
git rebase -i --autosquash <base>
|
||||
|
||||
# resume after resolving conflicts
|
||||
git rebase --continue
|
||||
|
||||
# drop the conflicting commit and continue
|
||||
git rebase --skip
|
||||
|
||||
# return to the state before the rebase
|
||||
git rebase --abort
|
||||
|
||||
# resolve conflicts with the configured merge tool
|
||||
git mergetool
|
||||
```
|
||||
|
||||
Create a new branch
|
||||
```shell
|
||||
git checkout -b [branch name]
|
||||
Resolving a conflict:
|
||||
|
||||
```bash
|
||||
# 1. see which files conflict
|
||||
git status
|
||||
|
||||
# 2. edit them and remove the <<<<<<< ======= >>>>>>> markers
|
||||
|
||||
# 3. mark each resolved file
|
||||
git add <file>
|
||||
|
||||
# 4. resume (during a merge: git merge --continue)
|
||||
git rebase --continue
|
||||
```
|
||||
|
||||
## Stashing
|
||||
> ⚠️ Rebasing rewrites commits. Never rebase a branch other people are already working on.
|
||||
|
||||
Save uncommitted changes for later
|
||||
```shell
|
||||
git stash
|
||||
```
|
||||
## 🌍 Remotes & Syncing
|
||||
|
||||
List all stashes
|
||||
```shell
|
||||
git stash list
|
||||
```
|
||||
|
||||
Apply most recent stash and remove it from the list
|
||||
```shell
|
||||
git stash pop
|
||||
```
|
||||
|
||||
Apply a specific stash without removing it
|
||||
```shell
|
||||
git stash apply stash@{2}
|
||||
```
|
||||
|
||||
## Inspecting
|
||||
|
||||
Show changes between working directory and staging
|
||||
```shell
|
||||
git diff
|
||||
```
|
||||
|
||||
Show changes between staging and last commit
|
||||
```shell
|
||||
git diff --staged
|
||||
```
|
||||
|
||||
Show commit history for a specific file
|
||||
```shell
|
||||
git log --follow [file]
|
||||
```
|
||||
|
||||
Show who changed each line in a file
|
||||
```shell
|
||||
git blame [file]
|
||||
```
|
||||
|
||||
## Remote
|
||||
|
||||
Add a remote repository
|
||||
```shell
|
||||
git remote add origin [url]
|
||||
```
|
||||
|
||||
List remote repositories
|
||||
```shell
|
||||
```bash
|
||||
# list remotes and their URLs
|
||||
git remote -v
|
||||
```
|
||||
|
||||
Push a branch and set upstream tracking
|
||||
```shell
|
||||
git push -u origin [branch-name]
|
||||
```
|
||||
# add a remote
|
||||
git remote add origin <url>
|
||||
|
||||
Delete a remote branch
|
||||
```shell
|
||||
git push origin --delete [branch-name]
|
||||
```
|
||||
# point a remote somewhere else
|
||||
git remote set-url origin <url>
|
||||
|
||||
## Tags
|
||||
# rename a remote
|
||||
git remote rename <old> <new>
|
||||
|
||||
Create a tagged release
|
||||
```shell
|
||||
git tag -a v1.0.0 -m "Release v1.0.0"
|
||||
```
|
||||
# forget a remote
|
||||
git remote remove <name>
|
||||
|
||||
Push tags to remote
|
||||
```shell
|
||||
# download objects and refs, change nothing
|
||||
git fetch origin
|
||||
|
||||
# fetch everything, drop deleted remote branches
|
||||
git fetch --all --prune
|
||||
|
||||
# fetch + merge (or rebase) the upstream branch
|
||||
git pull
|
||||
|
||||
# replay local commits on top of the upstream
|
||||
git pull --rebase
|
||||
|
||||
# push the current branch
|
||||
git push
|
||||
|
||||
# push and set the upstream tracking branch
|
||||
git push -u origin <branch>
|
||||
|
||||
# force-push, but refuse to clobber new commits
|
||||
git push --force-with-lease
|
||||
|
||||
# delete a remote branch
|
||||
git push origin --delete <branch>
|
||||
|
||||
# push all tags
|
||||
git push origin --tags
|
||||
```
|
||||
|
||||
## I've made a mistake
|
||||
> ⚠️ Prefer `--force-with-lease` over `--force`: it aborts if someone else pushed in the meantime.
|
||||
|
||||
Change last commit message
|
||||
```shell
|
||||
git commit --amend
|
||||
## 📦 Stashing
|
||||
|
||||
```bash
|
||||
# shelve tracked changes and clean the working tree
|
||||
git stash
|
||||
|
||||
# include untracked files
|
||||
git stash -u
|
||||
|
||||
# stash specific paths with a label
|
||||
git stash push -m "<message>" <file>
|
||||
|
||||
# list every stash
|
||||
git stash list
|
||||
|
||||
# show a stash as a patch
|
||||
git stash show -p stash@{0}
|
||||
|
||||
# re-apply the newest stash and drop it
|
||||
git stash pop
|
||||
|
||||
# re-apply a specific stash, keep it in the list
|
||||
git stash apply stash@{2}
|
||||
|
||||
# create a branch from a stash and apply it
|
||||
git stash branch <branch>
|
||||
|
||||
# delete one stash
|
||||
git stash drop stash@{0}
|
||||
|
||||
# delete all stashes
|
||||
git stash clear
|
||||
```
|
||||
|
||||
Undo most recent commit and keep changes
|
||||
```shell
|
||||
git reset HEAD~1
|
||||
```
|
||||
## 🔍 Inspecting
|
||||
|
||||
Undo the `N` most recent commit and keep changes
|
||||
```shell
|
||||
git reset HEAD~N
|
||||
```
|
||||
```bash
|
||||
# the whole history as a compact graph
|
||||
git log --oneline --graph --decorate --all
|
||||
|
||||
Undo most recent commit and get rid of changes
|
||||
```shell
|
||||
git reset HEAD~1 --hard
|
||||
```
|
||||
# history of a file, with diffs
|
||||
git log -p <file>
|
||||
|
||||
Reset branch to remote state
|
||||
```shell
|
||||
git fetch origin
|
||||
git reset --hard origin/[branch-name]
|
||||
```
|
||||
# history of a file, following renames
|
||||
git log --follow <file>
|
||||
|
||||
Revert a commit by creating a new undo commit (safe for shared branches)
|
||||
```shell
|
||||
git revert [commit-hash]
|
||||
```
|
||||
# filter by date and author
|
||||
git log --since="2 weeks ago" --author=<name>
|
||||
|
||||
Discard all uncommitted changes in working directory
|
||||
```shell
|
||||
git restore .
|
||||
```
|
||||
# search commit messages
|
||||
git log --grep="<pattern>"
|
||||
|
||||
Recover a deleted branch or lost commit
|
||||
```shell
|
||||
# commits that added or removed a string
|
||||
git log -S"<string>"
|
||||
|
||||
# commits in b that are not in a
|
||||
git log <branch-a>..<branch-b>
|
||||
|
||||
# commit count per author
|
||||
git shortlog -sn
|
||||
|
||||
# a single commit with its diff
|
||||
git show <commit>
|
||||
|
||||
# working tree vs. index
|
||||
git diff
|
||||
|
||||
# index vs. last commit
|
||||
git diff --staged
|
||||
|
||||
# between two commits
|
||||
git diff HEAD~1 HEAD
|
||||
|
||||
# changes since the branches diverged
|
||||
git diff <branch-a>...<branch-b>
|
||||
|
||||
# summary of changed files
|
||||
git diff --stat
|
||||
|
||||
# who last touched every line
|
||||
git blame <file>
|
||||
|
||||
# blame a range of lines only
|
||||
git blame -L 10,40 <file>
|
||||
|
||||
# search the tracked files
|
||||
git grep "<pattern>"
|
||||
|
||||
# every position HEAD has had — your safety net
|
||||
git reflog
|
||||
git checkout -b [branch-name] [commit-hash]
|
||||
```
|
||||
|
||||
## Cherry-pick
|
||||
## 🍒 Cherry-pick
|
||||
|
||||
Apply a specific commit from another branch
|
||||
```shell
|
||||
git cherry-pick [commit-hash]
|
||||
```bash
|
||||
# apply one commit onto the current branch
|
||||
git cherry-pick <commit>
|
||||
|
||||
# apply a range of commits
|
||||
git cherry-pick <commit-a>^..<commit-b>
|
||||
|
||||
# apply without committing
|
||||
git cherry-pick -n <commit>
|
||||
|
||||
# after resolving conflicts
|
||||
git cherry-pick --continue
|
||||
|
||||
# undo the whole cherry-pick
|
||||
git cherry-pick --abort
|
||||
```
|
||||
|
||||
## Cleaning
|
||||
## 🏷 Tags
|
||||
|
||||
Remove untracked files (dry run first)
|
||||
```shell
|
||||
```bash
|
||||
# list tags
|
||||
git tag
|
||||
|
||||
# list matching tags
|
||||
git tag -l "v1.*"
|
||||
|
||||
# annotated tag on HEAD
|
||||
git tag -a v1.0.0 -m "Release v1.0.0"
|
||||
|
||||
# tag an older commit
|
||||
git tag -a v1.0.0 <commit>
|
||||
|
||||
# show a tag and its commit
|
||||
git show v1.0.0
|
||||
|
||||
# push one tag
|
||||
git push origin v1.0.0
|
||||
|
||||
# push all tags
|
||||
git push origin --tags
|
||||
|
||||
# delete a local tag
|
||||
git tag -d v1.0.0
|
||||
|
||||
# delete a remote tag
|
||||
git push origin --delete v1.0.0
|
||||
|
||||
# closest tag to the current commit
|
||||
git describe --tags
|
||||
```
|
||||
|
||||
## 🧯 I've Made a Mistake
|
||||
|
||||
```bash
|
||||
# fix the last commit message
|
||||
git commit --amend
|
||||
|
||||
# undo the last commit, keep the changes unstaged
|
||||
git reset HEAD~1
|
||||
|
||||
# undo the last n commits, keep the changes
|
||||
git reset HEAD~<n>
|
||||
|
||||
# undo the last commit and throw the changes away
|
||||
git reset --hard HEAD~1
|
||||
|
||||
# discard changes to one file
|
||||
git restore <file>
|
||||
|
||||
# discard every uncommitted change
|
||||
git restore .
|
||||
|
||||
# restore a file as it was at a commit
|
||||
git restore --source=<commit> <file>
|
||||
|
||||
# undo a commit with a new commit (safe when shared)
|
||||
git revert <commit>
|
||||
|
||||
# stage the revert without committing
|
||||
git revert -n <commit>
|
||||
|
||||
# make the local branch match the remote (fetch first)
|
||||
git reset --hard origin/<branch>
|
||||
|
||||
# find the lost commit...
|
||||
git reflog
|
||||
|
||||
# bring a lost commit back on a new branch
|
||||
git switch -c <branch> <commit>
|
||||
```
|
||||
|
||||
`git reset` modes at a glance:
|
||||
|
||||
| Mode | Moves `HEAD` | Index (staging) | Working tree | Use it to |
|
||||
|-----------|--------------|-----------------|--------------|----------------------------------------------|
|
||||
| `--soft` | ✅ | untouched | untouched | Recommit differently, keep everything staged |
|
||||
| `--mixed` | ✅ | reset | untouched | Unstage but keep the edits (default) |
|
||||
| `--hard` | ✅ | reset | **reset** | Throw the changes away entirely ⚠️ |
|
||||
|
||||
> 💡 Nothing committed is really lost for ~90 days: `git reflog` lists every commit `HEAD` pointed at, even on deleted branches.
|
||||
|
||||
## 🧹 Cleaning
|
||||
|
||||
```bash
|
||||
# dry run: what would be removed
|
||||
git clean -n
|
||||
```
|
||||
|
||||
Remove untracked files and directories
|
||||
```shell
|
||||
# remove untracked files and directories
|
||||
git clean -fd
|
||||
|
||||
# also remove ignored files (build output, node_modules)
|
||||
git clean -fdx
|
||||
|
||||
# compress and tidy the object database
|
||||
git gc
|
||||
|
||||
# drop unreachable objects
|
||||
git prune
|
||||
|
||||
# check the repository for corruption
|
||||
git fsck
|
||||
```
|
||||
|
||||
## Miscellaneous
|
||||
## 🐛 Debugging
|
||||
|
||||
Renaming the local master branch to main
|
||||
```shell
|
||||
git branch -m master main
|
||||
```
|
||||
|
||||
Checking log graph
|
||||
```shell
|
||||
git log --graph
|
||||
```
|
||||
|
||||
Checking log graph (merges only)
|
||||
```shell
|
||||
git log --graph --merges
|
||||
```
|
||||
|
||||
Tracking down a bad commit using binary search
|
||||
```shell
|
||||
```bash
|
||||
# begin a binary search for a bad commit
|
||||
git bisect start
|
||||
git bisect good 13c988d4f15e06bcdd0b0af290086a3079cdadb0
|
||||
git bisect bad ca82a6dff817ec66f44342007202690a93763949
|
||||
|
||||
# a commit that is broken (often HEAD)
|
||||
git bisect bad <commit>
|
||||
|
||||
# a commit that was fine
|
||||
git bisect good <commit>
|
||||
|
||||
# let a test script decide automatically
|
||||
git bisect run <command>
|
||||
|
||||
# end the search and return to the original HEAD
|
||||
git bisect reset
|
||||
|
||||
# follow lines moved from other files
|
||||
git blame -C <file>
|
||||
```
|
||||
|
||||
Pulling new changes into current branch from mainline
|
||||
```shell
|
||||
git checkout [branch-name]
|
||||
git fetch origin [master-branch-name]
|
||||
git rebase origin/[master-branch-name]
|
||||
## 🧩 Submodules & Worktrees
|
||||
|
||||
```bash
|
||||
# add a submodule
|
||||
git submodule add <url> <path>
|
||||
|
||||
# check out every submodule
|
||||
git submodule update --init --recursive
|
||||
|
||||
# update submodules to their latest commit
|
||||
git submodule update --remote
|
||||
|
||||
# commit each submodule sits on
|
||||
git submodule status
|
||||
|
||||
# check out a second branch side by side
|
||||
git worktree add ../<dir> <branch>
|
||||
|
||||
# list linked working trees
|
||||
git worktree list
|
||||
|
||||
# remove one again
|
||||
git worktree remove ../<dir>
|
||||
```
|
||||
|
||||
## 🧠 Miscellaneous
|
||||
|
||||
```bash
|
||||
# rename the local default branch
|
||||
git branch -m master main
|
||||
|
||||
# graph of merge commits only
|
||||
git log --graph --merges
|
||||
|
||||
# export a snapshot without .git
|
||||
git archive -o release.zip HEAD
|
||||
|
||||
# contributions per author
|
||||
git shortlog -sn --no-merges
|
||||
|
||||
# full SHA of the current commit
|
||||
git rev-parse HEAD
|
||||
|
||||
# name of the current branch
|
||||
git rev-parse --abbrev-ref HEAD
|
||||
|
||||
# repository size on disk
|
||||
git count-objects -vH
|
||||
|
||||
# enable background repacking (Git 2.30+)
|
||||
git maintenance start
|
||||
```
|
||||
|
||||
Update a feature branch with the latest mainline:
|
||||
|
||||
```bash
|
||||
git switch <branch>
|
||||
git fetch origin <main-branch>
|
||||
git rebase origin/<main-branch> # or: git merge origin/<main-branch>
|
||||
```
|
||||
|
||||
## 📚 Resources
|
||||
|
||||
- [Official documentation](https://git-scm.com/docs)
|
||||
- [Pro Git (free book)](https://git-scm.com/book)
|
||||
- [Interactive branching tutorial](https://learngitbranching.js.org)
|
||||
- [Dangit, Git!?!](https://dangitgit.com) — recovering from the classic mistakes
|
||||
@@ -1,163 +1,186 @@
|
||||
# Conventional Commits Cheatsheet
|
||||
**Conventional Commits** is a convention for commit messages that both humans and tools can read: the type tells you what kind of change it is, and release tooling turns that into version numbers and changelogs automatically.
|
||||
|
||||
## Structure
|
||||
|
||||
A conventional commit message follows this structure:
|
||||
|
||||
```
|
||||
<type>[optional scope]: <description>
|
||||
```text
|
||||
<type>[optional scope][!]: <description>
|
||||
|
||||
[optional body]
|
||||
|
||||
[optional footer(s)]
|
||||
```
|
||||
|
||||
## Elements
|
||||
|
||||
### Type (Required)
|
||||
The type describes the kind of change being made. Common types include:
|
||||
|
||||
- **feat**: A new feature for the user
|
||||
- **fix**: A bug fix for the user
|
||||
- **docs**: Documentation changes
|
||||
- **style**: Code style changes (formatting, missing semicolons, etc.)
|
||||
- **refactor**: Code changes that neither fix a bug nor add a feature
|
||||
- **test**: Adding or updating tests
|
||||
- **chore**: Maintenance tasks, dependency updates, build changes
|
||||
- **perf**: Performance improvements
|
||||
- **ci**: Changes to CI/CD configuration
|
||||
- **build**: Changes to build system or external dependencies
|
||||
- **revert**: Reverting a previous commit
|
||||
|
||||
### Scope (Optional)
|
||||
The scope provides additional context about what part of the codebase is affected:
|
||||
|
||||
```text
|
||||
feat(auth): add OAuth2 login
|
||||
```
|
||||
|
||||
## 🏷 Types
|
||||
|
||||
| Type | Use it for | Release effect |
|
||||
|------------|---------------------------------------------------------|----------------|
|
||||
| `feat` | A new feature for the user | Minor |
|
||||
| `fix` | A bug fix for the user | Patch |
|
||||
| `docs` | Documentation only | None |
|
||||
| `style` | Formatting, whitespace, semicolons — no logic change | None |
|
||||
| `refactor` | Code change that neither fixes a bug nor adds a feature | None |
|
||||
| `perf` | A change that improves performance | Patch |
|
||||
| `test` | Adding or correcting tests | None |
|
||||
| `build` | Build system or dependencies | None |
|
||||
| `ci` | CI configuration and scripts | None |
|
||||
| `chore` | Maintenance that does not touch src or tests | None |
|
||||
| `revert` | Reverting an earlier commit | Depends |
|
||||
|
||||
> 💡 Any type with a `!` or a `BREAKING CHANGE:` footer triggers a **major** release, `feat` included.
|
||||
|
||||
## 🎯 Scope
|
||||
|
||||
The scope is an optional noun in parentheses naming the part of the codebase affected. Keep the list short and consistent — package names, modules, or layers.
|
||||
|
||||
```text
|
||||
feat(auth): add OAuth2 integration
|
||||
fix(api): resolve timeout issues
|
||||
docs(readme): update installation instructions
|
||||
fix(api): resolve timeout on slow upstreams
|
||||
docs(readme): update the installation steps
|
||||
refactor(parser)!: drop support for the legacy format
|
||||
```
|
||||
|
||||
### Description (Required)
|
||||
A brief description of the change:
|
||||
## ✍️ Description
|
||||
|
||||
- Use imperative mood ("add" not "added" or "adds")
|
||||
- Keep it concise (50 characters or less recommended)
|
||||
- Don't capitalize the first letter
|
||||
- Don't end with a period
|
||||
| Rule | ✅ Good | ❌ Avoid |
|
||||
|--------------------------------|-----------------------------|----------------------------------|
|
||||
| Imperative mood | `add user export` | `added user export` |
|
||||
| Lower case, no trailing period | `fix flaky login test` | `Fix flaky login test.` |
|
||||
| Say what changed, not where | `fix off-by-one in paging` | `fix bug in file` |
|
||||
| Around 50 characters | `feat(api): add rate limit` | a full sentence with sub-clauses |
|
||||
|
||||
### Body (Optional)
|
||||
Provides more detailed explanation of the change:
|
||||
## 📝 Body & Footers
|
||||
|
||||
- Separate from description with a blank line
|
||||
- Explain the motivation and contrast with previous behavior
|
||||
- Use imperative mood
|
||||
The body explains **why**, not how — the diff already shows how. Separate it from the description with a blank line and wrap at ~72 characters.
|
||||
|
||||
### Footer (Optional)
|
||||
Contains metadata about the commit:
|
||||
Footers come last, one per line:
|
||||
|
||||
- **Breaking changes**: Start with `BREAKING CHANGE:`
|
||||
- **Issue references**: `Closes #123`, `Fixes #456`
|
||||
- **Co-authors**: `Co-authored-by: Name <email>`
|
||||
| Footer | Meaning |
|
||||
|--------------------------------|-----------------------------------------------|
|
||||
| `BREAKING CHANGE: <what>` | Incompatible change, forces a major release |
|
||||
| `Closes #123` / `Fixes #456` | Closes the issue when merged (GitHub, GitLab) |
|
||||
| `Refs #789` | Related, but does not close it |
|
||||
| `Co-authored-by: Name <email>` | Credits a second author |
|
||||
| `Reviewed-by: Name <email>` | Records the reviewer |
|
||||
|
||||
## Examples
|
||||
## 💥 Breaking Changes
|
||||
|
||||
### Simple commit
|
||||
Two ways to mark one — the `!` is visible in `git log --oneline`, the footer explains the migration. Use both.
|
||||
|
||||
```text
|
||||
feat(api)!: require an email address when shipping
|
||||
|
||||
BREAKING CHANGE: POST /orders now rejects requests without a customer
|
||||
email. Add the field before upgrading; see docs/migrations/2026-08.md.
|
||||
```
|
||||
|
||||
## 📄 Examples
|
||||
|
||||
```text
|
||||
feat: add user authentication
|
||||
```
|
||||
|
||||
### With scope
|
||||
```
|
||||
fix(parser): handle edge case in JSON parsing
|
||||
```text
|
||||
fix(parser): handle a trailing comma in JSON input
|
||||
```
|
||||
|
||||
### With body
|
||||
```
|
||||
```text
|
||||
feat: add email notifications
|
||||
|
||||
Users can now receive email notifications for important events.
|
||||
This includes account changes, security alerts, and system updates.
|
||||
Users can now receive email notifications for account changes,
|
||||
security alerts and system updates. Delivery is queued so a slow
|
||||
SMTP server never blocks the request.
|
||||
```
|
||||
|
||||
### With footer
|
||||
```
|
||||
```text
|
||||
fix: prevent racing of requests
|
||||
|
||||
Introduce a request id and a reference to latest request. Dismiss
|
||||
incoming responses other than from latest request.
|
||||
Introduce a request id and a reference to the latest request.
|
||||
Dismiss incoming responses other than from the latest request.
|
||||
|
||||
Closes #123
|
||||
```
|
||||
|
||||
### Breaking change
|
||||
```
|
||||
feat!: send an email to the customer when a product is shipped
|
||||
```text
|
||||
feat(cart): add the ability to remove items
|
||||
|
||||
BREAKING CHANGE: The shipping service now requires an email address
|
||||
```
|
||||
|
||||
### Full example
|
||||
```
|
||||
feat(shopping cart): add ability to remove items
|
||||
|
||||
Users can now remove items from their shopping cart by clicking
|
||||
the remove button next to each item. This improves the user
|
||||
experience by allowing corrections without starting over.
|
||||
Users can remove items from the cart instead of starting over,
|
||||
which was the most common complaint in support tickets.
|
||||
|
||||
Closes #456
|
||||
Co-authored-by: Jane Doe <jane@example.com>
|
||||
```
|
||||
|
||||
## Common Tools
|
||||
## 🔢 Versioning
|
||||
|
||||
### Commitizen
|
||||
Interactive tool for creating conventional commits:
|
||||
Release tooling maps the history since the last tag onto a semantic version:
|
||||
|
||||
```shell
|
||||
npm install -g commitizen
|
||||
npm install -g cz-conventional-changelog
|
||||
| Commits since the last release | New version from `1.4.2` |
|
||||
|--------------------------------|--------------------------|
|
||||
| Only `docs`, `chore`, `style`… | no release |
|
||||
| At least one `fix` or `perf` | `1.4.3` |
|
||||
| At least one `feat` | `1.5.0` |
|
||||
| Any `!` or `BREAKING CHANGE:` | `2.0.0` |
|
||||
|
||||
## 🛠 Tooling
|
||||
|
||||
### Commitizen — a prompt instead of a blank editor
|
||||
|
||||
```bash
|
||||
npm install -g commitizen cz-conventional-changelog
|
||||
echo '{ "path": "cz-conventional-changelog" }' > ~/.czrc
|
||||
```
|
||||
|
||||
Usage:
|
||||
```shell
|
||||
# then commit with
|
||||
git cz
|
||||
```
|
||||
|
||||
### Commitlint
|
||||
Lints commit messages to ensure they follow conventional format:
|
||||
### commitlint — reject messages that do not follow the convention
|
||||
|
||||
```shell
|
||||
npm install --save-dev @commitlint/config-conventional @commitlint/cli
|
||||
```bash
|
||||
npm install --save-dev @commitlint/cli @commitlint/config-conventional
|
||||
```
|
||||
|
||||
Configuration in `.commitlintrc.json`:
|
||||
```json
|
||||
{
|
||||
"extends": ["@commitlint/config-conventional"]
|
||||
}
|
||||
```
|
||||
|
||||
### Husky
|
||||
Git hooks to enforce commit message format:
|
||||
### husky — run commitlint from a git hook
|
||||
|
||||
```shell
|
||||
```bash
|
||||
npm install --save-dev husky
|
||||
npx husky add .husky/commit-msg 'npx --no -- commitlint --edit ${1}'
|
||||
npx husky init
|
||||
|
||||
# husky v9+: write the hook file yourself
|
||||
echo 'npx --no -- commitlint --edit "$1"' > .husky/commit-msg
|
||||
```
|
||||
|
||||
### Semantic Release
|
||||
Automatically generates releases based on conventional commits:
|
||||
### Releases & changelogs
|
||||
|
||||
```shell
|
||||
```bash
|
||||
# decide the version, tag, and publish from the commit history
|
||||
npm install --save-dev semantic-release
|
||||
|
||||
# or just generate the changelog
|
||||
npx conventional-changelog-cli -p angular -i CHANGELOG.md -s
|
||||
|
||||
# a fast, language-agnostic alternative
|
||||
git cliff --tag v1.5.0 --output CHANGELOG.md
|
||||
```
|
||||
|
||||
### Conventional Changelog
|
||||
Generates changelogs from conventional commits:
|
||||
## ✅ Habits That Keep It Useful
|
||||
|
||||
```shell
|
||||
npm install -g conventional-changelog-cli
|
||||
conventional-changelog -p angular -i CHANGELOG.md -s
|
||||
```
|
||||
- One logical change per commit — if the description needs an "and", split it.
|
||||
- Write the message for the person bisecting in six months, not for the linter.
|
||||
- Keep the scope vocabulary small and documented in `CONTRIBUTING.md`.
|
||||
- Never invent types the tooling does not know; `config` or `wip` silently drop out of the changelog.
|
||||
- Squash-merging? The **PR title** becomes the commit message, so lint that too.
|
||||
|
||||
## 📚 Resources
|
||||
|
||||
- [Conventional Commits specification](https://www.conventionalcommits.org/)
|
||||
- [commitlint](https://commitlint.js.org/)
|
||||
- [semantic-release](https://semantic-release.gitbook.io/)
|
||||
- [Semantic Versioning](https://semver.org/)
|
||||
+181
-65
@@ -1,89 +1,205 @@
|
||||
# Processing JSON using jq
|
||||
**jq** is a command-line JSON processor: it slices, filters, maps and transforms structured data the way `sed` and `awk` do for text. A jq program is a *filter* — it takes an input, and produces an output.
|
||||
|
||||
jq is useful to slice, filter, map and transform structured json data.
|
||||
```bash
|
||||
# pretty-print a file
|
||||
jq . data.json
|
||||
|
||||
## Installing jq
|
||||
# read from a pipe
|
||||
curl -s https://api.example.com/users | jq '.[] | .name'
|
||||
```
|
||||
|
||||
### On Mac OS
|
||||
## 📦 Installation
|
||||
|
||||
`brew install jq`
|
||||
```bash
|
||||
brew install jq # macOS
|
||||
apt install jq # Debian, Ubuntu
|
||||
dnf install jq # Fedora, RHEL
|
||||
choco install jq # Windows
|
||||
```
|
||||
|
||||
### On AWS Linux
|
||||
## 🚩 Command-Line Flags
|
||||
|
||||
Not available as yum install on our current AMI. It should be on the latest AMI though: https://aws.amazon.com/amazon-linux-ami/2015.09-release-notes/
|
||||
| Flag | Description |
|
||||
|--------------------------|------------------------------------------------------------------|
|
||||
| `-r`, `--raw-output` | Print strings without quotes — what you want for shell variables |
|
||||
| `-c`, `--compact-output` | One compact line per result instead of pretty-printed JSON |
|
||||
| `-n`, `--null-input` | Do not read input; build JSON from scratch |
|
||||
| `-s`, `--slurp` | Read the whole input into one array |
|
||||
| `-e`, `--exit-status` | Exit non-zero when the last output is `false` or `null` |
|
||||
| `-j`, `--join-output` | Raw output with no newline between results |
|
||||
| `--tab` / `--indent n` | Indent with tabs, or with `n` spaces |
|
||||
| `-S`, `--sort-keys` | Sort object keys in the output |
|
||||
| `--arg name value` | Pass a string into the program as `$name` |
|
||||
| `--argjson name json` | Pass parsed JSON in as `$name` |
|
||||
| `--slurpfile name f` | Read a whole file into `$name` as an array |
|
||||
| `--raw-input`, `-R` | Treat each input line as a string instead of JSON |
|
||||
|
||||
Installing from the source proved to be tricky.
|
||||
## 🧭 Core Syntax
|
||||
|
||||
## Useful arguments
|
||||
| Filter | Description |
|
||||
|--------------|---------------------------------------------------------------|
|
||||
| `.` | The identity filter — the input, unchanged |
|
||||
| `.foo` | The value of field `foo` |
|
||||
| `.foo.bar` | Nested field access |
|
||||
| `.foo?` | Same, but stays quiet when the input is not an object |
|
||||
| `."odd key"` | A field whose name needs quoting |
|
||||
| `.[]` | Every element of an array (or every value of an object) |
|
||||
| `.[0]` | One element by index (`-1` is the last) |
|
||||
| `.[2:4]` | A slice — see the table below |
|
||||
| `\|` | Pipe: feed the result of the left filter into the right one |
|
||||
| `,` | Run both filters on the same input, emit both results |
|
||||
| `()` | Grouping |
|
||||
| `//` | Alternative: use the right side if the left is `false`/`null` |
|
||||
| `?` | Suppress errors from the preceding filter |
|
||||
|
||||
When running jq, the following arguments may become handy:
|
||||
## 📇 Objects
|
||||
|
||||
| Argument | Description |
|
||||
| ------ | :--------- |
|
||||
| `--version`| Output the jq version and exit with zero. |
|
||||
| `--sort-keys` | Output the fields of each object with the keys in sorted order.|
|
||||
| Task | Filter |
|
||||
|-----------------------------|---------------------------------------------------|
|
||||
| List the keys | `jq 'keys'` (`keys_unsorted` to keep order) |
|
||||
| Does a key exist? | `jq 'has("foo")'` |
|
||||
| Pick a few fields | `jq '{name, id}'` |
|
||||
| Rename while picking | `jq '{title: .name, ref: .id}'` |
|
||||
| Delete a key | `jq 'del(.foo)'` |
|
||||
| Add 1 to every value | `jq 'map_values(.+1)'` |
|
||||
| Merge two objects | `jq '.a * .b'` (deep) or `jq '.a + .b'` (shallow) |
|
||||
| Object → array of pairs | `jq 'to_entries'` |
|
||||
| Array of pairs → object | `jq 'from_entries'` |
|
||||
| Transform every entry | `jq 'with_entries(.value += 1)'` |
|
||||
| Build an object from values | `jq -n '{time: now, host: $ENV.HOSTNAME}'` |
|
||||
|
||||
## Basic concepts
|
||||
## 📚 Arrays
|
||||
|
||||
The syntax for jq is pretty coherent:
|
||||
### Slicing and filtering
|
||||
|
||||
| Syntax | Description |
|
||||
| ------ | :--------- |
|
||||
| , | Filters separated by a comma will produce multiple independent outputs|
|
||||
| ? | Will ignores error if the type is unexpected |
|
||||
| [] | Array construction |
|
||||
| {} | Object construction |
|
||||
| + | Concatenate or Add |
|
||||
| - | Difference of sets or Substract |
|
||||
| length | Size of selected element |
|
||||
| | | Pipes are used to chain commands in a similar fashion than bash|
|
||||
| Task | Filter |
|
||||
|------------------------------|--------------------------------------------------------------------------------------------------------------------|
|
||||
| Every element | `jq '.[]'` |
|
||||
| First / last | `jq '.[0]'` / `jq '.[-1]'` |
|
||||
| Range, first three, last two | `jq '.[2:4]'`, `jq '.[:3]'`, `jq '.[-2:]'` |
|
||||
| Numbers above a threshold | `jq 'map(select(. >= 2))'` |
|
||||
| Objects matching a field | `jq '.[] \| select(.id == "second")'` |
|
||||
| Several conditions | `jq '.[] \| select(.age > 18 and .city == "Oslo")'` |
|
||||
| By type | `jq '.[] \| numbers'` — also `strings`, `booleans`, `nulls`, `arrays`, `objects`, `iterables`, `scalars`, `values` |
|
||||
| Test a string field | `jq '.[] \| select(.name \| test("^a"; "i"))'` |
|
||||
|
||||
### Mapping and transforming
|
||||
|
||||
## Dealing with json objects
|
||||
| Task | Filter |
|
||||
|----------------------------|---------------------------------------------|
|
||||
| Apply to every element | `jq 'map(.+1)'` |
|
||||
| Pull one field out of each | `jq 'map(.name)'` or `jq '.[].name'` |
|
||||
| Drop elements by index | `jq 'del(.[1, 2])'` |
|
||||
| Concatenate nested arrays | `jq 'add'` |
|
||||
| Flatten | `jq 'flatten'` (`flatten(1)` for one level) |
|
||||
| Sort | `jq 'sort'` / `jq 'sort_by(.foo)'` |
|
||||
| Group | `jq 'group_by(.foo)'` |
|
||||
| Deduplicate | `jq 'unique'` / `jq 'unique_by(.foo)'` |
|
||||
| Reverse | `jq 'reverse'` |
|
||||
| Smallest / largest | `jq 'min'`, `jq 'max_by(.price)'` |
|
||||
| Count | `jq 'length'` |
|
||||
| Sum | `jq 'map(.amount) \| add'` |
|
||||
| Any / all match | `jq 'any(.active)'`, `jq 'all(.age > 18)'` |
|
||||
| A range of numbers | `jq '[range(2;4)]'` |
|
||||
| The type of each item | `jq 'map(type)'` |
|
||||
|
||||
| Description | Command |
|
||||
| ------ | :--------- |
|
||||
| Display all keys | `jq 'keys'` |
|
||||
| Adds + 1 to all items | `jq 'map_values(.+1)'` |
|
||||
| Delete a key| `jq 'del(.foo)'` |
|
||||
| Convert an object to array | `to_entries | map([.key, .value])` |
|
||||
## 🔤 Strings & Formatting
|
||||
|
||||
## Dealing with fields
|
||||
| Task | Filter |
|
||||
|---------------------------|------------------------------------------------|
|
||||
| Interpolate | `jq -r '"\(.name) is \(.age)"'` |
|
||||
| Change case | `jq 'ascii_downcase'` / `ascii_upcase` |
|
||||
| Split and join | `jq 'split(",")'` / `jq 'join(", ")'` |
|
||||
| Trim a prefix or suffix | `jq 'ltrimstr("v")'` / `rtrimstr(".json")` |
|
||||
| Match a regex | `jq 'test("^http")'` |
|
||||
| Replace | `jq 'sub("^v"; "")'` / `gsub("\\s+"; "-")` |
|
||||
| Capture named groups | `jq 'capture("(?<host>[^:]+):(?<port>\\d+)")'` |
|
||||
| CSV / TSV output | `jq -r '.[] \| [.id, .name] \| @csv'` |
|
||||
| Escape for a URL or shell | `jq -r '@uri "?q=\(.term)"'`, `@sh` |
|
||||
| Base64 | `jq -r '@base64'` / `jq -r '@base64d'` |
|
||||
| Object → JSON string | `jq 'tojson'` (and `fromjson` back) |
|
||||
|
||||
| Description | Command |
|
||||
| ------ | :--------- |
|
||||
| Concatenate two fields| `fieldNew=.field1+' '+.field2` |
|
||||
## 🔀 Conditionals & Errors
|
||||
|
||||
```bash
|
||||
# if / then / else — 'end' is required
|
||||
jq 'if .age >= 18 then "adult" else "minor" end'
|
||||
|
||||
## Dealing with json arrays
|
||||
# several branches
|
||||
jq 'if .n > 100 then "big" elif .n > 10 then "medium" else "small" end'
|
||||
|
||||
### Slicing and Filtering
|
||||
# fall back when a value is missing or null
|
||||
jq '.nickname // .name'
|
||||
|
||||
| Description | Command |
|
||||
| ------ | :--------- |
|
||||
| All | `jq .[]` |
|
||||
| First | `jq '.[0]'` |
|
||||
| Range | `jq '.[2:4]'` |
|
||||
| First 3 | `jq '.[:3]'` |
|
||||
| Last 2 | `jq '.[-2:]'` |
|
||||
| Before Last | `jq '.[-2]'`|
|
||||
| Select array of int by value | `jq 'map(select(. >= 2))'` |
|
||||
| Select array of objects by value| `jq '.[] | select(.id == "second")'` |
|
||||
| Select by type | `jq '.[] | numbers'` with type been arrays, objects, iterables, booleans, numbers, normals, finites, strings, nulls, values, scalars |
|
||||
# keep going when a filter would fail
|
||||
jq '.items[]? // empty'
|
||||
|
||||
### Mapping and Transforming
|
||||
# turn a failure into a value of your own
|
||||
jq 'try (.a.b.c) catch "not found"'
|
||||
|
||||
| Description | Command |
|
||||
| ------ | :--------- |
|
||||
| Add + 1 to all items | `jq 'map(.+1)'` |
|
||||
| Delete 2 items| `jq 'del(.[1, 2])'` |
|
||||
| Concatenate arrays | `jq 'add'` |
|
||||
| Flatten an array | `jq 'flatten'` |
|
||||
| Create a range of numbers | `jq '[range(2;4)]'` |
|
||||
| Display the type of each item| `jq 'map(type)'` |
|
||||
| Sort an array of basic type| `jq 'sort'` |
|
||||
| Sort an array of objects | `jq 'sort_by(.foo)'` |
|
||||
| Group by a key - opposite to flatten | `jq 'group_by(.foo)'` |
|
||||
| Minimun value of an array| `jq 'min'` .See also min, max, min_by(path_exp), max_by(path_exp) |
|
||||
| Remove duplicates| `jq 'unique'` or `jq 'unique_by(.foo)'` or `jq 'unique_by(length)'` |
|
||||
| Reverse an array | `jq 'reverse'` |
|
||||
# stop with a message
|
||||
jq 'if .id == null then error("id is required") else . end'
|
||||
```
|
||||
|
||||
## 🧮 Variables, Functions & Reduction
|
||||
|
||||
```bash
|
||||
# bind a value to a variable
|
||||
jq '.items[] as $item | $item.name'
|
||||
|
||||
# pass values in from the shell
|
||||
jq --arg env prod '.deploys[] | select(.env == $env)'
|
||||
jq --argjson min 10 '.[] | select(.count > $min)'
|
||||
|
||||
# define a reusable function
|
||||
jq 'def is_active: .status == "active"; map(select(is_active))'
|
||||
|
||||
# accumulate a value
|
||||
jq 'reduce .[] as $x (0; . + $x.amount)'
|
||||
|
||||
# keep the intermediate results too
|
||||
jq '[foreach .[] as $x (0; . + $x; .)]'
|
||||
|
||||
# read the environment
|
||||
jq -n 'env.HOME'
|
||||
```
|
||||
|
||||
## 🛠 Everyday Recipes
|
||||
|
||||
```bash
|
||||
# pretty-print, sorted, and back to a file
|
||||
jq -S . data.json > sorted.json
|
||||
|
||||
# one field, unquoted, for use in a shell variable
|
||||
version=$(jq -r .version package.json)
|
||||
|
||||
# every key of every object, deduplicated
|
||||
jq -r '[.[] | keys[]] | unique[]' data.json
|
||||
|
||||
# turn an array of objects into a CSV with a header row
|
||||
jq -r '(.[0] | keys_unsorted), (.[] | [.[]]) | @csv' data.json
|
||||
|
||||
# merge two JSON files
|
||||
jq -s '.[0] * .[1]' a.json b.json
|
||||
|
||||
# read newline-delimited JSON (one object per line)
|
||||
jq -c '.event' events.ndjson
|
||||
|
||||
# turn plain text lines into a JSON array
|
||||
printf 'a\nb\n' | jq -R -s 'split("\n") | map(select(length > 0))'
|
||||
|
||||
# find the path to every occurrence of a key
|
||||
jq -c 'paths(scalars) as $p | {path: $p, value: getpath($p)}' data.json
|
||||
|
||||
# rewrite every string in a document
|
||||
jq 'walk(if type == "string" then ascii_downcase else . end)' data.json
|
||||
|
||||
# use the exit status in a script
|
||||
jq -e '.errors | length == 0' report.json && echo "clean"
|
||||
```
|
||||
|
||||
## 📚 Resources
|
||||
|
||||
- [Manual](https://jqlang.github.io/jq/manual/)
|
||||
- [Try jq in the browser (jqplay)](https://jqplay.org)
|
||||
- [Tutorial](https://jqlang.github.io/jq/tutorial/)
|
||||
@@ -1,52 +1,121 @@
|
||||
## Syntax
|
||||
**JSONPath** is a query language for JSON, the way XPath is for XML. An expression walks the document from the root and returns every value that matches.
|
||||
|
||||
Depending on the client used JSONPath expressions do start with `$.` indicating the root element.
|
||||
Some clients omit the leading `$.`.
|
||||
> ℹ️ Expressions normally start with `$`, the root element. Some clients let you drop the leading `$.`. Implementations differ in the details — [RFC 9535](https://www.rfc-editor.org/rfc/rfc9535.html) standardised the syntax in 2024, but older libraries predate it.
|
||||
|
||||
| Syntax | Description |
|
||||
| ------ | :--------- |
|
||||
| `$.store.book[0].title` | |
|
||||
| `store.book[0].title` | With implicit `$.` |
|
||||
| `$['store']['book'][0]['title']` | Alternative notation similar to scripting languages |
|
||||
## 🧭 Syntax
|
||||
|
||||
## Tree Traversal
|
||||
| Syntax | Description |
|
||||
|----------------------------------|----------------------------------------------------------|
|
||||
| `$` | The root element |
|
||||
| `@` | The element currently being filtered |
|
||||
| `$.store.book[0].title` | Dot notation |
|
||||
| `store.book[0].title` | With the `$.` implied |
|
||||
| `$['store']['book'][0]['title']` | Bracket notation — needed for keys with spaces or dashes |
|
||||
| `..` | Recursive descent, at any depth |
|
||||
| `*` | Wildcard: every element or member |
|
||||
| `[]` | Subscript: index, slice, union or filter |
|
||||
|
||||
| Syntax | Description |
|
||||
| ------ | :--------- |
|
||||
| `$.parentNode.childNode.field` | XPath: `/parentNode/childNode/@field` (content of "field" of all "childNode"s of "parentNode") |
|
||||
| `$..anyChildNode` | XPath: `//anyChildNode` (all children at any depth named "anyChildNode") |
|
||||
| `$.parentNode.*` | XPath: `/parentNode/*` (all children below) |
|
||||
## 🌳 Tree Traversal
|
||||
|
||||
## Array Access
|
||||
| Syntax | Description |
|
||||
|--------------------------------|--------------------------------------------------------------------------|
|
||||
| `$.parentNode.childNode.field` | XPath `/parentNode/childNode/@field` — the field of every matching child |
|
||||
| `$..anyChildNode` | XPath `//anyChildNode` — every node with that name, at any depth |
|
||||
| `$.parentNode.*` | XPath `/parentNode/*` — every child of the node |
|
||||
| `$..*` | Every value in the document |
|
||||
| `$..book[*].author` | The author of every book, however deeply nested |
|
||||
|
||||
| Syntax | Description |
|
||||
| ------ | :--------- |
|
||||
| `$.myList[0]` | first element |
|
||||
| `$.myList[-1]` | last element |
|
||||
| `$.myList[2:3]` | range |
|
||||
| `$.myList[0,4,5]` | selection |
|
||||
## 🔢 Array Access
|
||||
|
||||
## Filtering
|
||||
| Syntax | Description |
|
||||
|-------------------|-----------------------------------------|
|
||||
| `$.myList[0]` | First element |
|
||||
| `$.myList[-1]` | Last element |
|
||||
| `$.myList[*]` | Every element |
|
||||
| `$.myList[2:4]` | Slice — index 2 up to, not including, 4 |
|
||||
| `$.myList[:3]` | The first three |
|
||||
| `$.myList[-2:]` | The last two |
|
||||
| `$.myList[::2]` | Every second element |
|
||||
| `$.myList[0,4,5]` | A union of specific indexes |
|
||||
|
||||
| Syntax | Description |
|
||||
| ------ | :--------- |
|
||||
| `$.customer[?(@.car)]` | Only "customer"s that have attribute "car" |
|
||||
| `$.customer[?(@.car == 'Ford Fiesta')]` | Only "customer"s with "Ford Fiesta"s |
|
||||
| `$.customer[?(@.age > 18)]` | Only adults |
|
||||
## 🔎 Filtering
|
||||
|
||||
## Complex Conditions
|
||||
Filters are written `[?(...)]` — `[?...]` in RFC 9535 — and keep the elements for which the expression is true.
|
||||
|
||||
| Syntax | Description |
|
||||
| ------ | :--------- |
|
||||
| `$.customer[?(@.age > 18 \|\| @.car == 'Ford Fiesta')]` | logical or |
|
||||
| `$.customer[?(@.age < 18 && @.hobby == 'Biking' )]` | logical and |
|
||||
| Syntax | Description |
|
||||
|-----------------------------------------|----------------------------------------------------|
|
||||
| `$.customer[?(@.car)]` | Customers that have a `car` field at all |
|
||||
| `$.customer[?(@.car == 'Ford Fiesta')]` | Customers with a Ford Fiesta |
|
||||
| `$.customer[?(@.age > 18)]` | Adults only |
|
||||
| `$.customer[?(@.age != 18)]` | Everyone but the 18-year-olds |
|
||||
| `$.book[?(@.price <= 10)]` | Also `<`, `>=`, `==` |
|
||||
| `$.book[?(@.title =~ /^The/)]` | Regular expression match (implementation-specific) |
|
||||
|
||||
## Output Mapping
|
||||
### Combining conditions
|
||||
|
||||
| Syntax | Description |
|
||||
| ------ | :--------- |
|
||||
| `$.[].{Name:name, Age:age, Hobbies:details.hobbies}` | Mapping fields/nested fields to new set
|
||||
| Syntax | Description |
|
||||
|---------------------------------------------------------|-------------|
|
||||
| `$.customer[?(@.age > 18 \|\| @.car == 'Ford Fiesta')]` | Logical or |
|
||||
| `$.customer[?(@.age < 18 && @.hobby == 'Biking')]` | Logical and |
|
||||
| `$.customer[?(!@.car)]` | Logical not |
|
||||
|
||||
## Credits
|
||||
## 🧮 Functions
|
||||
|
||||
Original author: https://gist.github.com/mackoj/5786f8b95da0a82e8e003f444c4295bf
|
||||
Available in RFC 9535 and in most modern implementations:
|
||||
|
||||
| Function | Example | Returns |
|
||||
|------------|---------------------------------------|----------------------------------------|
|
||||
| `length()` | `$.book[?(length(@.tags) > 2)]` | Length of a string, array or object |
|
||||
| `count()` | `$.book[?(count(@.authors.*) > 1)]` | Number of nodes a query matched |
|
||||
| `match()` | `$.book[?(match(@.isbn, "\\d{13}"))]` | True when the **whole** string matches |
|
||||
| `search()` | `$.book[?(search(@.title, "Moon"))]` | True when the pattern occurs anywhere |
|
||||
| `value()` | `$.book[?(value(@.price) > 10)]` | The value of a single-node query |
|
||||
|
||||
Older Jayway-style implementations instead offer `.length()`, `.min()`, `.max()`, `.avg()` and `.sum()` at the end of a path — `$.book[*].price.sum()`.
|
||||
|
||||
## 📄 Worked Example
|
||||
|
||||
```json
|
||||
{
|
||||
"store": {
|
||||
"book": [
|
||||
{ "title": "Sayings of the Century", "author": "Nigel Rees", "price": 8.95 },
|
||||
{ "title": "Moby Dick", "author": "Herman Melville", "price": 8.99 },
|
||||
{ "title": "The Lord of the Rings", "author": "J. R. R. Tolkien", "price": 22.99 }
|
||||
],
|
||||
"bicycle": { "color": "red", "price": 19.95 }
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
| Expression | Result |
|
||||
|-------------------------------------|---------------------------------------|
|
||||
| `$.store.book[*].author` | All three authors |
|
||||
| `$..price` | `8.95`, `8.99`, `22.99`, `19.95` |
|
||||
| `$.store.book[?(@.price < 10)]` | The two cheap books |
|
||||
| `$.store.book[-1].title` | `"The Lord of the Rings"` |
|
||||
| `$..book[?(@.author =~ /Tolkien/)]` | The Tolkien entry |
|
||||
| `$.store.*` | The book array and the bicycle object |
|
||||
|
||||
## 🗺 Output Mapping
|
||||
|
||||
| Syntax | Description |
|
||||
|------------------------------------------------------|-------------------------------------------------------------------------------------------|
|
||||
| `$.[].{Name:name, Age:age, Hobbies:details.hobbies}` | Map fields and nested fields into a new object (JMESPath-style, not supported everywhere) |
|
||||
|
||||
## 🧰 Where You'll Meet It
|
||||
|
||||
- `kubectl get pods -o jsonpath='{.items[*].metadata.name}'`
|
||||
- Jayway JSONPath in Java, REST Assured assertions
|
||||
- Karate, Postman and Newman test scripts
|
||||
- Nginx, Grafana and Prometheus JSON scrapers
|
||||
|
||||
> 💡 For heavier transformation on the command line, `jq` is usually the better tool — JSONPath selects, `jq` selects *and* reshapes.
|
||||
|
||||
## 📚 Resources
|
||||
|
||||
- [RFC 9535 — JSONPath](https://www.rfc-editor.org/rfc/rfc9535.html)
|
||||
- [Try expressions online (jsonpath.com)](https://jsonpath.com)
|
||||
- [Comparison of implementations](https://cburgmer.github.io/json-path-comparison/)
|
||||
|
||||
Original author: https://gist.github.com/mackoj/5786f8b95da0a82e8e003f444c4295bf
|
||||
@@ -1,183 +1,304 @@
|
||||
# Kubernetes
|
||||
**kubectl** is the CLI for the Kubernetes API. Almost every command follows the same shape: `kubectl <verb> <resource> <name> [flags]` — `get pods`, `describe node web-1`, `delete deployment api`.
|
||||
|
||||
* PDF: https://sematext.com/kubernetes-cheat-sheet/
|
||||
* WEBSITE: https://kubernetes.io/
|
||||
* DOCUMENTATION: https://kubernetes.io/docs/home
|
||||
> 💡 Set up completion and an alias once and everything below gets shorter: `source <(kubectl completion bash)`, `alias k=kubectl`, `complete -o default -F __start_kubectl k`.
|
||||
|
||||
## Client Configuration
|
||||
## ⚙️ Configuration & Context
|
||||
|
||||
* Setup autocomplete in bash; bash-completion package should be installed first
|
||||
```
|
||||
```bash
|
||||
# show the merged kubeconfig
|
||||
kubectl config view
|
||||
|
||||
# read a single value out of it
|
||||
kubectl config view -o jsonpath='{.clusters[*].name}'
|
||||
|
||||
# which cluster am I talking to?
|
||||
kubectl config current-context
|
||||
|
||||
# list every context
|
||||
kubectl config get-contexts
|
||||
|
||||
# switch clusters
|
||||
kubectl config use-context <context>
|
||||
|
||||
# change the default namespace for this context
|
||||
kubectl config set-context --current --namespace=<namespace>
|
||||
|
||||
# use a different kubeconfig for one command
|
||||
KUBECONFIG=~/.kube/staging.yaml kubectl get nodes
|
||||
|
||||
# shell completion (bash; use 'zsh' or 'fish' as needed)
|
||||
source <(kubectl completion bash)
|
||||
```
|
||||
|
||||
* View Kubernetes config
|
||||
```
|
||||
kubectl config view
|
||||
```
|
||||
## 🔍 Viewing & Finding Resources
|
||||
|
||||
* View specific config items by json path
|
||||
```
|
||||
kubectl config view -o jsonpath='{.users[?(@.name == "k8s")].user.password}'
|
||||
```
|
||||
```bash
|
||||
# the everyday listing
|
||||
kubectl get pods
|
||||
kubectl get pods -o wide
|
||||
kubectl get pods --all-namespaces
|
||||
|
||||
* Set credentials for `foo.kuberntes.com`
|
||||
```
|
||||
kubectl config set-credentials kubeuser/foo.kubernetes.com --username=kubeuser --password=kubepassword
|
||||
```
|
||||
# every kind of resource in a namespace
|
||||
kubectl get all -n <namespace>
|
||||
|
||||
* Set active namespace
|
||||
```
|
||||
kubectl config set-context --current --namespace=namespace_name
|
||||
```
|
||||
# full definition, as stored by the API server
|
||||
kubectl get pod <pod> -o yaml
|
||||
|
||||
## Viewing, Finding resources
|
||||
# just the fields you care about
|
||||
kubectl get pods -o custom-columns='NAME:.metadata.name,NODE:.spec.nodeName'
|
||||
|
||||
* List all services in the namespace
|
||||
```
|
||||
kubectl get services
|
||||
```
|
||||
# pull one value out with JSONPath
|
||||
kubectl get pods -o jsonpath='{.items[*].metadata.name}'
|
||||
|
||||
* List all pods in all namespaces in wide format
|
||||
```
|
||||
kubectl get pods -o wide --all-namespaces
|
||||
```
|
||||
# filter by label, or by field
|
||||
kubectl get pods -l app=nginx,tier=frontend
|
||||
kubectl get pods --field-selector status.phase=Running
|
||||
|
||||
* List all pods in json (or yaml) format
|
||||
```
|
||||
kubectl get pods -o json
|
||||
```
|
||||
|
||||
* Describe resource details (node, pod, svc)
|
||||
```
|
||||
kubectl describe nodes my-node
|
||||
```
|
||||
|
||||
* List services sorted by name
|
||||
```
|
||||
# sort the output
|
||||
kubectl get services --sort-by=.metadata.name
|
||||
```
|
||||
|
||||
* List pods sorted by restart count
|
||||
```
|
||||
kubectl get pods --sort-by='.status.containerStatuses[0].restartCount'
|
||||
|
||||
# watch changes as they happen
|
||||
kubectl get pods -w
|
||||
|
||||
# events, newest last — the first place to look when something is stuck
|
||||
kubectl get events --sort-by=.lastTimestamp
|
||||
|
||||
# the long-form story of one object
|
||||
kubectl describe pod <pod>
|
||||
|
||||
# what resources exist, and what are their short names?
|
||||
kubectl api-resources
|
||||
|
||||
# what fields does this kind have?
|
||||
kubectl explain deployment.spec.template.spec.containers
|
||||
```
|
||||
|
||||
* Rolling update pods for frontend-v1
|
||||
```
|
||||
kubectl rolling-update frontend-v1 -f frontend-v2.json
|
||||
## 🚀 Creating, Applying & Deleting
|
||||
|
||||
```bash
|
||||
# apply a manifest, creating or updating as needed
|
||||
kubectl apply -f ./manifest.yaml
|
||||
|
||||
# apply everything in a directory, recursively
|
||||
kubectl apply -R -f ./k8s/
|
||||
|
||||
# apply a kustomize overlay
|
||||
kubectl apply -k ./overlays/production
|
||||
|
||||
# generate a manifest instead of creating anything
|
||||
kubectl create deployment web --image=nginx --dry-run=client -o yaml > web.yaml
|
||||
|
||||
# quick one-off objects
|
||||
kubectl create deployment web --image=nginx:1.27
|
||||
kubectl run debug --image=busybox -it --rm -- sh
|
||||
kubectl expose deployment web --port=80 --target-port=8080 --type=ClusterIP
|
||||
|
||||
# see what an apply would change before doing it
|
||||
kubectl diff -f ./manifest.yaml
|
||||
|
||||
# edit an object in your $EDITOR
|
||||
kubectl edit deployment web
|
||||
|
||||
# change one field without an editor
|
||||
kubectl patch deployment web -p '{"spec":{"replicas":4}}'
|
||||
|
||||
# delete
|
||||
kubectl delete -f ./manifest.yaml
|
||||
kubectl delete pod <pod> --grace-period=0 --force
|
||||
```
|
||||
|
||||
* Scale a replicaset named 'foo' to 3
|
||||
```
|
||||
kubectl scale --replicas=3 rs/foo
|
||||
## 📈 Scaling & Rollouts
|
||||
|
||||
```bash
|
||||
# scale a deployment, or scale from the manifest
|
||||
kubectl scale deployment web --replicas=3
|
||||
kubectl scale --replicas=3 -f web.yaml
|
||||
|
||||
# only scale if it currently has 3 replicas
|
||||
kubectl scale --current-replicas=3 --replicas=5 deployment/web
|
||||
|
||||
# horizontal autoscaling
|
||||
kubectl autoscale deployment web --min=2 --max=10 --cpu-percent=70
|
||||
|
||||
# roll out a new image
|
||||
kubectl set image deployment/web nginx=nginx:1.27
|
||||
|
||||
# watch the rollout
|
||||
kubectl rollout status deployment/web
|
||||
|
||||
# rollout history, and going back
|
||||
kubectl rollout history deployment/web
|
||||
kubectl rollout undo deployment/web
|
||||
kubectl rollout undo deployment/web --to-revision=2
|
||||
|
||||
# restart every pod of a deployment (picks up new config or secrets)
|
||||
kubectl rollout restart deployment/web
|
||||
|
||||
# pause and resume a rollout
|
||||
kubectl rollout pause deployment/web
|
||||
kubectl rollout resume deployment/web
|
||||
```
|
||||
|
||||
* Scale a resource specified in "foo.yaml" to 3
|
||||
```
|
||||
kubectl scale --replicas=3 -f foo.yaml
|
||||
## 📜 Logs & Debugging
|
||||
|
||||
```bash
|
||||
# logs of a pod
|
||||
kubectl logs <pod>
|
||||
|
||||
# follow them
|
||||
kubectl logs -f <pod>
|
||||
|
||||
# a specific container in a multi-container pod
|
||||
kubectl logs -f <pod> -c <container>
|
||||
|
||||
# what the previous, crashed container printed
|
||||
kubectl logs <pod> --previous
|
||||
|
||||
# recent output only
|
||||
kubectl logs <pod> --since=10m --tail=100
|
||||
|
||||
# logs from every pod behind a label
|
||||
kubectl logs -l app=web --all-containers --max-log-requests=10
|
||||
|
||||
# a shell inside a running container
|
||||
kubectl exec -it <pod> -- sh
|
||||
kubectl exec -it <pod> -c <container> -- bash
|
||||
|
||||
# a one-off command
|
||||
kubectl exec <pod> -- env
|
||||
|
||||
# copy files in and out
|
||||
kubectl cp <pod>:/var/log/app.log ./app.log
|
||||
kubectl cp ./config.yaml <pod>:/etc/app/config.yaml
|
||||
|
||||
# reach a service or pod from your laptop
|
||||
kubectl port-forward pod/<pod> 8080:80
|
||||
kubectl port-forward svc/<service> 8080:80
|
||||
|
||||
# attach a debug container to a running pod (distroless-friendly)
|
||||
kubectl debug -it <pod> --image=busybox --target=<container>
|
||||
|
||||
# a throwaway pod on a specific node
|
||||
kubectl debug node/<node> -it --image=busybox
|
||||
|
||||
# resource usage (needs metrics-server)
|
||||
kubectl top node
|
||||
kubectl top pod
|
||||
kubectl top pod <pod> --containers
|
||||
```
|
||||
|
||||
* Execute a command in every pod / replica
|
||||
```
|
||||
for i in 0 1; do kubectl exec foo-$i -- sh -c 'echo $(hostname) > /usr/share/nginx/html/index.html'; done
|
||||
## 🔐 Secrets & ConfigMaps
|
||||
|
||||
```bash
|
||||
# create a secret from literals or files
|
||||
kubectl create secret generic db-creds \
|
||||
--from-literal=username=jane \
|
||||
--from-literal=password=s3cr3t
|
||||
kubectl create secret generic tls-key --from-file=./tls.key
|
||||
|
||||
# a registry pull secret
|
||||
kubectl create secret docker-registry regcred \
|
||||
--docker-server=ghcr.io --docker-username=<user> --docker-password=<token>
|
||||
|
||||
# configmaps work the same way
|
||||
kubectl create configmap app-config --from-file=./config.yaml
|
||||
kubectl create configmap app-config --from-literal=LOG_LEVEL=debug
|
||||
|
||||
# read a secret back (values are base64)
|
||||
kubectl get secret db-creds -o jsonpath='{.data.password}' | base64 -d
|
||||
|
||||
# every key at once
|
||||
kubectl get secret db-creds -o go-template='{{range $k,$v := .data}}{{$k}}={{$v | base64decode}}{{"\n"}}{{end}}'
|
||||
```
|
||||
|
||||
## Manage Resources
|
||||
## 🏷 Labels, Annotations & Namespaces
|
||||
|
||||
* Get documentation for pod or service
|
||||
```
|
||||
kubectl explain pods,svc
|
||||
```bash
|
||||
# add or change a label (--overwrite to replace an existing one)
|
||||
kubectl label pod <pod> env=prod --overwrite
|
||||
|
||||
# remove a label
|
||||
kubectl label pod <pod> env-
|
||||
|
||||
# annotations work the same way
|
||||
kubectl annotate deployment web kubernetes.io/change-cause="bump to 1.27"
|
||||
|
||||
# show the labels in the listing
|
||||
kubectl get pods --show-labels
|
||||
|
||||
# namespaces
|
||||
kubectl get namespaces
|
||||
kubectl create namespace staging
|
||||
kubectl delete namespace staging
|
||||
```
|
||||
|
||||
* Create resource(s) like pods, services or daemonsets
|
||||
```
|
||||
kubectl create -f ./my-manifest.yaml
|
||||
## 🖥 Nodes & Cluster
|
||||
|
||||
```bash
|
||||
# nodes and their state
|
||||
kubectl get nodes -o wide
|
||||
kubectl describe node <node>
|
||||
|
||||
# stop scheduling new pods onto a node
|
||||
kubectl cordon <node>
|
||||
|
||||
# evict the pods and prepare for maintenance
|
||||
kubectl drain <node> --ignore-daemonsets --delete-emptydir-data
|
||||
|
||||
# put it back into service
|
||||
kubectl uncordon <node>
|
||||
|
||||
# keep pods off a node unless they tolerate the taint
|
||||
kubectl taint nodes <node> key=value:NoSchedule
|
||||
|
||||
# where is the control plane?
|
||||
kubectl cluster-info
|
||||
|
||||
# what API versions does the cluster serve?
|
||||
kubectl api-versions
|
||||
|
||||
# am I allowed to do this?
|
||||
kubectl auth can-i create deployments --namespace production
|
||||
```
|
||||
|
||||
* Apply a configuration to a resource
|
||||
```
|
||||
kubectl apply -f ./my-manifest.yaml
|
||||
```
|
||||
## 🧰 Handy One-Liners
|
||||
|
||||
* Start a single instance of Nginx
|
||||
```
|
||||
kubectl run nginx --image=nginx
|
||||
```
|
||||
```bash
|
||||
# every pod that is not Running
|
||||
kubectl get pods -A --field-selector=status.phase!=Running
|
||||
|
||||
* Create a secret with several keys
|
||||
```
|
||||
cat <<EOF | kubectl create -f -
|
||||
# the ten pods that restarted most
|
||||
kubectl get pods -A --sort-by='.status.containerStatuses[0].restartCount' | tail -10
|
||||
|
||||
# clean up evicted pods
|
||||
kubectl get pods -A --field-selector=status.phase=Failed -o name | xargs -r kubectl delete
|
||||
|
||||
# which node is each pod on
|
||||
kubectl get pods -o custom-columns='POD:.metadata.name,NODE:.spec.nodeName'
|
||||
|
||||
# images running in the cluster
|
||||
kubectl get pods -A -o jsonpath='{.items[*].spec.containers[*].image}' | tr ' ' '\n' | sort -u
|
||||
|
||||
# run a command in several pods
|
||||
for pod in $(kubectl get pods -l app=web -o name); do kubectl exec "$pod" -- hostname; done
|
||||
|
||||
# apply a manifest straight from a heredoc
|
||||
kubectl apply -f - <<EOF
|
||||
apiVersion: v1
|
||||
kind: Secret
|
||||
kind: ConfigMap
|
||||
metadata:
|
||||
name: mysecret
|
||||
type: Opaque
|
||||
name: demo
|
||||
data:
|
||||
password: $(echo "s33msi4" | base64)
|
||||
username: $(echo "jane"| base64)
|
||||
LOG_LEVEL: debug
|
||||
EOF
|
||||
```
|
||||
|
||||
* Delete a resource
|
||||
```
|
||||
kubectl delete -f ./my-manifest.yaml
|
||||
```
|
||||
## 📚 Resources
|
||||
|
||||
## Monitoring & Logging
|
||||
- [kubectl reference](https://kubernetes.io/docs/reference/kubectl/)
|
||||
- [kubectl quick reference](https://kubernetes.io/docs/reference/kubectl/quick-reference/)
|
||||
- [Documentation home](https://kubernetes.io/docs/home/)
|
||||
- [JSONPath support in kubectl](https://kubernetes.io/docs/reference/kubectl/jsonpath/)
|
||||
|
||||
* Deploy Heapster from Github repository
|
||||
```
|
||||
kubectl create -f deploy/kube-config/standalone/
|
||||
```
|
||||
|
||||
* Show metrics for nodes
|
||||
```
|
||||
kubectl top node
|
||||
```
|
||||
|
||||
* Show metrics for pods
|
||||
```
|
||||
kubectl top pod
|
||||
```
|
||||
|
||||
* Show metrics for a given pod and its containers
|
||||
```
|
||||
kubectl top pod pod_name --containers
|
||||
```
|
||||
|
||||
* Dump pod logs (stdout)
|
||||
```
|
||||
kubectl logs pod_name
|
||||
```
|
||||
|
||||
* Stream pod container logs (stdout, multi-container case)
|
||||
```
|
||||
kubectl logs -f pod_name -c my-container
|
||||
```
|
||||
|
||||
## Interacting with running pods
|
||||
|
||||
* Run command in pod
|
||||
```
|
||||
kubectl exec pod_name -- command_name
|
||||
```
|
||||
|
||||
* Run command in pod with multiple containers
|
||||
```
|
||||
kubectl exec pod_name -c container_name -- command_name
|
||||
```
|
||||
|
||||
* Get terminal of pod
|
||||
```
|
||||
kubectl exec -it pod_name /bin/sh
|
||||
```
|
||||
|
||||
* Get terminal of a container running in pod with multiple containers
|
||||
```
|
||||
kubectl exec -it pod_name -c container_name /bin/sh
|
||||
```
|
||||
|
||||
### Credits
|
||||
|
||||
https://raw.githubusercontent.com/LeCoupa/awesome-cheatsheets/refs/heads/master/tools/kubernetes.md
|
||||
Original cheat sheet: https://github.com/LeCoupa/awesome-cheatsheets
|
||||
@@ -1,109 +1,150 @@
|
||||
Overview of nano's shortcuts
|
||||
=====================================
|
||||
**GNU nano** is a small, modeless terminal editor: you start typing straight away, and every command is a control or meta chord shown along the bottom of the screen.
|
||||
|
||||
# The editor's keystrokes and their functions
|
||||
> ℹ️ nano's own help writes `^X` for `Ctrl+X` and `M-X` for `Alt+X` (Meta). This sheet spells them out. On macOS, `Alt` is usually `Esc` pressed first, or `Option` with "Use Option as Meta" enabled.
|
||||
|
||||
## File handling**
|
||||
## 🚀 Starting nano
|
||||
|
||||
| Shortcut | Description |
|
||||
|:--|:--|
|
||||
| Ctrl+S | Save current file |
|
||||
| Ctrl+O | Offer to write file ("Save as") |
|
||||
| Ctrl+R | Insert a file into current one |
|
||||
| Ctrl+X | Close buffer, exit from nano |
|
||||
|
||||
## **Editing**
|
||||
|
||||
| Shortcut | Description |
|
||||
|:--|:--|
|
||||
| Ctrl+K | Cut current line into cutbuffer |
|
||||
| Alt+6 | Copy current line into cutbuffer |
|
||||
| Ctrl+U | Paste contents of cutbuffer |
|
||||
| Alt+T | Cut until end of buffer |
|
||||
| Ctrl+\] | Complete current word |
|
||||
| Alt+3 | Comment/uncomment line/region |
|
||||
| Alt+U | Undo last action |
|
||||
| Alt+E | Redo last undone action |
|
||||
|
||||
## **Search and replace**
|
||||
|
||||
| Shortcut | Description |
|
||||
|:--|:--|
|
||||
| Ctrl+Q | Start backward search |
|
||||
| Ctrl+W | Start forward search |
|
||||
| Alt+Q | Find next occurrence backward |
|
||||
| Alt+W | Find next occurrence forward |
|
||||
| Alt+R | Start a replacing session |
|
||||
|
||||
## Deletion
|
||||
|
||||
| Shortcut | Description |
|
||||
|:--|:--|
|
||||
| Ctrl+H | Delete character before cursor |
|
||||
| Ctrl+D | Delete character under cursor |
|
||||
| Alt+Bsp | Delete word to the left |
|
||||
| Ctrl+Del | Delete word to the right |
|
||||
| Alt+Del | Delete current line |
|
||||
|
||||
## Operations
|
||||
```bash
|
||||
# open a file (created on save if it does not exist)
|
||||
nano <file>
|
||||
|
||||
| Shortcut | Description |
|
||||
|:--|:--|
|
||||
| Ctrl+T | Execute some command |
|
||||
| Ctrl+J | Justify paragraph or region |
|
||||
| Alt+J | Justify entire buffer |
|
||||
| Alt+B | Run a syntax check |
|
||||
| Alt+F | Run a formatter/fixer/arranger |
|
||||
| Alt+: | Start/stop recording of macro |
|
||||
| Alt+; | Replay macro |
|
||||
# jump straight to a line, or a line and column
|
||||
nano +42 <file>
|
||||
nano +42,8 <file>
|
||||
|
||||
## Moving around
|
||||
# show line numbers
|
||||
nano -l <file>
|
||||
|
||||
| Shortcut | Description |
|
||||
|:--|:--|
|
||||
| Ctrl+B | One character backward |
|
||||
| Ctrl+F | One character forward |
|
||||
| Ctrl+**←** | One word backward |
|
||||
| Ctrl+**→** | One word forward |
|
||||
| Ctrl+A | To start of line |
|
||||
| Ctrl+E | To end of line |
|
||||
| Ctrl+P | One line up |
|
||||
| Ctrl+N | One line down |
|
||||
| Ctrl+**↑** | To previous block |
|
||||
| Ctrl+**↓** | To next block |
|
||||
| Ctrl+Y | One page up |
|
||||
| Ctrl+V | One page down |
|
||||
| Alt+\\ | To top of buffer |
|
||||
| Alt+/ | To end of buffer |
|
||||
|
||||
## Special movement
|
||||
# enable mouse support
|
||||
nano -m <file>
|
||||
|
||||
| Shortcut | Description |
|
||||
|:--|:--|
|
||||
| Alt+G | Go to specified line |
|
||||
| Alt+\] | Go to complementary bracket |
|
||||
| Alt+**↑** | Scroll viewport up |
|
||||
| Alt+**↓** | Scroll viewport down |
|
||||
| Alt+< | Switch to preceding buffer |
|
||||
| Alt+> | Switch to succeeding buffer |
|
||||
|
||||
## Information
|
||||
# keep the indentation of the previous line
|
||||
nano -i <file>
|
||||
|
||||
| Shortcut | Description |
|
||||
|:--|:--|
|
||||
| Ctrl+C | Report cursor position |
|
||||
| Alt+D | Report line/word/character count |
|
||||
| Ctrl+G | Display help text |
|
||||
|
||||
## Various
|
||||
# convert typed tabs to spaces, 4 wide
|
||||
nano -ET4 <file>
|
||||
|
||||
| Shortcut | Description |
|
||||
|:--|:--|
|
||||
| Alt+A | Turn the mark on/off |
|
||||
| Tab | Indent marked region |
|
||||
| Shift+Tab | Unindent marked region |
|
||||
| Alt+V | Enter next keystroke verbatim |
|
||||
| Alt+N | Turn line numbers on/off |
|
||||
| Alt+P | Turn visible whitespace on/off |
|
||||
| Alt+X | Hide or unhide the help lines |
|
||||
| Ctrl+L | Refresh the screen |
|
||||
# do not wrap long lines
|
||||
nano -w <file>
|
||||
|
||||
# keep a backup of the original as file~
|
||||
nano -B <file>
|
||||
|
||||
# read-only view mode
|
||||
nano -v <file>
|
||||
|
||||
# open at the position you left last time
|
||||
nano -P <file>
|
||||
```
|
||||
|
||||
## 💾 File Handling
|
||||
|
||||
| Shortcut | Action |
|
||||
|----------|-----------------------------------|
|
||||
| `Ctrl+S` | Save the current file |
|
||||
| `Ctrl+O` | Write to a named file ("save as") |
|
||||
| `Ctrl+R` | Insert another file into this one |
|
||||
| `Ctrl+X` | Close the buffer and leave nano |
|
||||
| `Alt+<` | Switch to the previous buffer |
|
||||
| `Alt+>` | Switch to the next buffer |
|
||||
|
||||
## ✏️ Editing
|
||||
|
||||
| Shortcut | Action |
|
||||
|---------------------|----------------------------------------------|
|
||||
| `Ctrl+K` | Cut the current line into the cutbuffer |
|
||||
| `Alt+6` | Copy the current line into the cutbuffer |
|
||||
| `Ctrl+U` | Paste the cutbuffer |
|
||||
| `Alt+T` | Cut from the cursor to the end of the buffer |
|
||||
| `Alt+U` | Undo the last action |
|
||||
| `Alt+E` | Redo the last undone action |
|
||||
| `Alt+3` | Comment or uncomment the line or selection |
|
||||
| `Ctrl+]` | Complete the current word |
|
||||
| `Alt+A` | Set the mark — move to select |
|
||||
| `Tab` / `Shift+Tab` | Indent / unindent the selection |
|
||||
|
||||
## 🔎 Search & Replace
|
||||
|
||||
| Shortcut | Action |
|
||||
|----------|-----------------------------|
|
||||
| `Ctrl+W` | Search forward ("where is") |
|
||||
| `Ctrl+Q` | Search backward |
|
||||
| `Alt+W` | Repeat the search forward |
|
||||
| `Alt+Q` | Repeat the search backward |
|
||||
| `Alt+R` | Search and replace |
|
||||
|
||||
> 💡 At the search prompt, `Alt+C` toggles case sensitivity, `Alt+R` switches to regular expressions and `Alt+B` limits the search to the selection.
|
||||
|
||||
## ⌫ Deletion
|
||||
|
||||
| Shortcut | Action |
|
||||
|------------|---------------------------------------|
|
||||
| `Ctrl+H` | Delete the character to the left |
|
||||
| `Ctrl+D` | Delete the character under the cursor |
|
||||
| `Alt+Bksp` | Delete the word to the left |
|
||||
| `Ctrl+Del` | Delete the word to the right |
|
||||
| `Alt+Del` | Delete the whole line |
|
||||
|
||||
## 🧭 Moving Around
|
||||
|
||||
| Shortcut | Action |
|
||||
|---------------------|-------------------------------------------|
|
||||
| `Ctrl+B` / `Ctrl+F` | One character left / right |
|
||||
| `Ctrl+←` / `Ctrl+→` | One word left / right |
|
||||
| `Ctrl+A` / `Ctrl+E` | Start / end of the line |
|
||||
| `Ctrl+P` / `Ctrl+N` | One line up / down |
|
||||
| `Ctrl+↑` / `Ctrl+↓` | Previous / next block |
|
||||
| `Ctrl+Y` / `Ctrl+V` | One page up / down |
|
||||
| `Alt+\` / `Alt+/` | Top / bottom of the buffer |
|
||||
| `Alt+G` | Go to a line number |
|
||||
| `Alt+]` | Jump to the matching bracket |
|
||||
| `Alt+↑` / `Alt+↓` | Scroll the view without moving the cursor |
|
||||
|
||||
## 🔧 Operations
|
||||
|
||||
| Shortcut | Action |
|
||||
|----------|----------------------------------------------|
|
||||
| `Ctrl+T` | Run a command, or pipe the buffer through it |
|
||||
| `Ctrl+J` | Justify the paragraph or selection |
|
||||
| `Alt+J` | Justify the whole buffer |
|
||||
| `Alt+B` | Run a syntax check (linter) |
|
||||
| `Alt+F` | Run a formatter |
|
||||
| `Alt+:` | Start or stop recording a macro |
|
||||
| `Alt+;` | Replay the macro |
|
||||
|
||||
## ℹ️ Information & Display
|
||||
|
||||
| Shortcut | Action |
|
||||
|----------|------------------------------------|
|
||||
| `Ctrl+G` | Open the help text |
|
||||
| `Ctrl+C` | Report the cursor position |
|
||||
| `Alt+D` | Count lines, words and characters |
|
||||
| `Alt+N` | Toggle line numbers |
|
||||
| `Alt+P` | Toggle visible whitespace |
|
||||
| `Alt+X` | Hide or show the shortcut bar |
|
||||
| `Alt+V` | Insert the next keystroke verbatim |
|
||||
| `Ctrl+L` | Redraw the screen |
|
||||
|
||||
## ⚙️ Configuration
|
||||
|
||||
Persistent settings live in `~/.nanorc` (system-wide: `/etc/nanorc`).
|
||||
|
||||
```bash
|
||||
set linenumbers
|
||||
set mouse
|
||||
set tabsize 4
|
||||
set tabstospaces
|
||||
set autoindent
|
||||
set constantshow
|
||||
set softwrap
|
||||
set positionlog
|
||||
|
||||
# syntax highlighting definitions shipped with nano
|
||||
include /usr/share/nano/*.nanorc
|
||||
```
|
||||
|
||||
## 📚 Resources
|
||||
|
||||
- [Official documentation](https://www.nano-editor.org/docs.php)
|
||||
- [Manual page](https://man7.org/linux/man-pages/man1/nano.1.html)
|
||||
- Built-in help: `Ctrl+G`
|
||||
@@ -1,218 +1,418 @@
|
||||
**nginx** is a web server, reverse proxy and load balancer. Configuration is a tree of *contexts* — `main` → `events` / `http` → `server` → `location` — and a directive is only valid in the contexts that define it.
|
||||
|
||||
## 🛠 Service & CLI
|
||||
|
||||
```bash
|
||||
##############################################################################
|
||||
# NGINX
|
||||
# DOCUMENTATION: https://nginx.org/en/docs/
|
||||
##############################################################################
|
||||
# check the configuration before you break production
|
||||
sudo nginx -t
|
||||
|
||||
sudo nginx -t # Check syntax
|
||||
sudo systemctl status nginx # nginx current status
|
||||
sudo systemctl reload nginx # Reload nginx
|
||||
sudo systemctl restart nginx # Restart nginx
|
||||
sudo ln -s /etc/nginx/sites-available/example.com /etc/nginx/sites-enabled/ # Link website
|
||||
sudo tail -f /var/log/nginx/access.log # Tail logs to inspect requests
|
||||
# print the entire resolved configuration, includes and all
|
||||
sudo nginx -T
|
||||
|
||||
# *****************************************************************************
|
||||
# General Settings
|
||||
# *****************************************************************************
|
||||
# reload without dropping connections
|
||||
sudo nginx -s reload
|
||||
sudo systemctl reload nginx
|
||||
|
||||
# Ports
|
||||
# start, stop, restart, status
|
||||
sudo systemctl start nginx
|
||||
sudo systemctl stop nginx
|
||||
sudo systemctl restart nginx
|
||||
sudo systemctl status nginx
|
||||
|
||||
server {
|
||||
# Use HTTP protocol
|
||||
listen 80;
|
||||
|
||||
# Use HTTPS protocol
|
||||
listen 443 ssl;
|
||||
|
||||
# Listen on port 80 using IPv6
|
||||
listen [::]:80;
|
||||
|
||||
# Listen on port 80 using **only** IPv6
|
||||
listen [::]:80 ipv6only=on;
|
||||
# version, and the modules it was built with
|
||||
nginx -V
|
||||
|
||||
# enable a site (Debian/Ubuntu layout)
|
||||
sudo ln -s /etc/nginx/sites-available/example.com /etc/nginx/sites-enabled/
|
||||
|
||||
# watch the requests come in
|
||||
sudo tail -f /var/log/nginx/access.log
|
||||
sudo tail -f /var/log/nginx/error.log
|
||||
```
|
||||
|
||||
> 💡 `nginx -t` then `nginx -s reload` — never `restart` on a live server if you can avoid it. A reload keeps existing connections alive; a restart drops them.
|
||||
|
||||
## 📁 Where Things Live
|
||||
|
||||
| Path | What it holds |
|
||||
|-------------------------------|------------------------------------------------------|
|
||||
| `/etc/nginx/nginx.conf` | The main file — worker settings and the `http` block |
|
||||
| `/etc/nginx/conf.d/*.conf` | Included by default on RHEL-style installs |
|
||||
| `/etc/nginx/sites-available/` | Site configs on Debian/Ubuntu |
|
||||
| `/etc/nginx/sites-enabled/` | Symlinks to the sites that are actually live |
|
||||
| `/var/log/nginx/access.log` | Every request |
|
||||
| `/var/log/nginx/error.log` | Errors, and anything the workers complain about |
|
||||
| `/var/www/html` | The conventional document root |
|
||||
|
||||
## 🧱 Config Skeleton
|
||||
|
||||
```nginx
|
||||
user www-data;
|
||||
worker_processes auto;
|
||||
|
||||
events {
|
||||
worker_connections 1024;
|
||||
}
|
||||
|
||||
# Domain name (server_name)
|
||||
http {
|
||||
include /etc/nginx/mime.types;
|
||||
default_type application/octet-stream;
|
||||
sendfile on;
|
||||
keepalive_timeout 65;
|
||||
|
||||
server {
|
||||
listen 80;
|
||||
server_name example.com;
|
||||
|
||||
location / {
|
||||
root /var/www/html;
|
||||
index index.html;
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## 🎧 listen & server_name
|
||||
|
||||
```nginx
|
||||
server {
|
||||
# Listen to example.com
|
||||
# plain HTTP
|
||||
listen 80;
|
||||
|
||||
# HTTPS, with HTTP/2
|
||||
listen 443 ssl;
|
||||
http2 on;
|
||||
|
||||
# IPv6 as well, or IPv6 only
|
||||
listen [::]:80;
|
||||
listen [::]:80 ipv6only=on;
|
||||
|
||||
# the fallback server for requests that match no other server_name
|
||||
listen 80 default_server;
|
||||
|
||||
# one name, several names, wildcards, or a regex
|
||||
server_name example.com;
|
||||
|
||||
# Listen to multiple domains
|
||||
server_name example.com www.example.com;
|
||||
|
||||
# Listen to all sub-domains
|
||||
server_name *.example.com;
|
||||
|
||||
# Listen to all top-level domains
|
||||
server_name example.*;
|
||||
|
||||
# Listen to unspecified hostnames (listens to IP address itself)
|
||||
server_name ~^(?<sub>.+)\.example\.com$;
|
||||
|
||||
# requests that arrive with no Host header
|
||||
server_name "";
|
||||
}
|
||||
```
|
||||
|
||||
# *****************************************************************************
|
||||
# Serving Files
|
||||
# *****************************************************************************
|
||||
|
||||
# Static assets (traditional web server)
|
||||
|
||||
server {
|
||||
listen 80;
|
||||
server_name example.com;
|
||||
|
||||
root /path/to/website;
|
||||
# root /www/data/ for example
|
||||
|
||||
# If there's no 'root' inside, it will look for /www/data/index.html
|
||||
location / {
|
||||
}
|
||||
|
||||
# If there's no 'root' inside, it will look for /www/data/images/index.html
|
||||
location /images/ {
|
||||
}
|
||||
|
||||
# Since there's 'root' inside, it will look for /www/media/videos/index.html
|
||||
location /videos/ {
|
||||
root /www/media;
|
||||
}
|
||||
}
|
||||
|
||||
# *****************************************************************************
|
||||
# Redirects
|
||||
# *****************************************************************************
|
||||
|
||||
# 301 Permanent
|
||||
|
||||
server {
|
||||
# Redirect www.example.com to example.com
|
||||
listen 80;
|
||||
server_name www.example.com;
|
||||
return 301 http://example.com$request_uri;
|
||||
}
|
||||
|
||||
server {
|
||||
# Redirect http to https
|
||||
listen 80;
|
||||
server_name example.com;
|
||||
return 301 https://example.com$request_uri;
|
||||
}
|
||||
|
||||
# 302 Temporary
|
||||
|
||||
server {
|
||||
listen 80;
|
||||
server_name yourdomain.com;
|
||||
return 302 http://otherdomain.com;
|
||||
}
|
||||
|
||||
# *****************************************************************************
|
||||
# Reverse proxy
|
||||
# *****************************************************************************
|
||||
|
||||
# Useful for Node.js, Streamlit, Jupyter, etc
|
||||
|
||||
# Basic
|
||||
## 📄 Serving Files
|
||||
|
||||
```nginx
|
||||
server {
|
||||
listen 80;
|
||||
server_name example.com;
|
||||
|
||||
root /var/www/example.com;
|
||||
index index.html index.htm;
|
||||
|
||||
# try the file, then the directory, then fall back — the SPA pattern
|
||||
location / {
|
||||
proxy_pass http://0.0.0.0:3000;
|
||||
# where 0.0.0.0:3000 is your Node.js Server bound on 0.0.0.0 listing on port 3000
|
||||
try_files $uri $uri/ /index.html;
|
||||
}
|
||||
|
||||
# cache fingerprinted assets hard
|
||||
location ~* \.(js|css|png|jpe?g|gif|svg|woff2?)$ {
|
||||
expires 1y;
|
||||
add_header Cache-Control "public, immutable";
|
||||
access_log off;
|
||||
}
|
||||
|
||||
# a directory listing, when you actually want one
|
||||
location /downloads/ {
|
||||
autoindex on;
|
||||
}
|
||||
|
||||
# a single file
|
||||
location = /robots.txt {
|
||||
root /var/www/example.com;
|
||||
log_not_found off;
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
# Basic + (upstream)
|
||||
> ⚠️ **`root` vs `alias`**: `root` appends the whole URI to the path, `alias` replaces the matched prefix. With `location /static/ { root /var/www; }` a request for `/static/a.png` reads `/var/www/static/a.png`; with `alias /var/www/assets/;` it reads `/var/www/assets/a.png`. Always end an `alias` path with `/`.
|
||||
|
||||
upstream node_js {
|
||||
server 0.0.0.0:3000;
|
||||
# where 0.0.0.0:3000 is your Node.js Server bound on 0.0.0.0 listing on port 3000
|
||||
## 📍 Location Matching
|
||||
|
||||
nginx does not pick locations top to bottom — it picks by modifier, in this order:
|
||||
|
||||
| Modifier | Example | Meaning | Priority |
|
||||
|----------|--------------------------|-----------------------------------------------------|----------------------|
|
||||
| `=` | `location = /health` | Exact match | 1 — wins immediately |
|
||||
| `^~` | `location ^~ /static/` | Prefix match that stops regex matching | 2 |
|
||||
| `~` | `location ~ \.php$` | Case-sensitive regex, first match in file order | 3 |
|
||||
| `~*` | `location ~* \.(jpe?g)$` | Case-insensitive regex | 3 |
|
||||
| *(none)* | `location /images/` | Prefix match — the longest one wins if no regex did | 4 |
|
||||
|
||||
```nginx
|
||||
location = /health { return 200 "ok\n"; } # checked first, cheapest
|
||||
location ^~ /assets/ { root /var/www; } # never falls through to the regex below
|
||||
location ~* \.(png|jpg)$ { expires 30d; }
|
||||
location / { try_files $uri $uri/ =404; } # the catch-all
|
||||
```
|
||||
|
||||
## 🔀 Redirects & Rewrites
|
||||
|
||||
```nginx
|
||||
# permanent redirect to the canonical host
|
||||
server {
|
||||
listen 80;
|
||||
server_name www.example.com;
|
||||
return 301 https://example.com$request_uri;
|
||||
}
|
||||
|
||||
# force HTTPS
|
||||
server {
|
||||
listen 80;
|
||||
server_name example.com;
|
||||
|
||||
location / {
|
||||
proxy_pass http://node_js;
|
||||
}
|
||||
}
|
||||
|
||||
# Upgraded Connection (useful for applications with support for WebSockets)
|
||||
|
||||
upstream node_js {
|
||||
server 0.0.0.0:3000;
|
||||
}
|
||||
|
||||
server {
|
||||
listen 80;
|
||||
server_name example.com;
|
||||
|
||||
location / {
|
||||
proxy_pass http://node_js;
|
||||
proxy_redirect off;
|
||||
proxy_http_version 1.1;
|
||||
proxy_set_header Upgrade $http_upgrade;
|
||||
proxy_set_header Connection "upgrade";
|
||||
proxy_set_header Host $host;
|
||||
|
||||
}
|
||||
}
|
||||
|
||||
# *****************************************************************************
|
||||
# HTTPS Protocol
|
||||
# *****************************************************************************
|
||||
|
||||
# The majority of SSL options depend on what your application does or needs
|
||||
|
||||
server {
|
||||
listen 443 ssl http2;
|
||||
server_name example.com;
|
||||
|
||||
ssl on;
|
||||
|
||||
ssl_certificate /path/to/cert.pem;
|
||||
ssl_certificate_key /path/to/privkey.pem;
|
||||
|
||||
ssl_stapling on;
|
||||
ssl_stapling_verify on;
|
||||
ssl_trusted_certificate /path/to/fullchain.pem;
|
||||
|
||||
ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
|
||||
ssl_session_timeout 1d;
|
||||
ssl_session_cache shared:SSL:50m;
|
||||
add_header Strict-Transport-Security max-age=15768000;
|
||||
}
|
||||
|
||||
# Permanent redirect to HTTPS secured domain
|
||||
|
||||
server {
|
||||
listen 80;
|
||||
server_name yourdomain.com;
|
||||
return 301 https://$host$request_uri;
|
||||
}
|
||||
|
||||
# You can easily secure you website/app using Let's Encrypt.
|
||||
# Go https://certbot.eff.org/lets-encrypt/ubuntuxenial-nginx.html for more information
|
||||
# temporary redirect of a single path
|
||||
location /old-page {
|
||||
return 302 /new-page;
|
||||
}
|
||||
|
||||
# *****************************************************************************
|
||||
# Load Balancing
|
||||
# *****************************************************************************
|
||||
# rewrite with a captured segment
|
||||
location /blog/ {
|
||||
rewrite ^/blog/(.*)$ /articles/$1 permanent;
|
||||
}
|
||||
|
||||
# Useful for large applications running in multiple instances. Below example is for reverse proxy
|
||||
upstream node_js {
|
||||
server 0.0.0.0:3000;
|
||||
server 0.0.0.0:4000;
|
||||
server 127.155.142.421;
|
||||
# serve a maintenance page for everyone but your own IP
|
||||
location / {
|
||||
if ($remote_addr != 203.0.113.7) {
|
||||
return 503;
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
> 💡 Prefer `return` over `rewrite` — it is faster and clearer. Reach for `rewrite` only when you need to transform the path.
|
||||
|
||||
## 🔁 Reverse Proxy
|
||||
|
||||
```nginx
|
||||
server {
|
||||
listen 80;
|
||||
server_name app.example.com;
|
||||
|
||||
location / {
|
||||
proxy_pass http://127.0.0.1:3000;
|
||||
|
||||
# pass the client's details through to the app
|
||||
proxy_set_header Host $host;
|
||||
proxy_set_header X-Real-IP $remote_addr;
|
||||
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
||||
proxy_set_header X-Forwarded-Proto $scheme;
|
||||
|
||||
# timeouts: raise these for slow backends
|
||||
proxy_connect_timeout 60s;
|
||||
proxy_send_timeout 60s;
|
||||
proxy_read_timeout 60s;
|
||||
|
||||
# buffering off for streaming responses (SSE, logs)
|
||||
proxy_buffering on;
|
||||
}
|
||||
|
||||
# WebSockets need the upgrade headers
|
||||
location /ws/ {
|
||||
proxy_pass http://127.0.0.1:3000;
|
||||
proxy_http_version 1.1;
|
||||
proxy_set_header Upgrade $http_upgrade;
|
||||
proxy_set_header Connection "upgrade";
|
||||
proxy_read_timeout 3600s;
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
> ⚠️ The trailing slash on `proxy_pass` changes everything. `proxy_pass http://backend;` forwards `/api/users` as `/api/users`; `proxy_pass http://backend/;` strips the matched location prefix first.
|
||||
|
||||
## ⚖️ Load Balancing
|
||||
|
||||
```nginx
|
||||
upstream backend {
|
||||
# least_conn; # send to the server with the fewest connections
|
||||
# ip_hash; # sticky sessions by client IP
|
||||
# hash $request_uri; # sticky by URI
|
||||
|
||||
server 10.0.0.1:3000 weight=3;
|
||||
server 10.0.0.2:3000;
|
||||
server 10.0.0.3:3000 max_fails=3 fail_timeout=30s;
|
||||
server 10.0.0.4:3000 backup;
|
||||
|
||||
keepalive 32; # reuse upstream connections
|
||||
}
|
||||
|
||||
server {
|
||||
listen 80;
|
||||
server_name example.com;
|
||||
|
||||
location / {
|
||||
proxy_pass http://node_js;
|
||||
proxy_pass http://backend;
|
||||
proxy_http_version 1.1;
|
||||
proxy_set_header Connection ""; # required for keepalive to work
|
||||
}
|
||||
}
|
||||
```
|
||||
```
|
||||
|
||||
| Method | Behaviour |
|
||||
|-------------------------|-------------------------------------------------|
|
||||
| *(default)* | Round robin, honouring `weight` |
|
||||
| `least_conn` | Fewest active connections wins |
|
||||
| `ip_hash` | Same client IP always reaches the same server |
|
||||
| `hash <key>` | Distribute by any variable, e.g. `$request_uri` |
|
||||
| `random two least_conn` | Pick two at random, then the less busy one |
|
||||
|
||||
## 🔐 HTTPS
|
||||
|
||||
```nginx
|
||||
server {
|
||||
listen 443 ssl;
|
||||
http2 on;
|
||||
server_name example.com;
|
||||
|
||||
ssl_certificate /etc/letsencrypt/live/example.com/fullchain.pem;
|
||||
ssl_certificate_key /etc/letsencrypt/live/example.com/privkey.pem;
|
||||
|
||||
ssl_protocols TLSv1.2 TLSv1.3;
|
||||
ssl_prefer_server_ciphers off;
|
||||
ssl_session_cache shared:SSL:10m;
|
||||
ssl_session_timeout 1d;
|
||||
ssl_stapling on;
|
||||
ssl_stapling_verify on;
|
||||
|
||||
# tell browsers to stay on HTTPS
|
||||
add_header Strict-Transport-Security "max-age=63072000" always;
|
||||
}
|
||||
```
|
||||
|
||||
```bash
|
||||
# get and renew certificates automatically
|
||||
sudo certbot --nginx -d example.com -d www.example.com
|
||||
sudo certbot renew --dry-run
|
||||
```
|
||||
|
||||
## 🚦 Limits & Access Control
|
||||
|
||||
```nginx
|
||||
http {
|
||||
# 10 MB of state ≈ 160k addresses; 10 requests/second per IP
|
||||
limit_req_zone $binary_remote_addr zone=req_limit:10m rate=10r/s;
|
||||
limit_conn_zone $binary_remote_addr zone=conn_limit:10m;
|
||||
|
||||
server {
|
||||
# allow short bursts, queue the rest without delaying legitimate users
|
||||
location /api/ {
|
||||
limit_req zone=req_limit burst=20 nodelay;
|
||||
limit_conn conn_limit 10;
|
||||
}
|
||||
|
||||
# the size of the largest upload you accept
|
||||
client_max_body_size 25m;
|
||||
|
||||
# HTTP basic auth (htpasswd -c /etc/nginx/.htpasswd user)
|
||||
location /admin/ {
|
||||
auth_basic "Restricted";
|
||||
auth_basic_user_file /etc/nginx/.htpasswd;
|
||||
}
|
||||
|
||||
# allow a subnet, deny the rest
|
||||
location /internal/ {
|
||||
allow 10.0.0.0/8;
|
||||
deny all;
|
||||
}
|
||||
|
||||
# do not serve dotfiles
|
||||
location ~ /\.(?!well-known) {
|
||||
deny all;
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## 🗜 Performance
|
||||
|
||||
```nginx
|
||||
http {
|
||||
# compress text responses
|
||||
gzip on;
|
||||
gzip_vary on;
|
||||
gzip_min_length 1024;
|
||||
gzip_types text/plain text/css application/json application/javascript
|
||||
text/xml application/xml image/svg+xml;
|
||||
|
||||
# kernel-level file sending
|
||||
sendfile on;
|
||||
tcp_nopush on;
|
||||
tcp_nodelay on;
|
||||
|
||||
# keep connections around for reuse
|
||||
keepalive_timeout 65;
|
||||
keepalive_requests 1000;
|
||||
|
||||
# cache open file handles
|
||||
open_file_cache max=10000 inactive=30s;
|
||||
open_file_cache_valid 60s;
|
||||
|
||||
# a proxy cache for upstream responses
|
||||
proxy_cache_path /var/cache/nginx keys_zone=app_cache:10m max_size=1g inactive=60m;
|
||||
|
||||
server {
|
||||
location / {
|
||||
proxy_cache app_cache;
|
||||
proxy_cache_valid 200 10m;
|
||||
add_header X-Cache-Status $upstream_cache_status;
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## 🧪 Useful Variables
|
||||
|
||||
| Variable | Holds |
|
||||
|------------------------------|--------------------------------------------------|
|
||||
| `$host` | The Host header, or the server name that matched |
|
||||
| `$uri` | The normalised path, without the query string |
|
||||
| `$request_uri` | The original path **with** the query string |
|
||||
| `$args` / `$arg_name` | The whole query string / one named parameter |
|
||||
| `$scheme` | `http` or `https` |
|
||||
| `$remote_addr` | The client's IP address |
|
||||
| `$proxy_add_x_forwarded_for` | The existing chain plus `$remote_addr` |
|
||||
| `$http_<name>` | Any request header, e.g. `$http_user_agent` |
|
||||
| `$request_method` | `GET`, `POST`, … |
|
||||
| `$status` | The response status |
|
||||
| `$upstream_addr` | Which backend actually served the request |
|
||||
| `$upstream_response_time` | How long the backend took |
|
||||
| `$request_time` | How long nginx took, start to finish |
|
||||
|
||||
## 🩺 Troubleshooting
|
||||
|
||||
| Symptom | Usual cause |
|
||||
|----------------------------------|----------------------------------------------------------------------------------|
|
||||
| `502 Bad Gateway` | The backend is down, or refused the connection — check the app and `proxy_pass` |
|
||||
| `504 Gateway Timeout` | The backend is too slow — raise `proxy_read_timeout`, then fix the app |
|
||||
| `413 Request Entity Too Large` | Raise `client_max_body_size` |
|
||||
| `403 Forbidden` on a static file | The worker user cannot read the path, or a directory lacks `x` |
|
||||
| The wrong site is served | No `server_name` matched, so the `default_server` answered |
|
||||
| Changes have no effect | The config was never reloaded, or the file is not symlinked into `sites-enabled` |
|
||||
|
||||
```bash
|
||||
# see the config nginx is actually running, includes resolved
|
||||
sudo nginx -T | less
|
||||
|
||||
# which worker user needs read access
|
||||
ps -o user= -C nginx | sort -u
|
||||
|
||||
# turn up the error log while debugging
|
||||
# error_log /var/log/nginx/error.log debug;
|
||||
```
|
||||
|
||||
## 📚 Resources
|
||||
|
||||
- [Official documentation](https://nginx.org/en/docs/)
|
||||
- [Directive index](https://nginx.org/en/docs/dirindex.html)
|
||||
- [Variable index](https://nginx.org/en/docs/varindex.html)
|
||||
- [Mozilla SSL configuration generator](https://ssl-config.mozilla.org/)
|
||||
- [Certbot — Let's Encrypt](https://certbot.eff.org/)
|
||||
@@ -1,220 +1,434 @@
|
||||
## Variables
|
||||
**PowerShell** is a shell and scripting language built on .NET. Unlike POSIX shells it passes **objects** down the pipeline rather than text, so `Get-Process | Sort-Object CPU` sorts real numbers instead of parsing columns.
|
||||
|
||||
| Syntax | Description |
|
||||
|-------|-------------|
|
||||
| `$var = "string"` | Assign variable |
|
||||
| `[Type]$var="typedVar"` | Assign strong typed variable |
|
||||
| `[ValidateRange(1,9)][int]$x=1` | Strong typed attribute‑controlled variable |
|
||||
| `$a,$b,$c = 0` or `$a,$b = 'a','b'` | Assign multiple variables |
|
||||
| `$a,$b = $b,$a` | Flip variables |
|
||||
| *Scopes* | global, local, private, script |
|
||||
| `$global:var = "var"` | Assign global scoped variable |
|
||||
> ℹ️ **PowerShell 7+** (`pwsh`) is cross-platform and current; **Windows PowerShell 5.1** (`powershell.exe`) ships with Windows and is in maintenance. Check with `$PSVersionTable`.
|
||||
|
||||
## Arrays
|
||||
## 🔎 Finding Your Way
|
||||
|
||||
| Syntax | Description |
|
||||
|--------|-------------|
|
||||
| `"a","b","c"` | Array of strings |
|
||||
| `@()` | Empty array |
|
||||
| `1,(2,3),4` | Array within array |
|
||||
| `,"hi"` | Array of one element |
|
||||
| `$arr[5]` | Sixth element |
|
||||
| `$arr[2..20]` | Elements 3–21 |
|
||||
| `$arr[-1]` | Last element |
|
||||
| `$arr[-3..-1]` | Last three elements |
|
||||
| `$arr[1,4+6..9]` | Elements at positions 1,4,6–9 |
|
||||
| `@(Get-Process)` | Force result to array |
|
||||
| `$arr[($arr.length-1)..0]` | Reverse array |
|
||||
| `$arr[1] += 200` | Add to existing item |
|
||||
| `$b = $arr[0,1 + 3..6]` | New array from elements |
|
||||
| `$z = $arrA + $arrB` | Combine arrays |
|
||||
```powershell
|
||||
# what commands exist for a noun?
|
||||
Get-Command *service*
|
||||
Get-Command -Module Microsoft.PowerShell.Management
|
||||
|
||||
## Hash Tables (Associative Arrays)
|
||||
# help, with the part everyone actually wants
|
||||
Get-Help Get-ChildItem -Examples
|
||||
Get-Help Get-ChildItem -Full
|
||||
Update-Help
|
||||
|
||||
| Syntax | Description |
|
||||
|--------|-------------|
|
||||
| `$hash = @{}` | Empty hash table |
|
||||
| `@{foo=1; bar='value2'}` | Initialize hash |
|
||||
| `[ordered]@{a=1; b=2; c=3}` | Ordered dictionary |
|
||||
| `$hash.key1 = 1` | Assign value |
|
||||
| `$hash.key1` | Get value |
|
||||
| `$hash["key1"]` | Get value |
|
||||
| `$hash.GetEnumerator \| sort Key` | Sort by key |
|
||||
| `[pscustomobject]@{x=1;z="z"}` | Custom object |
|
||||
# what properties and methods does this object have?
|
||||
Get-Process | Get-Member
|
||||
|
||||
## Strings
|
||||
# what did that command actually return?
|
||||
Get-Process | Select-Object -First 1 | Format-List *
|
||||
```
|
||||
|
||||
| Syntax | Description |
|
||||
|--------|-------------|
|
||||
| `"$var expand"` | Expand variables |
|
||||
| `'$var no expand'` | No expansion |
|
||||
| `@" ... "@` | Here‑string (double quotes allow expressions) |
|
||||
## 📦 Variables
|
||||
|
||||
## Comments & Escape Characters
|
||||
| Syntax | Description |
|
||||
|---------------------------------|-----------------------------------------------|
|
||||
| `$var = "string"` | Assign a variable |
|
||||
| `[int]$var = 5` | Strongly typed variable |
|
||||
| `[ValidateRange(1,9)][int]$x=1` | Typed and constrained |
|
||||
| `$a, $b = 'a', 'b'` | Assign several at once |
|
||||
| `$a, $b = $b, $a` | Swap two values |
|
||||
| `$global:var = "v"` | Scope: `global`, `script`, `local`, `private` |
|
||||
| `${my var}` | A name containing spaces or punctuation |
|
||||
| `$env:PATH` | An environment variable |
|
||||
| `Remove-Variable x` | Delete a variable |
|
||||
|
||||
| Syntax | Description |
|
||||
|--------|-------------|
|
||||
| `# Comment` | Comment |
|
||||
| `<# comment #>` | Multiline comment |
|
||||
| `"A '\"test`\""` | Escape `'` |
|
||||
| `` `t `` | Tab |
|
||||
| `` `n `` | New line |
|
||||
| `` ` `` | Line continuation |
|
||||
## 📚 Arrays
|
||||
|
||||
## Basics of Text & Files
|
||||
| Syntax | Description |
|
||||
|---------------------------------------------------------|---------------------------------------------------|
|
||||
| `"a","b","c"` | Array of strings |
|
||||
| `@()` | Empty array |
|
||||
| `,"hi"` | Array of exactly one element |
|
||||
| `1,(2,3),4` | Nested array |
|
||||
| `$arr[5]` | Sixth element (0-based) |
|
||||
| `$arr[2..20]` | A range of elements |
|
||||
| `$arr[-1]` | The last element |
|
||||
| `$arr[-3..-1]` | The last three |
|
||||
| `$arr[1,4+6..9]` | Positions 1, 4 and 6–9 |
|
||||
| `$arr[($arr.length-1)..0]` | Reversed |
|
||||
| `@(Get-Process)` | Force a single result into an array |
|
||||
| `$z = $arrA + $arrB` | Concatenate |
|
||||
| `$list = [System.Collections.Generic.List[int]]::new()` | A growable list — much faster than `+=` in a loop |
|
||||
|
||||
| Command | Description |
|
||||
|---------|-------------|
|
||||
| `Get-Location` | Current directory |
|
||||
| `Set-Location` | Change directory |
|
||||
| `Get-Content` | Read file |
|
||||
| `Add-Content` | Append |
|
||||
| `Set-Content` | Overwrite |
|
||||
| `Out-File` | Formatted output |
|
||||
| `Out-Null` | Discard output |
|
||||
| `Out-String` | Convert to string |
|
||||
| `Copy-Item` | Copy |
|
||||
| `Remove-Item` | Delete |
|
||||
| `Move-Item` | Move |
|
||||
| `Rename-Item` | Rename |
|
||||
| `Set-Item` | Set file contents |
|
||||
| `Clear-Item` | Clear file |
|
||||
| `New-Item` | New file/dir |
|
||||
## 🗂 Hash Tables
|
||||
|
||||
## Flow Control
|
||||
| Syntax | Description |
|
||||
|--------------------------------------------|----------------------------|
|
||||
| `$hash = @{}` | Empty hash table |
|
||||
| `@{foo=1; bar='two'}` | Initialise with values |
|
||||
| `[ordered]@{a=1; b=2}` | Keeps insertion order |
|
||||
| `$hash.key1` / `$hash["key1"]` | Read a value |
|
||||
| `$hash.key1 = 1` | Write a value |
|
||||
| `$hash.Remove("key1")` | Delete a key |
|
||||
| `$hash.ContainsKey("key1")` | Test for a key |
|
||||
| `$hash.GetEnumerator() \| Sort-Object Key` | Iterate in key order |
|
||||
| `[pscustomobject]@{x=1; z="z"}` | Turn a hash into an object |
|
||||
|
||||
| Syntax | Description |
|
||||
|--------|-------------|
|
||||
| `If($x -eq 5){ } elseif($x -gt 5){ } else{ }` | If |
|
||||
| `$x=1; while($x -lt 10){$x;$x++}` | While |
|
||||
| `for($i=0; $i -lt 10; $i++){ $i }` | For |
|
||||
| `foreach($file in dir C:\){$file.Name}` | Foreach |
|
||||
| `1..10 \| foreach{$_}` | Foreach pipeline |
|
||||
| `Switch -options (...) { Pattern { } Default { } }` | Switch |
|
||||
## 🔤 Strings
|
||||
|
||||
## Operators
|
||||
| Syntax | Description |
|
||||
|----------------------------------|-------------------------------------------------------|
|
||||
| `"$var expands"` | Double quotes interpolate |
|
||||
| `'$var does not'` | Single quotes are literal |
|
||||
| `"Total: $($items.Count)"` | `$( )` evaluates an expression inside a string |
|
||||
| `"{0} of {1}" -f $a, $b` | The format operator |
|
||||
| `` "col1`tcol2`n" `` | `` `t `` tab, `` `n `` newline, `` `` ` `` `` escapes |
|
||||
| `@"…"@` / `@'…'@` | Here-string, expanding / literal |
|
||||
| `"abc".ToUpper()` | .NET methods work on strings |
|
||||
| `$s -split ','` / `$a -join ','` | Split and join |
|
||||
| `$s.Trim()` / `.PadLeft(5)` | Trim and pad |
|
||||
| `$s -replace 'a','b'` | Regex replace |
|
||||
|
||||
### Assignment, Logical, Comparison
|
||||
## 💬 Comments & Escaping
|
||||
|
||||
| Syntax | Description |
|
||||
|--------|-------------|
|
||||
| `= += == *= /= %= ++ --` | Assignment |
|
||||
| `-and -or -xor -not !` | Logical |
|
||||
| `-eq -ne` | Equal / not equal |
|
||||
| `-gt -ge` | Greater than / ≥ |
|
||||
| `-lt -le` | Less than / ≤ |
|
||||
| `-replace` | Replace text |
|
||||
| `-match -notmatch` | Regex |
|
||||
| `-like -notlike` | Wildcards |
|
||||
| `-contains -notcontains` | Array contains |
|
||||
| `-in -notin` | Reverse contains |
|
||||
```powershell
|
||||
# a single-line comment
|
||||
|
||||
## Other Operators
|
||||
<#
|
||||
a block comment, also used for help text
|
||||
#>
|
||||
|
||||
| Syntax | Description |
|
||||
|--------|-------------|
|
||||
| `-split` | Split string |
|
||||
| `-join` | Join strings |
|
||||
| `..` | Range |
|
||||
| `-is -isnot` | Type check |
|
||||
| `-as` | Cast |
|
||||
| `-f` | Format strings |
|
||||
| `[type]` | Cast operator |
|
||||
| `$( )` | Subexpression |
|
||||
| `@()` | Array subexpression |
|
||||
| `&` | Invocation |
|
||||
# the backtick is the escape character
|
||||
Write-Output "He said `"hi`""
|
||||
Write-Output "line one`nline two"
|
||||
|
||||
## Objects
|
||||
# and the line-continuation character
|
||||
Get-ChildItem -Path C:\ `
|
||||
-Recurse `
|
||||
-Filter *.log
|
||||
```
|
||||
|
||||
| Syntax | Description |
|
||||
|--------|-------------|
|
||||
| `(Get-Date).Date` | Date property |
|
||||
| `Get-Date \| Get-Member` | List members |
|
||||
| `[DateTime]::Now` | Static property |
|
||||
| `"string".ToUpper()` | Method call |
|
||||
| `[system.Net.Dns]::GetHostByAddress("127.0.0.1")` | Static method |
|
||||
| `new-object -com excel.application` | Excel COM object |
|
||||
## 📁 Files & Paths
|
||||
|
||||
## Filter, Sort, Group, Format
|
||||
| Command | Does |
|
||||
|-----------------------------------------------------------|---------------------------------------|
|
||||
| `Get-Location` (`pwd`) | Current directory |
|
||||
| `Set-Location` (`cd`) | Change directory |
|
||||
| `Get-ChildItem` (`ls`, `dir`) | List a directory |
|
||||
| `Get-Content` (`cat`) | Read a file |
|
||||
| `Set-Content` / `Add-Content` | Overwrite / append |
|
||||
| `Out-File` / `Out-Null` / `Out-String` | Write to a file / discard / stringify |
|
||||
| `New-Item -ItemType Directory` | Create a file or folder |
|
||||
| `Copy-Item` / `Move-Item` / `Rename-Item` / `Remove-Item` | Copy, move, rename, delete |
|
||||
| `Test-Path` | Does it exist? |
|
||||
| `Split-Path -Parent` | The directory part of a path |
|
||||
| `Join-Path a b` | Build a path portably |
|
||||
| `Resolve-Path` | Expand to a full path |
|
||||
| `Get-FileHash file.zip` | Checksum a file |
|
||||
|
||||
| Command | Description |
|
||||
|---------|-------------|
|
||||
| `dir C:\pub \| where LastWriteTime -gt (Get-Date).AddDays(-1)` | Files modified yesterday |
|
||||
| `ps \| where { $_.path -like "C:\windows\system32*" -and $_.company -notlike "Microsoft*" }` | Filter processes |
|
||||
| `ps Explorer \| select ProcessName -ExpandProperty Modules \| format-list` | Expand modules |
|
||||
| `ps \| sort WorkingSet \| select -Last 5` | Top 5 memory |
|
||||
| `"a","b","a" \| select -Unique` | Unique values |
|
||||
| `Get-Service \| Group-Object Status` | Group by status |
|
||||
| `dir \| Group-Object { $_.Length -gt 100KB }` | Group by size |
|
||||
| `Get-Content pcs.txt \| Select-String "q-" \| sls "win7"` | Search text |
|
||||
| `ps \| Format-Table Name,StartTime -AutoSize` | Format table |
|
||||
| `ps \| Format-Table ProcessName, @{Label="Total Run Time"; Expression={(Get-Date)-$_.StartTime}}` | Custom column |
|
||||
| `Get-EventLog -Log System \| select -First 5 \| ft -wrap` | First 5 events |
|
||||
| `gi C:\Users \| fl *` | All properties |
|
||||
| `"{0}``t{1}``n" -f $a,5` | Format operator |
|
||||
```powershell
|
||||
# read a large file line by line without loading it all
|
||||
Get-Content big.log -ReadCount 1000 | ForEach-Object { $_ }
|
||||
|
||||
## Common Commands
|
||||
# tail -f
|
||||
Get-Content app.log -Wait -Tail 20
|
||||
|
||||
- `Get-EventLog`
|
||||
- `Get-WinEvent`
|
||||
- `Get-CimInstance`
|
||||
- `Get-Date`
|
||||
- `Start-Sleep`
|
||||
- `Compare-Object`
|
||||
- `Start-Job`
|
||||
- `Get-Credential`
|
||||
- `Test-Connection`
|
||||
- `New-PSSession`
|
||||
- `Test-Path`
|
||||
- `Split-Path`
|
||||
# every .log modified in the last day
|
||||
Get-ChildItem C:\logs -Recurse -Filter *.log |
|
||||
Where-Object LastWriteTime -gt (Get-Date).AddDays(-1)
|
||||
```
|
||||
|
||||
## Importing, Exporting, Converting
|
||||
## 🔀 Flow Control
|
||||
|
||||
- `Export-Clixml`
|
||||
- `Import-Clixml`
|
||||
- `ConvertTo-Xml`
|
||||
- `ConvertTo-Html`
|
||||
- `Export-Csv`
|
||||
- `Import-Csv`
|
||||
- `ConvertTo-Csv`
|
||||
- `ConvertFrom-Csv`
|
||||
```powershell
|
||||
if ($x -eq 5) { "five" } elseif ($x -gt 5) { "more" } else { "less" }
|
||||
|
||||
## Automatic Variables
|
||||
while ($x -lt 10) { $x; $x++ }
|
||||
|
||||
| Variable | Description |
|
||||
|----------|-------------|
|
||||
| `$_` / `$PSItem` | Pipeline object |
|
||||
| `$Args` | Script args |
|
||||
| `$Error` | Errors |
|
||||
| `$True` / `$False` | Boolean |
|
||||
| `$null` | Null |
|
||||
| `$profile` | Profile paths |
|
||||
do { $x++ } while ($x -lt 10)
|
||||
|
||||
## PSDrives
|
||||
for ($i = 0; $i -lt 10; $i++) { $i }
|
||||
|
||||
| Drive | Description |
|
||||
|-------|-------------|
|
||||
| `Alias:` | Aliases |
|
||||
| `Cert:` | Certificates |
|
||||
| `Env:` | Environment variables |
|
||||
| `Function:` | Functions |
|
||||
| `HKLM:` | Registry (Local Machine) |
|
||||
| `HKCU:` | Registry (Current User) |
|
||||
| `Variable:` | Variables |
|
||||
| `WSMan:` | WinRM |
|
||||
| `AD:` | Active Directory |
|
||||
| `Set-Location HKLM:` | Navigate registry |
|
||||
| `gci variable:` | List variables |
|
||||
foreach ($file in Get-ChildItem C:\) { $file.Name }
|
||||
|
||||
## Regular Expressions
|
||||
1..10 | ForEach-Object { $_ * 2 }
|
||||
|
||||
| Pattern | Meaning |
|
||||
|---------|---------|
|
||||
| `\w` | Word char `[a-zA-Z0-9]` |
|
||||
| `\W` | Non‑word char |
|
||||
| `\s` | Whitespace |
|
||||
| `\S` | Non‑whitespace |
|
||||
| `\d` / `\D` | Digit / non‑digit |
|
||||
| `{n} {n,} {n,m}` | Quantifiers |
|
||||
| More | Google .NET Regex |
|
||||
# parallel, PowerShell 7+
|
||||
1..10 | ForEach-Object -Parallel { Start-Sleep 1; $_ } -ThrottleLimit 5
|
||||
|
||||
switch ($value) {
|
||||
'a' { 'letter a'; break }
|
||||
{ $_ -gt 10 } { 'big' }
|
||||
default { 'something else' }
|
||||
}
|
||||
```
|
||||
|
||||
## ⚖️ Operators
|
||||
|
||||
| Operator | Meaning |
|
||||
|--------------------------------|-----------------------------------------|
|
||||
| `= += -= *= /= %= ++ --` | Assignment |
|
||||
| `-eq` / `-ne` | Equal / not equal |
|
||||
| `-gt` `-ge` `-lt` `-le` | Numeric comparison |
|
||||
| `-and` `-or` `-xor` `-not` `!` | Logical |
|
||||
| `-like` / `-notlike` | Wildcard match (`*`, `?`) |
|
||||
| `-match` / `-notmatch` | Regex match — fills `$Matches` |
|
||||
| `-replace 'a','b'` | Regex replace |
|
||||
| `-contains` / `-in` | Array membership, either way round |
|
||||
| `-split` / `-join` | Split a string / join an array |
|
||||
| `-is` / `-isnot` / `-as` | Type test and conversion |
|
||||
| `-f` | Format a string |
|
||||
| `..` | Range |
|
||||
| `$( )` / `@( )` | Sub-expression / array sub-expression |
|
||||
| `&` / `.` | Invoke a command / dot-source a script |
|
||||
| `??` / `??=` | Null-coalescing (PowerShell 7+) |
|
||||
| `?.` / `?[ ]` | Null-conditional access (PowerShell 7+) |
|
||||
|
||||
> ⚠️ Comparisons are **case-insensitive** by default. Use `-ceq`, `-clike`, `-cmatch` when case matters.
|
||||
|
||||
## 🧱 Objects & the Pipeline
|
||||
|
||||
```powershell
|
||||
# properties and methods
|
||||
(Get-Date).Date
|
||||
(Get-Date).AddDays(-7)
|
||||
"string".ToUpper()
|
||||
|
||||
# static members
|
||||
[DateTime]::Now
|
||||
[Math]::Round(3.14159, 2)
|
||||
[System.Net.Dns]::GetHostByAddress("127.0.0.1")
|
||||
|
||||
# build your own objects
|
||||
[pscustomobject]@{ Name = 'web01'; Status = 'up' }
|
||||
|
||||
# add a calculated property
|
||||
Get-Process | Select-Object Name, @{ Name = 'MB'; Expression = { [math]::Round($_.WorkingSet / 1MB, 1) } }
|
||||
```
|
||||
|
||||
## 🔍 Filter, Sort, Group, Format
|
||||
|
||||
| Example | Does |
|
||||
|-----------------------------------------------------------|---------------------------|
|
||||
| `Get-Process \| Where-Object CPU -gt 100` | Filter (simple syntax) |
|
||||
| `Get-Process \| Where-Object { $_.Name -like "chrome*" }` | Filter (script block) |
|
||||
| `Get-Process \| Sort-Object WorkingSet -Descending` | Sort |
|
||||
| `Get-Process \| Select-Object -First 5` | Take the first few |
|
||||
| `"a","b","a" \| Select-Object -Unique` | Deduplicate |
|
||||
| `Get-Service \| Group-Object Status` | Group |
|
||||
| `Get-Process \| Measure-Object WorkingSet -Sum -Average` | Aggregate |
|
||||
| `Get-Process \| Select-Object -ExpandProperty Modules` | Flatten a nested property |
|
||||
| `Get-Process \| Format-Table Name, Id -AutoSize` | Table output |
|
||||
| `Get-Item C:\ \| Format-List *` | Every property |
|
||||
| `Get-Content log.txt \| Select-String "error"` | grep |
|
||||
| `Compare-Object $a $b` | Diff two collections |
|
||||
|
||||
> 💡 `Format-*` is always the **last** step — its output is display text, not objects, so nothing downstream can filter it.
|
||||
|
||||
## 🧰 Functions & Scripts
|
||||
|
||||
```powershell
|
||||
function Get-Square {
|
||||
param(
|
||||
[Parameter(Mandatory)][int]$Number,
|
||||
[switch]$Verbose
|
||||
)
|
||||
$Number * $Number
|
||||
}
|
||||
|
||||
Get-Square -Number 7
|
||||
|
||||
# an advanced function: pipeline input, -WhatIf, -Verbose for free
|
||||
function Remove-OldLog {
|
||||
[CmdletBinding(SupportsShouldProcess)]
|
||||
param(
|
||||
[Parameter(ValueFromPipeline)][string]$Path,
|
||||
[int]$Days = 30
|
||||
)
|
||||
process {
|
||||
if ($PSCmdlet.ShouldProcess($Path, "delete")) {
|
||||
Remove-Item $Path
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## 🚨 Error Handling
|
||||
|
||||
```powershell
|
||||
try {
|
||||
Get-Content missing.txt -ErrorAction Stop
|
||||
}
|
||||
catch [System.IO.FileNotFoundException] {
|
||||
Write-Warning "no such file"
|
||||
}
|
||||
catch {
|
||||
Write-Error "unexpected: $($_.Exception.Message)"
|
||||
}
|
||||
finally {
|
||||
"always runs"
|
||||
}
|
||||
|
||||
# make non-terminating errors terminate, so catch can see them
|
||||
Get-Item missing -ErrorAction Stop
|
||||
|
||||
# per-session default
|
||||
$ErrorActionPreference = 'Stop'
|
||||
|
||||
# the most recent error, and the exit code of the last native command
|
||||
$Error[0]
|
||||
$LASTEXITCODE
|
||||
$?
|
||||
```
|
||||
|
||||
## 🌐 Web & Data
|
||||
|
||||
```powershell
|
||||
# REST calls, parsed into objects
|
||||
$data = Invoke-RestMethod https://api.github.com/repos/vuejs/core
|
||||
$data.stargazers_count
|
||||
|
||||
# with headers and a JSON body
|
||||
Invoke-RestMethod -Uri $url -Method Post `
|
||||
-Headers @{ Authorization = "Bearer $token" } `
|
||||
-ContentType 'application/json' `
|
||||
-Body (@{ name = 'demo' } | ConvertTo-Json)
|
||||
|
||||
# the raw response, for scraping or downloads
|
||||
Invoke-WebRequest $url -OutFile page.html
|
||||
|
||||
# convert between formats
|
||||
Get-Process | Select-Object Name, Id | ConvertTo-Json
|
||||
Get-Content data.json | ConvertFrom-Json
|
||||
Import-Csv users.csv | Where-Object Dept -eq 'IT' | Export-Csv it.csv -NoTypeInformation
|
||||
Export-Clixml / Import-Clixml # round-trips real objects
|
||||
```
|
||||
|
||||
## 🖥 System Administration
|
||||
|
||||
```powershell
|
||||
# processes
|
||||
Get-Process | Sort-Object CPU -Descending | Select-Object -First 10
|
||||
Stop-Process -Name notepad -Force
|
||||
|
||||
# services
|
||||
Get-Service | Where-Object Status -eq 'Running'
|
||||
Restart-Service -Name Spooler
|
||||
Set-Service -Name Spooler -StartupType Manual
|
||||
|
||||
# background jobs
|
||||
$job = Start-Job { Start-Sleep 10; "done" }
|
||||
Receive-Job $job -Wait
|
||||
|
||||
# scheduled tasks, event logs, system info
|
||||
Get-ScheduledTask
|
||||
Get-WinEvent -LogName System -MaxEvents 20
|
||||
Get-CimInstance Win32_LogicalDisk | Select-Object DeviceID, FreeSpace
|
||||
Test-Connection example.com -Count 2
|
||||
```
|
||||
|
||||
## 🔗 Remoting
|
||||
|
||||
```powershell
|
||||
# one command on many machines
|
||||
Invoke-Command -ComputerName web01, web02 -ScriptBlock { Get-Service Spooler }
|
||||
|
||||
# an interactive session
|
||||
Enter-PSSession -ComputerName web01
|
||||
Exit-PSSession
|
||||
|
||||
# a reusable session
|
||||
$s = New-PSSession -ComputerName web01 -Credential (Get-Credential)
|
||||
Invoke-Command -Session $s { hostname }
|
||||
Remove-PSSession $s
|
||||
```
|
||||
|
||||
## 📦 Modules & Execution Policy
|
||||
|
||||
```powershell
|
||||
# find, install, load
|
||||
Find-Module Pester
|
||||
Install-Module Pester -Scope CurrentUser
|
||||
Import-Module Pester
|
||||
Get-Module -ListAvailable
|
||||
|
||||
# where does PowerShell look for modules?
|
||||
$env:PSModulePath -split [IO.Path]::PathSeparator
|
||||
|
||||
# scripts refuse to run? check and relax the policy
|
||||
Get-ExecutionPolicy -List
|
||||
Set-ExecutionPolicy RemoteSigned -Scope CurrentUser
|
||||
|
||||
# your profile script
|
||||
notepad $PROFILE
|
||||
```
|
||||
|
||||
## 🧾 Automatic Variables
|
||||
|
||||
| Variable | Holds |
|
||||
|--------------------------|-------------------------------------------|
|
||||
| `$_` / `$PSItem` | The current pipeline object |
|
||||
| `$Args` | Unbound arguments to a script or function |
|
||||
| `$Error` | The error history, newest first |
|
||||
| `$Matches` | Capture groups from the last `-match` |
|
||||
| `$PSVersionTable` | Version and edition information |
|
||||
| `$PWD` / `$HOME` | Current directory / home directory |
|
||||
| `$PSScriptRoot` | The folder the running script lives in |
|
||||
| `$LASTEXITCODE` | Exit code of the last native command |
|
||||
| `$?` | Did the last command succeed? |
|
||||
| `$true` `$false` `$null` | Constants |
|
||||
| `$PROFILE` | Path to your profile script |
|
||||
|
||||
## 💾 PSDrives
|
||||
|
||||
| Drive | Contents |
|
||||
|-------------------|-----------------------|
|
||||
| `Env:` | Environment variables |
|
||||
| `Alias:` | Command aliases |
|
||||
| `Function:` | Defined functions |
|
||||
| `Variable:` | Variables |
|
||||
| `Cert:` | Certificate stores |
|
||||
| `HKLM:` / `HKCU:` | The registry |
|
||||
| `WSMan:` | WinRM configuration |
|
||||
|
||||
```powershell
|
||||
Set-Location HKLM:\SOFTWARE
|
||||
Get-ChildItem Env: | Sort-Object Name
|
||||
Get-ChildItem Variable:
|
||||
```
|
||||
|
||||
## ⌨️ Aliases Worth Knowing
|
||||
|
||||
| Alias | Real command | Alias | Real command |
|
||||
|--------------------|-----------------|-------------|--------------------------------|
|
||||
| `ls`, `dir`, `gci` | `Get-ChildItem` | `%` | `ForEach-Object` |
|
||||
| `cat`, `gc` | `Get-Content` | `?` | `Where-Object` |
|
||||
| `cd`, `sl` | `Set-Location` | `select` | `Select-Object` |
|
||||
| `cp`, `copy` | `Copy-Item` | `sort` | `Sort-Object` |
|
||||
| `rm`, `del` | `Remove-Item` | `ft` / `fl` | `Format-Table` / `Format-List` |
|
||||
| `ps`, `gps` | `Get-Process` | `sls` | `Select-String` |
|
||||
| `kill` | `Stop-Process` | `gm` | `Get-Member` |
|
||||
|
||||
> ⚠️ Aliases are for the console, not for scripts — write the full cmdlet name in anything you commit.
|
||||
|
||||
## 🔣 Regular Expressions
|
||||
|
||||
| Pattern | Matches |
|
||||
|----------------------|-----------------------------|
|
||||
| `\w` / `\W` | Word character / non-word |
|
||||
| `\s` / `\S` | Whitespace / non-whitespace |
|
||||
| `\d` / `\D` | Digit / non-digit |
|
||||
| `{n}` `{n,}` `{n,m}` | Quantifiers |
|
||||
| `^` / `$` | Start / end of the string |
|
||||
| `(?<name>…)` | Named capture group |
|
||||
|
||||
```powershell
|
||||
# -match fills $Matches
|
||||
if ("build-1234" -match '(?<name>\w+)-(?<id>\d+)') { $Matches.id }
|
||||
|
||||
# extract every match from a file
|
||||
Select-String -Path app.log -Pattern '\b\d{1,3}(\.\d{1,3}){3}\b' -AllMatches |
|
||||
ForEach-Object { $_.Matches.Value } | Sort-Object -Unique
|
||||
```
|
||||
|
||||
PowerShell uses the .NET regex engine — see [.NET regular expressions](https://learn.microsoft.com/dotnet/standard/base-types/regular-expressions).
|
||||
|
||||
## 📚 Resources
|
||||
|
||||
- [PowerShell documentation](https://learn.microsoft.com/powershell/)
|
||||
- [Cmdlet reference](https://learn.microsoft.com/powershell/module/)
|
||||
- [PowerShell Gallery](https://www.powershellgallery.com/)
|
||||
- [Style guide (community)](https://poshcode.gitbook.io/powershell-practice-and-style/)
|
||||
@@ -1,121 +1,174 @@
|
||||
### Normal characters
|
||||
**Regular expressions** describe a pattern that text either matches or does not. The syntax below is the JavaScript flavour; PCRE, Python, Go and Java agree on almost all of it — the differences are called out where they bite.
|
||||
|
||||
Expression | Description
|
||||
:--|:--
|
||||
`.` or `[^\n\r]` | any character *excluding* a newline or carriage return
|
||||
`[A-Za-z]` | alphabet
|
||||
`[a-z]` | lowercase alphabet
|
||||
`[A-Z]` | uppercase alphabet
|
||||
`\d` or `[0-9]` | digit
|
||||
`\D` or `[^0-9]` | non-digit
|
||||
`_` | underscore
|
||||
`\w` or `[A-Za-z0-9_]` | alphabet, digit or underscore
|
||||
`\W` or `[^A-Za-z0-9_]` | inverse of `\w`
|
||||
`\S` | inverse of `\s`
|
||||
```js
|
||||
/^\d{4}-\d{2}-\d{2}$/.test('2026-08-22') // true
|
||||
'a1b2'.replace(/\d/g, '#') // 'a#b#'
|
||||
```
|
||||
|
||||
### Whitespace characters
|
||||
## 🔡 Character Classes
|
||||
|
||||
Expression | Description
|
||||
:--|:--
|
||||
` ` | space
|
||||
`\t` | tab
|
||||
`\n` | newline
|
||||
`\r` | carriage return
|
||||
`\s` | space, tab, newline or carriage return
|
||||
| Expression | Matches |
|
||||
|------------------------|---------------------------------------------------------------|
|
||||
| `.` | Any character except a newline (any at all with the `s` flag) |
|
||||
| `[A-Za-z]` | A letter |
|
||||
| `[a-z]` / `[A-Z]` | A lower-case / upper-case letter |
|
||||
| `\d` or `[0-9]` | A digit |
|
||||
| `\D` or `[^0-9]` | Anything but a digit |
|
||||
| `\w` or `[A-Za-z0-9_]` | A letter, digit or underscore |
|
||||
| `\W` | The inverse of `\w` |
|
||||
| `\s` | Whitespace: space, tab, newline, carriage return |
|
||||
| `\S` | The inverse of `\s` |
|
||||
| `\p{L}` / `\p{Nd}` | Any Unicode letter / decimal digit (needs the `u` flag) |
|
||||
| `\x41` / `\u00E9` | A character by hex code, or by Unicode code point |
|
||||
|
||||
### Character set
|
||||
## ␣ Whitespace
|
||||
|
||||
Expression | Description
|
||||
:--|:--
|
||||
`[xyz]` | either `x`, `y` or `z`
|
||||
`[^xyz]` | neither `x`, `y` nor `z`
|
||||
`[1-3]` | either `1`, `2` or `3`
|
||||
`[^1-3]` | neither `1`, `2` nor `3`
|
||||
| Expression | Matches |
|
||||
|------------|-------------------------|
|
||||
| ` ` | A literal space |
|
||||
| `\t` | Tab |
|
||||
| `\n` | Newline |
|
||||
| `\r` | Carriage return |
|
||||
| `\f` `\v` | Form feed, vertical tab |
|
||||
| `\s` | Any of the above |
|
||||
|
||||
- Think of a character set as an `OR` operation on the single characters that are enclosed between the square brackets.
|
||||
- Use `^` after the opening `[` to “negate” the character set.
|
||||
- Within a character set, `.` means a literal period.
|
||||
## 🎯 Character Sets
|
||||
|
||||
### Characters that require escaping
|
||||
| Expression | Matches |
|
||||
|-------------|-------------------------------------------|
|
||||
| `[xyz]` | `x`, `y` or `z` |
|
||||
| `[^xyz]` | Anything except `x`, `y`, `z` |
|
||||
| `[1-3]` | `1`, `2` or `3` |
|
||||
| `[^1-3]` | Anything except `1`, `2`, `3` |
|
||||
| `[a-z0-9_]` | Ranges and single characters can be mixed |
|
||||
|
||||
#### Outside a character set
|
||||
- A set is an **or** over single characters — `[abc]` matches one character, not the string `abc`.
|
||||
- `^` right after the opening `[` negates the set.
|
||||
- Inside a set, most metacharacters lose their meaning: `.` is a literal period, `+` a literal plus.
|
||||
|
||||
Expression | Description
|
||||
:--|:--
|
||||
`\.` | period
|
||||
`\^` | caret
|
||||
`\$` | dollar sign
|
||||
`\|` | pipe
|
||||
`\\` | back slash
|
||||
`\/` | forward slash
|
||||
`\(` | opening bracket
|
||||
`\)` | closing bracket
|
||||
`\[` | opening square bracket
|
||||
`\]` | closing square bracket
|
||||
`\{` | opening curly bracket
|
||||
`\}` | closing curly bracket
|
||||
## 🔒 Escaping
|
||||
|
||||
#### Inside a character set
|
||||
Outside a character set, these need a backslash to be taken literally:
|
||||
|
||||
Expression | Description
|
||||
:--|:--
|
||||
`\\` | back slash
|
||||
`\]` | closing square bracket
|
||||
| Escape | Character | Escape | Character |
|
||||
|--------|----------------------------------------|--------|---------------------|
|
||||
| `\.` | Period | `\(` | Opening parenthesis |
|
||||
| `\^` | Caret | `\)` | Closing parenthesis |
|
||||
| `\$` | Dollar sign | `\[` | Opening bracket |
|
||||
| `\\|` | Pipe | `\]` | Closing bracket |
|
||||
| `\\` | Backslash | `\{` | Opening brace |
|
||||
| `\/` | Forward slash (only in `/…/` literals) | `\}` | Closing brace |
|
||||
|
||||
- A `^` must be escaped only if it occurs immediately after the opening `[` of the character set.
|
||||
- A `-` must be escaped only if it occurs between two alphabets or two digits.
|
||||
Inside a character set only `\\` and `\]` always need escaping. A `^` needs it only immediately after `[`, and a `-` only between two other characters.
|
||||
|
||||
### Quantifiers
|
||||
## 🔢 Quantifiers
|
||||
|
||||
Expression | Description
|
||||
:--|:--
|
||||
`{2}` | exactly 2
|
||||
`{2,}` | at least 2
|
||||
`{2,7}` | at least 2 but no more than 7
|
||||
`*` | 0 or more
|
||||
`+` | 1 or more
|
||||
`?` | exactly 0 or 1
|
||||
| Expression | Repeats the preceding item |
|
||||
|-------------------------|-------------------------------------------------------|
|
||||
| `{2}` | Exactly twice |
|
||||
| `{2,}` | Two or more times |
|
||||
| `{2,7}` | Between two and seven times |
|
||||
| `*` | Zero or more |
|
||||
| `+` | One or more |
|
||||
| `?` | Zero or once |
|
||||
| `*?` `+?` `??` `{2,7}?` | The lazy form — match as *few* characters as possible |
|
||||
|
||||
- The quantifier goes *after* the expression to be quantified.
|
||||
```js
|
||||
'<b>bold</b>'.match(/<.+>/)[0] // '<b>bold</b>' — greedy
|
||||
'<b>bold</b>'.match(/<.+?>/)[0] // '<b>' — lazy
|
||||
```
|
||||
|
||||
### Boundaries
|
||||
## 📍 Anchors & Boundaries
|
||||
|
||||
Expression | Description
|
||||
:--|:--
|
||||
`^` | start of string
|
||||
`$` | end of string
|
||||
`\b` | word boundary
|
||||
| Expression | Matches at |
|
||||
|------------|-------------------------------------------------------------------------|
|
||||
| `^` | Start of the string (start of a line with the `m` flag) |
|
||||
| `$` | End of the string (end of a line with the `m` flag) |
|
||||
| `\b` | A word boundary |
|
||||
| `\B` | Anywhere that is *not* a word boundary |
|
||||
| `\A` `\z` | Start / end of the string in PCRE, Python, Java — **not** in JavaScript |
|
||||
|
||||
- How word boundary matching works:
|
||||
- At the beginning of the string if the first character is `\w`.
|
||||
- Between two adjacent characters within the string, if the first character is `\w` and the second character is `\W`.
|
||||
- At the end of the string if the last character is `\w`.
|
||||
A word boundary sits between a `\w` and a `\W` (or the edge of the string), so `\bcat\b` matches `cat` in `the cat sat` but not in `concatenate`.
|
||||
|
||||
### Matching
|
||||
## 🔀 Alternation & Lookaround
|
||||
|
||||
Expression | Description
|
||||
:--|:--
|
||||
`foo\|bar` | match either `foo` or `bar`
|
||||
`foo(?=bar)` | match `foo` if it’s before `bar`
|
||||
`foo(?!bar)` | match `foo` if it’s *not* before `bar`
|
||||
`(?<=bar)foo` | match `foo` if it’s after `bar`
|
||||
`(?<!bar)foo` | match `foo` if it’s *not* after `bar`
|
||||
| Expression | Matches |
|
||||
|---------------|-------------------------------------------|
|
||||
| `foo\|bar` | Either `foo` or `bar` |
|
||||
| `foo(?=bar)` | `foo` only when followed by `bar` |
|
||||
| `foo(?!bar)` | `foo` only when **not** followed by `bar` |
|
||||
| `(?<=bar)foo` | `foo` only when preceded by `bar` |
|
||||
| `(?<!bar)foo` | `foo` only when **not** preceded by `bar` |
|
||||
|
||||
### Grouping and capturing
|
||||
Lookaround is *zero-width*: it tests the surroundings without consuming them, so nothing it matches ends up in the result.
|
||||
|
||||
Expression | Description
|
||||
:--|:--
|
||||
`(foo)` | capturing group; match and capture `foo`
|
||||
`(?:foo)` | non-capturing group; match `foo` but *without* capturing `foo`
|
||||
`(foo)bar\1` | `\1` is a backreference to the 1st capturing group; match `foobarfoo`
|
||||
## 🎁 Groups & Backreferences
|
||||
|
||||
- Capturing groups are only relevant in the following methods:
|
||||
- `string.match(regexp)`
|
||||
- `string.matchAll(regexp)`
|
||||
- `string.replace(regexp, callback)`
|
||||
- `\N` is a backreference to the `Nth` capturing group. Capturing groups are numbered starting from 1.
|
||||
| Expression | Meaning |
|
||||
|------------------|--------------------------------------------------|
|
||||
| `(foo)` | Capturing group — matches `foo` and remembers it |
|
||||
| `(?:foo)` | Non-capturing group — groups without capturing |
|
||||
| `(?<year>\d{4})` | Named capturing group |
|
||||
| `\1` | Backreference to the first group |
|
||||
| `\k<year>` | Backreference to a named group |
|
||||
|
||||
## References and tools
|
||||
```js
|
||||
// named groups make replacements readable
|
||||
'2026-08-22'.replace(/(?<y>\d{4})-(?<m>\d{2})-(?<d>\d{2})/, '$<d>/$<m>/$<y>')
|
||||
// '22/08/2026'
|
||||
|
||||
- [MDN](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Guide/Regular_Expressions)
|
||||
const { groups } = /(?<user>\w+)@(?<host>[\w.]+)/.exec('jane@example.com');
|
||||
// groups.user === 'jane', groups.host === 'example.com'
|
||||
```
|
||||
|
||||
## 🚩 Flags
|
||||
|
||||
| Flag | Name | Effect |
|
||||
|------|-------------|-----------------------------------------------------------|
|
||||
| `g` | global | Find every match, not just the first |
|
||||
| `i` | ignore case | Case-insensitive matching |
|
||||
| `m` | multiline | `^` and `$` match at line breaks |
|
||||
| `s` | dotAll | `.` also matches newlines |
|
||||
| `u` | unicode | Treat the pattern as Unicode code points; enables `\p{…}` |
|
||||
| `y` | sticky | Match only from `lastIndex`, no scanning ahead |
|
||||
| `d` | indices | Report the start/end index of every group |
|
||||
|
||||
## 🔁 Replacement Patterns
|
||||
|
||||
| Token | Inserts |
|
||||
|------------|-----------------------------|
|
||||
| `$&` | The whole match |
|
||||
| `` $` `` | Everything before the match |
|
||||
| `$'` | Everything after the match |
|
||||
| `$1`, `$2` | Capture group 1, 2, … |
|
||||
| `$<name>` | A named capture group |
|
||||
| `$$` | A literal dollar sign |
|
||||
|
||||
> ℹ️ Other tools spell these differently: `\1` in `sed`, `\g<1>` in Python, `${1}` in Go.
|
||||
|
||||
## 🧪 Where the Groups Show Up (JavaScript)
|
||||
|
||||
```js
|
||||
'2026-08-22'.match(/\d+/g) // ['2026', '08', '22'] — no groups with /g
|
||||
[...'a1b2'.matchAll(/([a-z])(\d)/g)] // every match, with its groups
|
||||
/(\d+)/.exec('abc 42') // ['42', '42', index: 4, ...]
|
||||
/\d/.test('abc') // false
|
||||
'a1'.replace(/(\w)(\d)/, (m, a, b) => b + a) // '1a'
|
||||
'a, b;c'.split(/[,;]\s*/) // ['a', 'b', 'c']
|
||||
```
|
||||
|
||||
## ⚠️ Gotchas
|
||||
|
||||
- **`.` does not match newlines** unless you set the `s` flag — a common surprise on multi-line input.
|
||||
- **A `/g` regex is stateful.** Reusing the same object across `test()` calls advances `lastIndex` and skips matches; build a fresh one, or reset `lastIndex = 0`.
|
||||
- **Nested quantifiers backtrack catastrophically.** `(a+)+$` against a long non-matching string can hang the process — avoid overlapping repetition, prefer possessive/atomic groups where the flavour has them.
|
||||
- **Escape user input** before putting it in a pattern; in JS, `RegExp.escape()` where available, otherwise `s.replace(/[.*+?^${}()|[\]\\]/g, '\\$&')`.
|
||||
- **Don't parse HTML with regex.** Use a parser; regex cannot handle nesting.
|
||||
- **Backslashes double up in strings**: `new RegExp('\\d+')` is the same pattern as `/\d+/`.
|
||||
|
||||
## 📚 Resources
|
||||
|
||||
- [MDN — regular expressions](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Guide/Regular_expressions)
|
||||
- [regex101 — build and explain patterns](https://regex101.com)
|
||||
- [RegExplained](https://leaverou.github.io/regexplained/)
|
||||
- [Regular-Expressions.info](https://www.regular-expressions.info/)
|
||||
@@ -1,15 +1,13 @@
|
||||
This sheet gives you a complete, visual, color‑accurate reference for Cat6 wiring.
|
||||
A complete, colour-accurate reference for terminating **RJ45 / Cat6** Ethernet cables: the two pinout standards, what each pair does, PoE, and the rules that decide whether the run passes a certifier.
|
||||
|
||||
## 1. T568A & T568B Pinouts
|
||||
## 🎨 T568A & T568B Pinouts
|
||||
|
||||
These are the two official wiring standards for Ethernet cables.
|
||||
Both work electrically the same — the only difference is the **color order**.
|
||||
Most modern installations use **T568B**, but government and residential specs sometimes require **T568A**.
|
||||
Both standards are electrically identical — only the colour order differs. **T568B** is the default in most commercial installations; **T568A** turns up in US government and residential specifications. Pick one and use it on both ends of every cable in the building.
|
||||
|
||||
### T568A Pinout
|
||||
### T568A
|
||||
|
||||
| Pin | Color block | Name |
|
||||
|-----|-------------|------|
|
||||
| Pin | Colour | Name |
|
||||
|-----|--------|------|
|
||||
| 1 | <span style="display:inline-block;width:14px;height:14px;background:linear-gradient(90deg,#ffffff 50%,#008000 50%);border:1px solid #000;"></span> | White/Green |
|
||||
| 2 | <span style="display:inline-block;width:14px;height:14px;background:#008000;border:1px solid #000;"></span> | Green |
|
||||
| 3 | <span style="display:inline-block;width:14px;height:14px;background:linear-gradient(90deg,#ffffff 50%,#ff8c00 50%);border:1px solid #000;"></span> | White/Orange |
|
||||
@@ -19,14 +17,12 @@ Most modern installations use **T568B**, but government and residential specs so
|
||||
| 7 | <span style="display:inline-block;width:14px;height:14px;background:linear-gradient(90deg,#ffffff 50%,#8b4513 50%);border:1px solid #000;"></span> | White/Brown |
|
||||
| 8 | <span style="display:inline-block;width:14px;height:14px;background:#8b4513;border:1px solid #000;"></span> | Brown |
|
||||
|
||||
### T568B Pinout (Most Common)
|
||||
|
||||
This is the standard used in most commercial networks, patch panels, and pre‑made cables.
|
||||
### T568B (most common)
|
||||
|
||||
;border:1px solid #000;"></span> | White/Orange |
|
||||
| 2 | <span style="display:inline-block;width:14px;height:14px;background:#ff8c00;border:1px solid #000;"></span> | Orange |
|
||||
| 3 | <span style="display:inline-block;width:14px;height:14px;background:linear-gradient(90deg,#ffffff 50%,#008000 50%);border:1px solid #000;"></span> | White/Green |
|
||||
@@ -36,14 +32,22 @@ This is the standard used in most commercial networks, patch panels, and pre‑m
|
||||
| 7 | <span style="display:inline-block;width:14px;height:14px;background:linear-gradient(90deg,#ffffff 50%,#8b4513 50%);border:1px solid #000;"></span> | White/Brown |
|
||||
| 8 | <span style="display:inline-block;width:14px;height:14px;background:#8b4513;border:1px solid #000;"></span> | Brown |
|
||||
|
||||
## 2. Straight‑Through Cable (T568B ↔ T568B)
|
||||
> 💡 The mnemonic for T568B: **orange, green, blue, brown** — striped before solid, except that the blue pair sits inside the green pair on pins 4 and 5.
|
||||
|
||||
This is the **standard Ethernet patch cable** used to connect:
|
||||
- computers → switches
|
||||
- switches → routers
|
||||
- wall jacks → patch panels
|
||||
## 🔌 What Each Pin Does
|
||||
|
||||
Both ends use the **same** wiring standard.
|
||||
| Pins | Pair | 10/100BASE-TX | 1000BASE-T and faster |
|
||||
|-------|------|----------------------|-----------------------|
|
||||
| 1 – 2 | 2 | Transmit (TX+ / TX−) | Bi-directional pair A |
|
||||
| 3 – 6 | 3 | Receive (RX+ / RX−) | Bi-directional pair B |
|
||||
| 4 – 5 | 1 | Unused | Bi-directional pair C |
|
||||
| 7 – 8 | 4 | Unused | Bi-directional pair D |
|
||||
|
||||
Gigabit and 10 Gigabit Ethernet use **all four pairs in both directions** at once — which is why a "Fast Ethernet works, Gigabit doesn't" fault is almost always a bad termination on pins 4, 5, 7 or 8.
|
||||
|
||||
## 🔗 Straight-Through Cable (T568B ↔ T568B)
|
||||
|
||||
The standard patch cable: computer → switch, switch → router, wall jack → patch panel. Both ends use the same standard, so every pin lands on the same pin.
|
||||
|
||||
```text
|
||||
End A (T568B) End B (T568B)
|
||||
@@ -58,17 +62,9 @@ End A (T568B) End B (T568B)
|
||||
8 Brown ------------------ Brown 8
|
||||
```
|
||||
|
||||
A straight‑through cable keeps all pins aligned. Modern devices use auto‑MDI/MDI‑X, so this is the only cable type you typically need today.
|
||||
## 🔀 Crossover Cable (T568A ↔ T568B)
|
||||
|
||||
## 3. Crossover Cable (T568A ↔ T568B)
|
||||
|
||||
This cable swaps the transmit and receive pairs.
|
||||
It was historically used for:
|
||||
- switch ↔ switch
|
||||
- computer ↔ computer
|
||||
- router ↔ router
|
||||
|
||||
Today, almost all hardware auto‑detects, so crossover cables are mostly legacy.
|
||||
Historically needed for switch ↔ switch, PC ↔ PC and router ↔ router links. Every device made in the last twenty years negotiates **Auto-MDI/MDI-X**, so this is now a legacy cable — but it is still what you grab for two ancient devices, or a lab exercise.
|
||||
|
||||
```text
|
||||
End A (T568A) End B (T568B)
|
||||
@@ -78,46 +74,71 @@ End A (T568A) End B (T568B)
|
||||
3 White/Orange ------------------ White/Green 3
|
||||
6 Orange ------------------ Green 6
|
||||
|
||||
Pins 4, 5, 7, 8 remain straight‑through.
|
||||
Pins 4, 5, 7 and 8 stay straight-through for 10/100.
|
||||
```
|
||||
|
||||
Only the pairs used for data (1–2 and 3–6) are swapped.
|
||||
Gigabit Ethernet uses all four pairs, but still auto‑corrects internally.
|
||||
A true **gigabit crossover** also swaps the other two pairs: 4↔7 and 5↔8.
|
||||
|
||||
## 4. Pair Mapping (Color‑Block Reference)
|
||||
## 🧵 Pair Mapping
|
||||
|
||||
Ethernet uses **four twisted pairs**, each with a solid and striped conductor.
|
||||
This table helps you quickly identify which colors belong together.
|
||||
Four twisted pairs, each a solid conductor and its white-striped partner. Keeping a pair together is what cancels noise.
|
||||
|
||||
| Pair | T568A colors | T568B colors | Explanation |
|
||||
|------|--------------|--------------|-------------|
|
||||
| Pair 1 (4–5) | 🔵 / 🔵⬜ | same | Used for PoE power delivery |
|
||||
| Pair 2 (1–2) | 🟧⬜ / 🟧 | 🟧⬜ / 🟧 | Primary transmit/receive pair |
|
||||
| Pair 3 (3–6) | 🟩⬜ / 🟩 | 🟩⬜ / 🟩 | Secondary transmit/receive pair |
|
||||
| Pair 4 (7–8) | 🟫⬜ / 🟫 | same | Not used in 10/100 Mbps, used in Gigabit |
|
||||
| Pair | Pins | T568A colours | T568B colours | Notes |
|
||||
|------|-------|---------------|---------------|-----------------------------------------|
|
||||
| 1 | 4 – 5 | 🔵⬜ / 🔵 | same | Sits inside the middle pair |
|
||||
| 2 | 1 – 2 | 🟩⬜ / 🟩 | 🟧⬜ / 🟧 | The pair that swaps between the standards |
|
||||
| 3 | 3 – 6 | 🟧⬜ / 🟧 | 🟩⬜ / 🟩 | Split across pins 3 and 6 |
|
||||
| 4 | 7 – 8 | 🟫⬜ / 🟫 | same | Idle on 10/100, active on Gigabit |
|
||||
|
||||
Pair grouping matters for signal integrity — never untwist more than ~13 mm when terminating.
|
||||
> ⚠️ Never untwist more than about **13 mm (½ inch)** at the connector, and never split a pair across two pairs' positions — a "works but slow" cable is usually a split pair, and only a certifier will tell you.
|
||||
|
||||
## 5. Cat5e / Cat6 / Cat6A Comparison
|
||||
## ⚡ Power over Ethernet
|
||||
|
||||
This helps you choose the right cable for your installation.
|
||||
| Standard | Name | Power at the port | Pairs used |
|
||||
|-----------------|---------|-------------------|-----------------------|
|
||||
| IEEE 802.3af | PoE | 15.4 W | Two pairs (Mode A or B) |
|
||||
| IEEE 802.3at | PoE+ | 30 W | Two pairs |
|
||||
| IEEE 802.3bt T3 | PoE++ | 60 W | All four pairs |
|
||||
| IEEE 802.3bt T4 | PoE++ | 100 W | All four pairs |
|
||||
|
||||
| Feature | Cat5e | Cat6 | Cat6A | Explanation |
|
||||
|--------|-------|------|-------|-------------|
|
||||
| Max speed | 1 Gbps | 10 Gbps (≤55 m) | 10 Gbps (100 m) | Cat6A is the only one supporting full‑distance 10G |
|
||||
| Bandwidth | 100 MHz | 250 MHz | 500 MHz | Higher bandwidth = better noise resistance |
|
||||
| Shielding | UTP/STP | UTP/STP | Often STP | Cat6A often shielded for long‑distance 10G |
|
||||
| PoE | Yes | Yes | Yes | All support PoE/PoE+/PoE++ |
|
||||
| Typical use | Home | Office | Data center | Cat6A is bulkier but future‑proof |
|
||||
- **Mode A** puts power on the data pairs 1–2 and 3–6; **Mode B** uses the spare pairs 4–5 and 7–8. A powered device must accept either.
|
||||
- Because 802.3bt energises all four pairs, sloppy terminations on 4/5/7/8 now break power as well as gigabit speed.
|
||||
- Power raises cable temperature; in a full, bundled tray, derate the length or move to a shielded, larger-gauge cable.
|
||||
|
||||
## 6. Termination Best Practices
|
||||
## 📏 Categories & Limits
|
||||
|
||||
These rules ensure your cable meets Cat6 performance:
|
||||
| Feature | Cat5e | Cat6 | Cat6A | Cat8 |
|
||||
|-------------|-----------|-------------------|------------------|-----------------|
|
||||
| Max speed | 1 Gbps | 10 Gbps (≤ 55 m) | 10 Gbps (100 m) | 25/40 Gbps (30 m) |
|
||||
| Bandwidth | 100 MHz | 250 MHz | 500 MHz | 2000 MHz |
|
||||
| Shielding | UTP/STP | UTP/STP | Often F/UTP | Always shielded |
|
||||
| Typical use | Home, PoE cameras | Office floor | 10G to the desk, dense PoE | Data-centre top-of-rack |
|
||||
|
||||
- Keep twist pairs intact as long as possible
|
||||
- Use Cat6‑rated connectors (larger conductor spacing)
|
||||
- Ensure the jacket is inside the strain‑relief clamp
|
||||
- Test with a cable tester
|
||||
- Use T568B unless your environment requires T568A
|
||||
- Maximum channel length is **100 m**: 90 m of solid-core in the wall plus 10 m of stranded patch leads.
|
||||
- Use **solid** conductor for permanent runs and **stranded** for patch cords — stranded flexes, solid does not survive repeated bending; connectors are rated for one or the other.
|
||||
|
||||
Cat6 performance depends heavily on maintaining pair geometry and minimizing untwist.
|
||||
## 🔧 Termination Checklist
|
||||
|
||||
- Strip only the jacket, never nick the insulation of the conductors.
|
||||
- Keep the pairs twisted right up to the point of termination.
|
||||
- The jacket must sit **inside** the connector's strain-relief crimp.
|
||||
- Use Cat6-rated plugs (wider conductor spacing) with a load bar for 23 AWG cable.
|
||||
- For keystone jacks, follow the colour code printed on the jack and punch down with a 110 tool — do not push the wires in with a screwdriver.
|
||||
- Terminate to **T568B** unless the site specification says otherwise, and label both ends.
|
||||
- Test every run: a cheap continuity tester catches swaps and opens; only a certifier catches split pairs, NEXT and length problems.
|
||||
|
||||
## 🧯 Common Faults
|
||||
|
||||
| Symptom | Likely cause |
|
||||
|-------------------------------------|-------------------------------------------------------|
|
||||
| No link at all | Open or crossed pair on 1–2 / 3–6, or a bad crimp |
|
||||
| 100 Mbps instead of 1 Gbps | One of pins 4, 5, 7, 8 not terminated properly |
|
||||
| Link drops under load | Split pair, excessive untwist, or a run over 100 m |
|
||||
| PoE device does not power up | Break on the pairs the injector uses, or a budget limit on the switch |
|
||||
| Errors only on hot afternoons | Bundled PoE runs overheating, or cable near a power line |
|
||||
|
||||
## 📚 Resources
|
||||
|
||||
- [TIA-568 wiring standards overview](https://en.wikipedia.org/wiki/TIA-568)
|
||||
- [Power over Ethernet standards](https://en.wikipedia.org/wiki/Power_over_Ethernet)
|
||||
- [Ethernet physical layers and distances](https://en.wikipedia.org/wiki/Ethernet_physical_layer)
|
||||
@@ -1,129 +1,139 @@
|
||||
# 🖥️ GNU Screen Command Cheatsheet
|
||||
**GNU Screen** is a terminal multiplexer: it runs several shell sessions inside one terminal, keeps them alive after you disconnect, and lets you reattach later from anywhere.
|
||||
|
||||
GNU Screen is a terminal multiplexer that lets you run multiple shell sessions inside a single terminal window, detach them, and reattach later.
|
||||
> ℹ️ Every shortcut starts with the **prefix**, `Ctrl+a` by default: press the prefix, release it, then press the key. In the tables below `Ctrl+a c` means "prefix, then `c`".
|
||||
|
||||
## 📦 Starting & Managing Screen Sessions
|
||||
## 🚀 Sessions from the Shell
|
||||
|
||||
- **Start a new session**
|
||||
```bash
|
||||
screen
|
||||
```
|
||||
- **Start with a session name**
|
||||
```bash
|
||||
screen -S session_name
|
||||
```
|
||||
- **List running sessions**
|
||||
```bash
|
||||
screen -ls
|
||||
```
|
||||
- **Attach to a session**
|
||||
```bash
|
||||
screen -r session_name
|
||||
```
|
||||
- **Attach to the last detached session**
|
||||
```bash
|
||||
screen -r
|
||||
```
|
||||
- **Kill a session**
|
||||
```bash
|
||||
screen -X -S session_name quit
|
||||
```
|
||||
```bash
|
||||
# start a session
|
||||
screen
|
||||
|
||||
## ⌨️ Essential Key Bindings
|
||||
# start a named session
|
||||
screen -S <name>
|
||||
|
||||
> Default prefix key: `Ctrl-a` (press before other keys)
|
||||
# list running sessions
|
||||
screen -ls
|
||||
|
||||
| Key Binding | Action |
|
||||
|-|--|
|
||||
| `Ctrl-a c` | Create new window |
|
||||
| `Ctrl-a n` | Next window |
|
||||
| `Ctrl-a p` | Previous window |
|
||||
| `Ctrl-a "` | List all windows |
|
||||
| `Ctrl-a 0..9` | Switch to window by number |
|
||||
| `Ctrl-a A` | Rename current window |
|
||||
| `Ctrl-a d` | Detach session |
|
||||
| `Ctrl-a ?` | Help (show key bindings) |
|
||||
| `Ctrl-a k` | Kill current window |
|
||||
| `Ctrl-a \` | Kill all windows & exit screen |
|
||||
| `Ctrl-a x` | Lock screen |
|
||||
| `Ctrl-a S` | Split screen horizontally |
|
||||
| `Ctrl-a |` | Split screen vertically |
|
||||
| `Ctrl-a tab` | Switch focus between regions |
|
||||
| `Ctrl-a Q` | Close all regions except current |
|
||||
# reattach to a session
|
||||
screen -r <name>
|
||||
|
||||
## 🔀 Window & Region Management
|
||||
# reattach, detaching whoever else is attached
|
||||
screen -d -r <name>
|
||||
|
||||
- **Split horizontally**
|
||||
```bash
|
||||
Ctrl-a S
|
||||
```
|
||||
- **Split vertically**
|
||||
```bash
|
||||
Ctrl-a |
|
||||
```
|
||||
- **Switch region focus**
|
||||
```bash
|
||||
Ctrl-a tab
|
||||
```
|
||||
- **Remove all splits except current**
|
||||
```bash
|
||||
Ctrl-a Q
|
||||
```
|
||||
# reattach if possible, otherwise create the session
|
||||
screen -R <name>
|
||||
|
||||
## 📂 Session Persistence
|
||||
# attach in parallel with another client (shared screen)
|
||||
screen -x <name>
|
||||
|
||||
- **Detach session (keep running in background)**
|
||||
```bash
|
||||
Ctrl-a d
|
||||
```
|
||||
- **Reattach session**
|
||||
```bash
|
||||
screen -r session_name
|
||||
```
|
||||
# start a detached session running a command
|
||||
screen -dmS <name> <command>
|
||||
|
||||
# run a command inside an existing session
|
||||
screen -S <name> -X stuff "make test\n"
|
||||
|
||||
# quit a session from outside
|
||||
screen -X -S <name> quit
|
||||
|
||||
# clean up dead sessions
|
||||
screen -wipe
|
||||
```
|
||||
|
||||
## ⌨️ Windows
|
||||
|
||||
| Keys | Action |
|
||||
|-----------------|-------------------------------------|
|
||||
| `Ctrl+a c` | Create a window |
|
||||
| `Ctrl+a A` | Rename the current window |
|
||||
| `Ctrl+a n` | Next window |
|
||||
| `Ctrl+a p` | Previous window |
|
||||
| `Ctrl+a Ctrl+a` | Toggle between the last two windows |
|
||||
| `Ctrl+a 0…9` | Jump to a window by number |
|
||||
| `Ctrl+a "` | Choose a window from a list |
|
||||
| `Ctrl+a w` | Show the window bar |
|
||||
| `Ctrl+a k` | Kill the current window |
|
||||
|
||||
## 🪟 Split Regions
|
||||
|
||||
| Keys | Action |
|
||||
|--------------|-------------------------------------------|
|
||||
| `Ctrl+a S` | Split into top / bottom regions |
|
||||
| `Ctrl+a \|` | Split into left / right regions |
|
||||
| `Ctrl+a Tab` | Move the focus to the next region |
|
||||
| `Ctrl+a X` | Close the focused region |
|
||||
| `Ctrl+a Q` | Close every region except the focused one |
|
||||
|
||||
> 💡 A new region starts empty — press `Ctrl+a c` to open a shell in it, or `Ctrl+a "` to move an existing window there.
|
||||
|
||||
## 🔌 Session Control
|
||||
|
||||
| Keys | Action |
|
||||
|--------------|------------------------------------------|
|
||||
| `Ctrl+a d` | Detach and leave everything running |
|
||||
| `Ctrl+a D D` | Detach and log out |
|
||||
| `Ctrl+a x` | Lock the terminal (password required) |
|
||||
| `Ctrl+a :` | Command prompt (`:quit`, `:sessionname`) |
|
||||
| `Ctrl+a ?` | List every key binding |
|
||||
| `Ctrl+a \` | Kill every window and quit screen |
|
||||
| `Ctrl+a a` | Send a literal `Ctrl+a` (nested screens) |
|
||||
|
||||
## 📋 Copy Mode & Scrollback
|
||||
|
||||
| Keys | Action |
|
||||
|-------------|-------------------------------------------------|
|
||||
| `Ctrl+a [` | Enter copy/scrollback mode |
|
||||
| `space` | Start the selection, then `space` again to copy |
|
||||
| `Ctrl+a ]` | Paste the buffer |
|
||||
| `/` and `?` | Search forward / backward |
|
||||
| `Ctrl+a >` | Write the buffer to a file |
|
||||
| `Esc` | Leave copy mode |
|
||||
|
||||
## 📝 Logging & Monitoring
|
||||
|
||||
| Keys | Action |
|
||||
|------------|-----------------------------------------------|
|
||||
| `Ctrl+a H` | Toggle logging of the window to `screenlog.n` |
|
||||
| `Ctrl+a M` | Notify me when this window shows activity |
|
||||
| `Ctrl+a _` | Notify me when this window goes quiet |
|
||||
| `Ctrl+a C` | Clear the window |
|
||||
|
||||
```bash
|
||||
# start a session with logging enabled
|
||||
screen -L -S <name>
|
||||
|
||||
# keep 5000 lines of scrollback
|
||||
screen -h 5000
|
||||
```
|
||||
|
||||
## ⚙️ Configuration
|
||||
|
||||
- **Default config file:** `~/.screenrc`
|
||||
- Example `.screenrc`:
|
||||
```bash
|
||||
# Start with a named session
|
||||
startup_message off
|
||||
defscrollback 5000
|
||||
hardstatus alwayslastline
|
||||
hardstatus string '%{= kG}[%H] %{= kw}%?%-Lw%?%{= kR}%n*%f %t%?(%u)%?%{= kw}%?%+Lw%? %{= kG}[%H]'
|
||||
```
|
||||
Settings live in `~/.screenrc`:
|
||||
|
||||
## 🛠️ Useful Options
|
||||
```bash
|
||||
# no splash screen, plenty of scrollback
|
||||
startup_message off
|
||||
defscrollback 10000
|
||||
|
||||
- **Scrollback buffer size**
|
||||
```bash
|
||||
screen -h 5000
|
||||
```
|
||||
- **Execute command in new screen**
|
||||
```bash
|
||||
screen -dmS session_name command
|
||||
```
|
||||
- **Log output**
|
||||
```bash
|
||||
Ctrl-a H
|
||||
```
|
||||
(toggle logging in current window)
|
||||
# a status line with the window list and the hostname
|
||||
hardstatus alwayslastline
|
||||
hardstatus string '%{= kG}[%H] %{= kw}%?%-Lw%?%{= kR}%n*%f %t%?(%u)%?%{= kw}%?%+Lw%?'
|
||||
|
||||
## 🚪 Exiting Screen
|
||||
# use Ctrl+z as the prefix instead of Ctrl+a
|
||||
# escape ^z^z
|
||||
|
||||
- **Kill current window**
|
||||
```bash
|
||||
Ctrl-a k
|
||||
```
|
||||
- **Kill all windows & quit**
|
||||
```bash
|
||||
Ctrl-a \
|
||||
```
|
||||
- **Exit normally**
|
||||
- Close all programs in the window, then type `exit`.
|
||||
# turn on mouse-wheel scrolling in the scrollback
|
||||
termcapinfo xterm* ti@:te@
|
||||
```
|
||||
|
||||
## 🧠 Tips
|
||||
|
||||
- Use **named sessions** (`screen -S`) to avoid confusion.
|
||||
- Combine with **SSH** to keep remote jobs running after disconnect.
|
||||
- Use `.screenrc` to customize behavior and status line.
|
||||
- Name your sessions (`screen -S deploy`) — `screen -ls` is unreadable once you have three sessions called `12345.pts-0.host`.
|
||||
- Over SSH, start work inside screen: the job survives a dropped connection, and `screen -d -r` picks it back up.
|
||||
- Nested screens (local + remote): press `Ctrl+a a` to send the prefix through to the inner session.
|
||||
- `screen -x` attaches several clients to the same session — handy for pair debugging.
|
||||
|
||||
## 📚 Resources
|
||||
|
||||
- [GNU Screen manual](https://www.gnu.org/software/screen/manual/screen.html)
|
||||
- [Manual page](https://man7.org/linux/man-pages/man1/screen.1.html)
|
||||
- Built-in help: `Ctrl+a ?`
|
||||
+229
-167
@@ -1,211 +1,273 @@
|
||||
# 🧾 `sed` Command Cheatsheet (Stream Editor)
|
||||
|
||||
`sed` is a powerful Unix utility for parsing and transforming text using **stream editing**.
|
||||
|
||||
## 📌 Basic Syntax
|
||||
**`sed`** is the stream editor: it reads input line by line into the *pattern space*, applies a script to each line, and prints the result. It never needs the whole file in memory, so it happily edits gigabytes.
|
||||
|
||||
```bash
|
||||
sed [OPTIONS] 'script' [file...]
|
||||
sed [options] 'script' [file...]
|
||||
```
|
||||
|
||||
- `'script'`: One or more editing commands.
|
||||
- `[file]`: Input file(s). If omitted, `sed` reads from standard input.
|
||||
> ℹ️ Examples here use **GNU sed** (Linux). BSD/macOS differs on `-i`, `-E` and the `a`/`i`/`c` commands — see the callouts. `brew install gnu-sed` gives you `gsed` on macOS.
|
||||
|
||||
## ⚙️ Common Options
|
||||
## ⚙️ Options
|
||||
|
||||
| Option | Description |
|
||||
|--------|-------------|
|
||||
| `-n` | Suppress automatic printing of pattern space. Use `p` to print explicitly. |
|
||||
| `-e` | Add multiple editing commands. |
|
||||
| `-i` | Edit files in-place (with optional backup: `-i.bak`). |
|
||||
| `-f` | Read commands from a file. |
|
||||
| Option | Description |
|
||||
|----------------|--------------------------------------------------------------------|
|
||||
| `-n` | Do not print automatically — pair it with `p` |
|
||||
| `-e <script>` | Add a script; repeat for several |
|
||||
| `-f <file>` | Read the script from a file |
|
||||
| `-i[SUFFIX]` | Edit files in place, optionally keeping a backup |
|
||||
| `-E` (or `-r`) | Extended regular expressions — no backslashes before `(`, `+`, `?` |
|
||||
| `-s` | Treat each file separately instead of one long stream |
|
||||
| `-z` | Split on NUL instead of newline (pairs with `find -print0`) |
|
||||
| `--debug` | Show how sed executes the script (GNU 4.6+) |
|
||||
|
||||
## ✂️ Basic Commands
|
||||
## 📍 Addressing
|
||||
|
||||
| Command | Description |
|
||||
|---------|-------------|
|
||||
| `p` | Print the current pattern space. |
|
||||
| `d` | Delete the current pattern space. |
|
||||
| `s` | Substitute text using regex. |
|
||||
| `q` | Quit after processing first match. |
|
||||
| `a` | Append text after current line. |
|
||||
| `i` | Insert text before current line. |
|
||||
| `c` | Replace line with new text. |
|
||||
| `y` | Translate characters (like `tr`). |
|
||||
An address decides *which* lines a command applies to. Without one, the command applies to every line.
|
||||
|
||||
## 🔍 Substitution Syntax
|
||||
| Address | Matches |
|
||||
|-----------------|-----------------------------------------------------|
|
||||
| `5` | Line 5 |
|
||||
| `$` | The last line |
|
||||
| `/regex/` | Every line matching the pattern |
|
||||
| `\%regex%` | Same, with `%` as the delimiter (handy for paths) |
|
||||
| `2,7` | Lines 2 to 7 |
|
||||
| `2,$` | Line 2 to the end |
|
||||
| `/start/,/end/` | From the first `start` to the next `end`, inclusive |
|
||||
| `2,+3` | Line 2 and the three lines after it |
|
||||
| `0~3` | Every third line (GNU) |
|
||||
| `2~5` | Line 2, then every fifth line (GNU) |
|
||||
| `/regex/!` | Invert — every line that does **not** match |
|
||||
|
||||
```bash
|
||||
# print line 5
|
||||
sed -n '5p' file
|
||||
|
||||
# print lines 3 to 5
|
||||
sed -n '3,5p' file
|
||||
|
||||
# print everything between two markers
|
||||
sed -n '/BEGIN/,/END/p' file
|
||||
|
||||
# delete every line except the ones matching
|
||||
sed -n '/keep/p' file
|
||||
sed '/keep/!d' file
|
||||
|
||||
# count the lines (like wc -l)
|
||||
sed -n '$=' file
|
||||
```
|
||||
|
||||
## 🔁 Substitution
|
||||
|
||||
```bash
|
||||
sed 's/pattern/replacement/flags' file
|
||||
```
|
||||
|
||||
### Flags
|
||||
|
||||
| Flag | Description |
|
||||
|------|-------------|
|
||||
| `g` | Global replacement (all matches in line). |
|
||||
| `i` | Case-insensitive matching. |
|
||||
| `p` | Print the line if substitution occurred. |
|
||||
| `n` | Replace nth occurrence only. |
|
||||
|
||||
### Examples
|
||||
| Flag | Effect |
|
||||
|--------|------------------------------------------------------|
|
||||
| *none* | Replace the first match on each line |
|
||||
| `g` | Replace every match on the line |
|
||||
| `i` | Case-insensitive matching |
|
||||
| `p` | Print the line when a replacement happened |
|
||||
| `2` | Replace only the second match — any number works |
|
||||
| `2g` | Replace from the second match to the end of the line |
|
||||
| `w f` | Write the changed lines to file `f` |
|
||||
| `e` | Run the result as a shell command (GNU) |
|
||||
|
||||
```bash
|
||||
sed 's/foo/bar/' file # Replace first 'foo' with 'bar'
|
||||
sed 's/foo/bar/g' file # Replace all 'foo' with 'bar'
|
||||
sed 's/foo/bar/2' file # Replace second 'foo' only
|
||||
sed 's/foo/bar/ip' file # Case-insensitive + print
|
||||
```
|
||||
# first match on each line
|
||||
sed 's/foo/bar/' file
|
||||
|
||||
## 🗑️ Deletion with `sed`
|
||||
# every match on the line
|
||||
sed 's/foo/bar/g' file
|
||||
|
||||
`sed` can delete lines based on line numbers, patterns, or ranges.
|
||||
# only the second match
|
||||
sed 's/foo/bar/2' file
|
||||
|
||||
### 🔢 Delete by Line Number
|
||||
# every match, ignoring case
|
||||
sed 's/foo/bar/gI' file
|
||||
|
||||
```bash
|
||||
sed '2d' file # Delete line 2
|
||||
sed '5,10d' file # Delete lines 5 through 10
|
||||
```
|
||||
# & stands for the whole match — wrap every number in brackets
|
||||
sed 's/[0-9]\+/[&]/g' file
|
||||
|
||||
### 🔍 Delete by Pattern
|
||||
|
||||
```bash
|
||||
sed '/error/d' file # Delete lines containing 'error'
|
||||
sed '/^$/d' file # Delete blank lines
|
||||
sed '/^#/d' file # Delete comment lines (starting with #)
|
||||
```
|
||||
|
||||
### 🧮 Delete by Range and Pattern
|
||||
|
||||
```bash
|
||||
sed '1,/pattern/d' file # Delete from line 1 to first match of 'pattern'
|
||||
sed '/start/,/end/d' file # Delete lines between 'start' and 'end' (inclusive)
|
||||
```
|
||||
|
||||
### 🧠 Conditional Deletion
|
||||
|
||||
```bash
|
||||
sed -n '/pattern/!p' file # Print only lines NOT matching 'pattern'
|
||||
```
|
||||
|
||||
### 🧹 Delete Last Line
|
||||
|
||||
```bash
|
||||
sed '$d' file # Delete the last line
|
||||
```
|
||||
|
||||
## 📍 Addressing Lines
|
||||
|
||||
### Line Numbers
|
||||
|
||||
```bash
|
||||
sed '2d' file # Delete line 2
|
||||
sed '3,5p' file # Print lines 3 to 5
|
||||
```
|
||||
|
||||
### Patterns
|
||||
|
||||
```bash
|
||||
sed '/error/d' file # Delete lines containing 'error'
|
||||
sed '/^#/d' file # Delete comment lines
|
||||
```
|
||||
|
||||
### Combined
|
||||
|
||||
```bash
|
||||
sed '1,/pattern/d' file # Delete from line 1 to first match of 'pattern'
|
||||
```
|
||||
|
||||
## 🧪 Advanced Substitution
|
||||
|
||||
### Using Capture Groups
|
||||
|
||||
```bash
|
||||
# capture groups: \( \) in basic regex, ( ) with -E
|
||||
sed 's/\(foo\)bar/\1baz/' file
|
||||
```
|
||||
sed -E 's/(\w+)@(\w+)/\2 at \1/' file
|
||||
|
||||
- `\(...\)` captures a group.
|
||||
- `\1`, `\2`, etc., refer to captured groups.
|
||||
|
||||
### Escaping Special Characters
|
||||
|
||||
```bash
|
||||
sed 's/\/usr\/bin/\/usr\/local\/bin/' file
|
||||
```
|
||||
|
||||
Or use alternate delimiter:
|
||||
|
||||
```bash
|
||||
# any character can be the delimiter — pick one that is not in the text
|
||||
sed 's|/usr/bin|/usr/local/bin|' file
|
||||
sed 's#http://#https://#' file
|
||||
|
||||
# GNU case conversion: upper-case the first word
|
||||
sed 's/\w\+/\U&/' file
|
||||
|
||||
# ...and lower-case a whole line
|
||||
sed -E 's/(.*)/\L\1/' file
|
||||
|
||||
# only substitute on lines that match an address
|
||||
sed '/^deb /s/http:/https:/' sources.list
|
||||
```
|
||||
|
||||
## ✂️ Command Reference
|
||||
|
||||
| Command | Does |
|
||||
|----------------------|-----------------------------------------------------------------|
|
||||
| `p` | Print the pattern space |
|
||||
| `d` | Delete it and start the next cycle |
|
||||
| `s` | Substitute |
|
||||
| `y` | Transliterate characters, like `tr` |
|
||||
| `a text` | Append a line after the current one |
|
||||
| `i text` | Insert a line before the current one |
|
||||
| `c text` | Replace the matched line(s) |
|
||||
| `q` | Quit — `q5` also sets the exit status |
|
||||
| `Q` | Quit without printing the current line |
|
||||
| `r file` | Read a file in after the current line |
|
||||
| `w file` | Write the pattern space to a file |
|
||||
| `=` | Print the current line number |
|
||||
| `n` / `N` | Load the next line, replacing / appending to the pattern space |
|
||||
| `D` / `P` | Delete / print up to the first newline of the pattern space |
|
||||
| `h` `H` | Copy / append the pattern space into the hold space |
|
||||
| `g` `G` | Copy / append the hold space into the pattern space |
|
||||
| `x` | Swap the pattern and hold spaces |
|
||||
| `b` `t` `T` `:label` | Branching — jump, jump if a substitution happened, jump if none |
|
||||
|
||||
## 🗑 Deleting & Printing
|
||||
|
||||
```bash
|
||||
# by line number
|
||||
sed '2d' file
|
||||
sed '5,10d' file
|
||||
sed '$d' file
|
||||
|
||||
# lines containing a word
|
||||
sed '/error/d' file
|
||||
|
||||
# blank lines
|
||||
sed '/^$/d' file
|
||||
|
||||
# comment lines
|
||||
sed '/^#/d' file
|
||||
|
||||
# blank or whitespace-only lines
|
||||
sed '/^\s*$/d' file
|
||||
|
||||
# strip comments and blank lines from a config
|
||||
sed -e '/^#/d' -e '/^$/d' /etc/ssh/sshd_config
|
||||
|
||||
# from the top up to the first match
|
||||
sed '1,/pattern/d' file
|
||||
|
||||
# everything between two markers
|
||||
sed '/start/,/end/d' file
|
||||
|
||||
# keep only what matches
|
||||
sed -n '/pattern/p' file
|
||||
```
|
||||
|
||||
## 🧵 Insert, Append & Change
|
||||
|
||||
```bash
|
||||
# GNU one-line form
|
||||
sed '/pattern/a appended line' file
|
||||
sed '/pattern/i inserted line' file
|
||||
sed '/pattern/c replacement line' file
|
||||
|
||||
# portable form (works on BSD/macOS too)
|
||||
sed '/pattern/a\
|
||||
appended line' file
|
||||
|
||||
# add a line at the very top or bottom
|
||||
sed '1i #!/bin/bash' file
|
||||
sed '$a # end of file' file
|
||||
|
||||
# insert the contents of another file after a marker
|
||||
sed '/INCLUDE HERE/r snippet.txt' file
|
||||
```
|
||||
|
||||
## 🧠 Multi-line & Hold Space
|
||||
|
||||
The hold space is a second buffer that survives between lines — that is what makes sed more than a line-at-a-time filter.
|
||||
|
||||
```bash
|
||||
# join every pair of lines
|
||||
sed 'N;s/\n/ /' file
|
||||
|
||||
# reverse the file (what tac does)
|
||||
sed -n '1!G;h;$p' file
|
||||
|
||||
# print the line before each match
|
||||
sed -n '/pattern/{x;p;x};h' file
|
||||
|
||||
# print the line after each match
|
||||
sed -n '/pattern/{n;p}' file
|
||||
|
||||
# squeeze runs of blank lines into one
|
||||
sed '/^$/{N;/^\n$/D}' file
|
||||
|
||||
# delete the last two lines
|
||||
sed 'N;$!P;$!D;$d' file
|
||||
```
|
||||
|
||||
## 🧨 In-Place Editing
|
||||
|
||||
```bash
|
||||
sed -i 's/foo/bar/g' file # Edit file directly
|
||||
sed -i.bak 's/foo/bar/g' file # Backup original as file.bak
|
||||
# GNU: edit in place
|
||||
sed -i 's/foo/bar/g' file
|
||||
|
||||
# keep a backup as file.bak
|
||||
sed -i.bak 's/foo/bar/g' file
|
||||
|
||||
# several files at once
|
||||
sed -i 's/foo/bar/g' *.conf
|
||||
|
||||
# every matching file in a tree
|
||||
find . -name '*.py' -exec sed -i 's/old_api/new_api/g' {} +
|
||||
```
|
||||
|
||||
## 📂 Multiple Commands
|
||||
> ⚠️ **BSD/macOS `sed` requires an argument to `-i`.** `sed -i '' 's/foo/bar/' file` edits without a backup there, while the same command on GNU sed reads `''` as the script. Write `sed -i.bak` for something that works on both, or use `gsed`.
|
||||
>
|
||||
> 💡 Always run the command without `-i` first and read the output. `-i` has no undo.
|
||||
|
||||
### Inline
|
||||
## 🧰 Recipes
|
||||
|
||||
```bash
|
||||
sed -e 's/foo/bar/' -e '/baz/d' file
|
||||
```
|
||||
# trim leading and trailing whitespace
|
||||
sed 's/^[[:space:]]*//; s/[[:space:]]*$//' file
|
||||
|
||||
### Block Syntax
|
||||
# collapse repeated spaces
|
||||
sed 's/ */ /g' file
|
||||
|
||||
```bash
|
||||
sed '
|
||||
s/foo/bar/
|
||||
s/baz/qux/
|
||||
' file
|
||||
```
|
||||
|
||||
## 🧵 Append, Insert, Change
|
||||
|
||||
```bash
|
||||
sed '/pattern/a\Text to append' file
|
||||
sed '/pattern/i\Text to insert' file
|
||||
sed '/pattern/c\New line content' file
|
||||
```
|
||||
|
||||
## 🔄 Translate Characters
|
||||
|
||||
```bash
|
||||
sed 'y/abc/ABC/' file # a→A, b→B, c→C
|
||||
```
|
||||
|
||||
## 🧠 Useful Recipes
|
||||
|
||||
### Remove Blank Lines
|
||||
|
||||
```bash
|
||||
sed '/^$/d' file
|
||||
```
|
||||
|
||||
### Remove Leading/Trailing Whitespace
|
||||
|
||||
```bash
|
||||
sed 's/^[ \t]*//' file # Remove leading
|
||||
sed 's/[ \t]*$//' file # Remove trailing
|
||||
```
|
||||
|
||||
### Replace Tabs with Spaces
|
||||
|
||||
```bash
|
||||
# tabs to four spaces
|
||||
sed 's/\t/ /g' file
|
||||
```
|
||||
|
||||
### Number Lines
|
||||
# strip Windows carriage returns
|
||||
sed 's/\r$//' file
|
||||
|
||||
```bash
|
||||
# strip HTML tags
|
||||
sed -e 's/<[^>]*>//g' page.html
|
||||
|
||||
# number the lines
|
||||
sed = file | sed 'N;s/\n/\t/'
|
||||
|
||||
# print a specific line and quit early (fast on huge files)
|
||||
sed -n '1000{p;q}' file
|
||||
|
||||
# extract the value of a key from a config file
|
||||
sed -n 's/^Port[[:space:]]*//p' /etc/ssh/sshd_config
|
||||
|
||||
# replace only in lines between two markers
|
||||
sed '/BEGIN/,/END/ s/old/new/g' file
|
||||
|
||||
# change a value only on the first match, then stop
|
||||
sed '0,/version:/s//version: 2/' file
|
||||
|
||||
# comment out a line
|
||||
sed '/^ExecStart/s/^/#/' unit.service
|
||||
|
||||
# uncomment a line
|
||||
sed '/^#ExecStart/s/^#//' unit.service
|
||||
```
|
||||
|
||||
## 📚 Resources
|
||||
|
||||
- `man sed`
|
||||
- GNU sed manual: [https://www.gnu.org/software/sed/manual/sed.html](https://www.gnu.org/software/sed/manual/sed.html)
|
||||
- [GNU sed manual](https://www.gnu.org/software/sed/manual/sed.html)
|
||||
- [One-liners, annotated (sed1line)](https://sed.sourceforge.io/sed1line.txt)
|
||||
- Manual page: `man 1 sed`
|
||||
@@ -1,57 +1,170 @@
|
||||
# 🖥️ tmux Linux Command Cheatsheet
|
||||
**tmux** is a terminal multiplexer: it keeps shell sessions alive on the server, splits one terminal into windows and panes, and lets you detach and reattach without losing anything.
|
||||
|
||||
## 🔑 Basics
|
||||
- **Start new session**: `tmux`
|
||||
- **Start named session**: `tmux new -s <name>`
|
||||
- **List sessions**: `tmux ls`
|
||||
- **Attach to last session**: `tmux attach`
|
||||
- **Attach to named session**: `tmux attach -t <name>`
|
||||
- **Detach from session**: `Ctrl+b d`
|
||||
- **Kill session**: `tmux kill-session -t <name>`
|
||||
- **Rename session**: `Ctrl+b $`
|
||||
> ℹ️ Almost every shortcut starts with the **prefix**, `Ctrl+b` by default: press the prefix, release it, then press the key. In the tables below `Ctrl+b c` means "prefix, then `c`".
|
||||
|
||||
## 📂 Windows (like tabs)
|
||||
- **Create new window**: `Ctrl+b c`
|
||||
- **Rename window**: `Ctrl+b ,`
|
||||
- **Next window**: `Ctrl+b n`
|
||||
- **Previous window**: `Ctrl+b p`
|
||||
- **List windows**: `Ctrl+b w`
|
||||
- **Switch to window #**: `Ctrl+b <0-9>`
|
||||
- **Kill current window**: `Ctrl+b &`
|
||||
## 🚀 Sessions from the Shell
|
||||
|
||||
```bash
|
||||
# start a session
|
||||
tmux
|
||||
|
||||
# start a named session
|
||||
tmux new -s <name>
|
||||
|
||||
# start a named session in the background
|
||||
tmux new -s <name> -d
|
||||
|
||||
# attach if the session exists, create it otherwise
|
||||
tmux new-session -A -s <name>
|
||||
|
||||
# list sessions
|
||||
tmux ls
|
||||
|
||||
# attach to the most recent session
|
||||
tmux attach
|
||||
|
||||
# attach to a named session
|
||||
tmux attach -t <name>
|
||||
|
||||
# attach and detach any other client (steal the session)
|
||||
tmux attach -d -t <name>
|
||||
|
||||
# rename a session
|
||||
tmux rename-session -t <old> <new>
|
||||
|
||||
# kill one session
|
||||
tmux kill-session -t <name>
|
||||
|
||||
# kill the server and every session on it
|
||||
tmux kill-server
|
||||
```
|
||||
|
||||
## ⌨️ Sessions
|
||||
|
||||
| Keys | Action |
|
||||
|------------|----------------------------------|
|
||||
| `Ctrl+b d` | Detach from the session |
|
||||
| `Ctrl+b s` | Interactive session list |
|
||||
| `Ctrl+b $` | Rename the current session |
|
||||
| `Ctrl+b (` | Previous session |
|
||||
| `Ctrl+b )` | Next session |
|
||||
| `Ctrl+b w` | Tree of every session and window |
|
||||
|
||||
## 📂 Windows (tabs)
|
||||
|
||||
| Keys | Action |
|
||||
|--------------|----------------------------------|
|
||||
| `Ctrl+b c` | Create a window |
|
||||
| `Ctrl+b ,` | Rename the current window |
|
||||
| `Ctrl+b n` | Next window |
|
||||
| `Ctrl+b p` | Previous window |
|
||||
| `Ctrl+b l` | Last (previously used) window |
|
||||
| `Ctrl+b 0…9` | Jump to a window by number |
|
||||
| `Ctrl+b w` | Choose a window from a list |
|
||||
| `Ctrl+b f` | Find a window by name |
|
||||
| `Ctrl+b .` | Move the window to another index |
|
||||
| `Ctrl+b &` | Kill the current window |
|
||||
|
||||
## 🪟 Panes (splits)
|
||||
- **Split vertically**: `Ctrl+b %`
|
||||
- **Split horizontally**: `Ctrl+b "`
|
||||
- **Move between panes**: `Ctrl+b <arrow keys>`
|
||||
- **Cycle panes**: `Ctrl+b o`
|
||||
- **Toggle pane zoom**: `Ctrl+b z`
|
||||
- **Kill pane**: `Ctrl+b x`
|
||||
- **Swap panes**: `Ctrl+b {` (swap with previous), `Ctrl+b }` (swap with next)
|
||||
|
||||
## 📋 Copy Mode
|
||||
- **Enter copy mode**: `Ctrl+b [`
|
||||
- **Start selection**: `<Space>`
|
||||
- **Copy selection**: `<Enter>`
|
||||
- **Quit copy mode**: `q`
|
||||
- **Paste buffer**: `Ctrl+b ]`
|
||||
| Keys | Action |
|
||||
|-------------------|--------------------------------------------|
|
||||
| `Ctrl+b %` | Split into left / right panes |
|
||||
| `Ctrl+b "` | Split into top / bottom panes |
|
||||
| `Ctrl+b ←↑↓→` | Move the focus in that direction |
|
||||
| `Ctrl+b o` | Cycle through the panes |
|
||||
| `Ctrl+b ;` | Jump to the previously focused pane |
|
||||
| `Ctrl+b q` | Show pane numbers (press one to jump) |
|
||||
| `Ctrl+b z` | Zoom the pane to full screen, and back |
|
||||
| `Ctrl+b x` | Kill the current pane |
|
||||
| `Ctrl+b {` / `}` | Swap the pane with the previous / next one |
|
||||
| `Ctrl+b space` | Cycle through the preset layouts |
|
||||
| `Ctrl+b !` | Break the pane out into its own window |
|
||||
| `Ctrl+b Alt+←↑↓→` | Resize the pane in that direction |
|
||||
|
||||
## 📋 Copy Mode & Scrollback
|
||||
|
||||
| Keys | Action |
|
||||
|-------------|-----------------------------------------------|
|
||||
| `Ctrl+b [` | Enter copy mode (scroll with the arrows/PgUp) |
|
||||
| `space` | Start the selection |
|
||||
| `Enter` | Copy the selection and leave copy mode |
|
||||
| `Ctrl+b ]` | Paste the buffer |
|
||||
| `/` and `?` | Search forward / backward (vi mode) |
|
||||
| `q` | Leave copy mode |
|
||||
|
||||
> 💡 `setw -g mode-keys vi` gives you vi navigation in copy mode; `v` then starts the selection and `y` yanks it.
|
||||
|
||||
## ⚙️ Configuration
|
||||
- **Reload config**: `tmux source-file ~/.tmux.conf`
|
||||
- **Show options**: `tmux show-options -g`
|
||||
- **List all shortcuts**: `Ctrl+b ?`
|
||||
|
||||
## 🧭 Useful Commands
|
||||
- **Command prompt**: `Ctrl+b :`
|
||||
- **List sessions/windows/panes**: `tmux info`
|
||||
- **Kill server (all sessions)**: `tmux kill-server`
|
||||
```bash
|
||||
# reload the configuration without restarting
|
||||
tmux source-file ~/.tmux.conf
|
||||
|
||||
## 📝 Notes
|
||||
- **Prefix key**: Default is `Ctrl+b`. Press it before any shortcut.
|
||||
- **Custom prefix**: Can be changed in `~/.tmux.conf` (e.g., `set -g prefix C-a`).
|
||||
# every key binding currently active
|
||||
tmux list-keys
|
||||
|
||||
### Example Workflow
|
||||
1. Start a named session: `tmux new -s dev`
|
||||
2. Split into panes: `Ctrl+b %` and `Ctrl+b "`
|
||||
3. Create multiple windows: `Ctrl+b c`
|
||||
4. Detach: `Ctrl+b d`
|
||||
5. Reattach later: `tmux attach -t dev`
|
||||
# every option and its value
|
||||
tmux show-options -g
|
||||
```
|
||||
|
||||
A reasonable `~/.tmux.conf` starting point:
|
||||
|
||||
```bash
|
||||
# use Ctrl+a as the prefix, like GNU Screen
|
||||
set -g prefix C-a
|
||||
unbind C-b
|
||||
bind C-a send-prefix
|
||||
|
||||
# number windows from 1 and close the gaps
|
||||
set -g base-index 1
|
||||
set -g renumber-windows on
|
||||
|
||||
# mouse: focus panes, resize splits, scroll
|
||||
set -g mouse on
|
||||
|
||||
# vi keys in copy mode, and a longer scrollback
|
||||
setw -g mode-keys vi
|
||||
set -g history-limit 10000
|
||||
|
||||
# reload with prefix + r
|
||||
bind r source-file ~/.tmux.conf \; display "Config reloaded"
|
||||
```
|
||||
|
||||
## 🧭 Scripting & Automation
|
||||
|
||||
```bash
|
||||
# run a command in a target pane
|
||||
tmux send-keys -t <session>:<window>.<pane> "make test" Enter
|
||||
|
||||
# print the contents of a pane
|
||||
tmux capture-pane -p -t <session>:<window>
|
||||
|
||||
# open the command prompt inside tmux (prefix + :)
|
||||
tmux command-prompt
|
||||
|
||||
# build a session non-interactively
|
||||
tmux new -d -s dev -n editor
|
||||
tmux split-window -t dev -h
|
||||
tmux attach -t dev
|
||||
```
|
||||
|
||||
## 📄 Example Workflow
|
||||
|
||||
```bash
|
||||
# 1. start a named session
|
||||
tmux new -s dev
|
||||
|
||||
# 2. split the window: prefix % (left/right), prefix " (top/bottom)
|
||||
# 3. add another window with prefix c
|
||||
# 4. detach with prefix d — everything keeps running
|
||||
|
||||
# 5. come back later
|
||||
tmux attach -t dev
|
||||
```
|
||||
|
||||
## 📚 Resources
|
||||
|
||||
- [tmux wiki and getting-started guide](https://github.com/tmux/tmux/wiki)
|
||||
- [Manual page](https://man7.org/linux/man-pages/man1/tmux.1.html)
|
||||
- Built-in help: `Ctrl+b ?` lists every active binding
|
||||
File diff suppressed because it is too large.
Load diff
+247
-248
@@ -1,248 +1,247 @@
|
||||
> Credit to the original author: https://gist.github.com/jmaccabee/550a0b9fcfdc7e6b170cd34c6ec7bc56
|
||||
|
||||
## 0. XPath Examples.
|
||||
|
||||
> More: http://xpath.alephzarro.com/content/cheatsheet.html
|
||||
|
||||
|
||||
| XPath | Description |
|
||||
| -------------- | ----------- |
|
||||
| `//hr[@class="edge" and position()=1]` | every first hr of 'edge' class
|
||||
| `//table[count(tr)=1 and count(tr/td)=2]` | all tables with 1 row and 2 cols
|
||||
| `//div/form/parent::*` | all divs that have form
|
||||
| `./div/b` | a relative path
|
||||
| `//table[parent::div[@class="pad"] and not(@id)]//a` | any anchor in a table without id, contained in a div of "pad" class
|
||||
| `/html/body/div/*[preceding-sibling::h4]` | give me whatever after h4
|
||||
| `//tr/td[font[@class="head" and text()="TRACK"]]` | all td that has font of a "head" class and text "TRACK"
|
||||
| `./table/tr[last()]` | the last row of a table
|
||||
| `//rdf:Seq/rdf:li/em:id` | using namespaces
|
||||
| `//a/@href` | hrefs of all anchors
|
||||
| `//*[count(*)=3]` | all nodes with 3 children
|
||||
| `//var\|//acronym` | all vars and acronyms
|
||||
|
||||
|
||||
## 1. General.
|
||||
|
||||
|
||||
| XPath | Description |
|
||||
| -------------- | ----------- |
|
||||
| `/html` | whole web page (css: html)
|
||||
| `/html/body` | whole web page body (css: body)
|
||||
| `//text()` | all text nodes of web page
|
||||
| `/html/body/.../.../.../E` | element \<E\> by absolute reference (css: body > … > … > … > E)
|
||||
|
||||
|
||||
## 2. Tag.
|
||||
|
||||
|
||||
| XPath | Description |
|
||||
| -------------- | ----------- |
|
||||
| `//E` | element \<E\> by relative reference (css: E)
|
||||
| `(//E)[2]` | second \<E\> element anywhere on page
|
||||
| `//img` | image element (css: img)
|
||||
| `//E[@A]` | element \<E\> with attribute A (css: E[A])
|
||||
| `//E[@A="t"]` | element \<E\> with attribute A containing text 't' exactly (css: E[A='t'])
|
||||
| `//E[contains(@A,"t")]` | element \<E\> with attribute A containing text 't' (css: E[A*='t'])
|
||||
| `//E[starts-with(@A, "t")]` | element \<E\> whose attribute A begins with 't' (css: E[A^='t'])
|
||||
| `//E[ends-with(@A, "t")]` | element \<E\> whose attribute A ends with 't' (css: E[A$='t'])
|
||||
| `//E[contains(concat(" ", @A, " "), " w ")` | element \<E\> with attribute A containing word 'w' (css: E[A~='w'])
|
||||
| `//E[matches(@A, "r")]` | element \<E\> with attribute A matching regex ‘r’
|
||||
| `//E1[@id=I1] \| //E2[@id=I2]` | element \<E1\> with id I1 or element \<E2\> with id I2 (css: E1#I1, E2#I2)
|
||||
| `//E1[@id=I1 or @id=I2]` | element \<E1\> with id I1 or id I2 (css: E1#I1, E1#I2)
|
||||
|
||||
|
||||
## 3. Attribute.
|
||||
|
||||
|
||||
| XPath | Description |
|
||||
| -------------- | ----------- |
|
||||
| `//E/@A` | attribute A of element \<E\> (css: E@A)
|
||||
| `//*/@A` | attribute A of any element (css: *@A)
|
||||
| `//E[@A2="t"]/@A1` | attribute A1 of element \<E\> where attribute A2 is 't' exactly (css: E[A2='t']@A1)
|
||||
| `//E[contains(@A,"t")]/@A` | attribute A of element \<E\> where A contains 't' (css: E[A*='t']@A)
|
||||
|
||||
|
||||
## 4. ID and Name.
|
||||
|
||||
|
||||
| XPath | Description |
|
||||
| -------------- | ----------- |
|
||||
| `//*[@id="I"]` | element with id I (css: #I)
|
||||
| `//E[@id="I"]` | element \<E\> with id I (css: E#I)
|
||||
| `//*[@name="N"]` | element with name (css: [name='N'])
|
||||
| `//E[@name="N"]` | element \<E\> with name (css: E[name='N'])
|
||||
| `//*[@id="X" or @name="X"]` | element with id X or, failing that, a name X
|
||||
| `//*[@name="N"][v+1]` | element with name N & specified 0-based index ‘v’ (css: [name='N']:nth-child(v+1))
|
||||
| `//*[@name="N"][@value="v"]` | element with name N & specified value ‘v’ (css: *[name='N'][value='v’])
|
||||
|
||||
|
||||
## 5. Lang and Class.
|
||||
|
||||
|
||||
| XPath | Description |
|
||||
| -------------- | ----------- |
|
||||
| `//E[@lang="L" or starts-with(@lang, concat("L", "-"))]` | element \<E\> is explicitly in language L or subcode (css: E[lang\|=L])
|
||||
| `//*[contains(concat(" ", @class, " "), " C ")]` | element with a class C (css: .C)
|
||||
| `//E[contains(concat(" ", @class, " "), " C ")]` | element \<E\> with a class C (css: E.C)
|
||||
|
||||
|
||||
## 6. Text and Link.
|
||||
|
||||
|
||||
| XPath | Description |
|
||||
| -------------- | ----------- |
|
||||
| `//*[.="t"]` | element containing text 't' exactly
|
||||
| `//E[contains(text(), "t")]` | element \<E\> containing text 't' (css: E:contains('t'))
|
||||
| `//a` | link element (css: a)
|
||||
| `//a[.="t"]` | element \<a\> containing text 't' exactly
|
||||
| `//a[contains(text(), "t")]` | element \<a\> containing text 't' (css: a:contains('t'))
|
||||
| `//a[@href="url"]` | \<a\> with target link 'url' (css: a[href='url'])
|
||||
| `//a[.="t"]/@href` | link URL labeled with text 't' exactly
|
||||
|
||||
|
||||
## 7. Parent and Child.
|
||||
|
||||
|
||||
| XPath | Description |
|
||||
| -------------- | ----------- |
|
||||
| `//E/*[1]` | first child of element \<E\> (css: E > *:first-child)
|
||||
| `//E[1]` | first \<E\> child (css: E:first-of-type)
|
||||
| `//E/*[last()]` | last child of element E (css: E *:last-child)
|
||||
| `//E[last()]` | last \<E\> child (css: E:last-of-type)
|
||||
| `//E[2]` | second \<E\> child (css: E:nth-of-type(2))
|
||||
| `//*[2][name()="E"]` | second child that is an \<E\> element (css: E:nth-child(2))
|
||||
| `//E[last()-1]` | second-to-last \<E\> child (css: E:nth-last-of-type(2))
|
||||
| `//*[last()-1][name()="E"]` | second-to-last child that is an \<E\> element (css: E:nth-last-child(2))
|
||||
| `//E1/[E2 and not( *[not(self::E2)])]` | element \<E1\> with only \<E2\> children
|
||||
| `//E/..` | parent of element \<E\>
|
||||
| `//*[@id="I"]/.../.../.../E` | descendant \<E\> of element with id I using specific path (css: #I > … > … > … > E)
|
||||
| `//*[@id="I"]//E` | descendant \<E\> of element with id I using unspecified path (css: #I E)
|
||||
| `//E[count(*)=0]` | element \<E\> with no children (E:empty)
|
||||
| `//E[count(*)=1]` | element \<E\> with an only child
|
||||
| `//E[count(preceding-sibling::*)+count(following-sibling::*)=0]` | element \<E\> that is an only child (css: E:only-child)
|
||||
| `//E[count(../E) = 1]` | element \<E\> with no \<E\> siblings (css: E:only-of-type)
|
||||
| `//E[position() mod N = M + 1]` | every Nth element starting with the (M+1)th (css: E:nth-child(Nn+M))
|
||||
|
||||
|
||||
## 8. Sibling.
|
||||
|
||||
|
||||
| XPath | Description |
|
||||
| -------------- | ----------- |
|
||||
| `//E2/following-sibling::E1` | element \<E1\> following some sibling \<E2\> (css: E2 ~ E1)
|
||||
| `//E2/following-sibling::*[1][name()="E1"]` | element \<E1\> immediately following sibling \<E2\> (css: E2 + E1)
|
||||
| `//E2/following-sibling::*[2][name()="E1"]` | element \<E1\> following sibling \<E2\> with one intermediary (css: E2 + * + E1)
|
||||
| `//E/following-sibling::*` | sibling element immediately following \<E\> (css: E + *)
|
||||
| `//E2/preceding-sibling::E1` | element \<E1\> preceding some sibling \<E2\>
|
||||
| `//E2/preceding-sibling::*[1][name()="E1"]` | element \<E1\> immediately preceding sibling \<E2\>
|
||||
| `//E2/preceding-sibling::*[2][name()="E1"]` | element \<E1\> preceding sibling \<E2\> with one intermediary
|
||||
| `//E/preceding-sibling::*[1]` | sibling element immediately preceding \<E\>
|
||||
|
||||
|
||||
## 9. Table Cell.
|
||||
|
||||
|
||||
| XPath | Description |
|
||||
| -------------- | ----------- |
|
||||
| `//*[@id="TestTable"]//tr[3]//td[2]` | cell by row and column (e.g. 3rd row, 2nd column) (css: #TestTable tr:nth-child(3) td:nth-child(2))
|
||||
| `//td[preceding-sibling::td="t"]` | cell immediately following cell containing 't' exactly
|
||||
| `td[preceding-sibling::td[contains(.,"t")]]` | cell immediately following cell containing 't' (css: td:contains('t') ~ td)
|
||||
|
||||
|
||||
## 10. Dynamic.
|
||||
|
||||
|
||||
| XPath | Description |
|
||||
| -------------- | ----------- |
|
||||
| `//E[@disabled]` | user interface element \<E\> that is disabled (css: E:disabled)
|
||||
| `//*[not(@disabled)]` | user interface element that is enabled (css: E:enabled)
|
||||
| `//*[@checked]` | checkbox (or radio button) that is checked (css: *:checked)
|
||||
|
||||
|
||||
## 11. XPath Functions.
|
||||
|
||||
> https://developer.mozilla.org/en-US/docs/Web/XPath/Functions
|
||||
|
||||
|
||||
### 11.1. Conversion.
|
||||
|
||||
|
||||
| XPath | Description |
|
||||
| -------------- | ----------- |
|
||||
| `boolean(expression)` | evaluates an expression and returns true or false.
|
||||
| `string([object])` | converts the given argument to a string.
|
||||
| `number([object])` | converts an object to a number and returns the number.
|
||||
|
||||
|
||||
### 11.2. Math.
|
||||
|
||||
|
||||
| XPath | Description |
|
||||
| -------------- | ----------- |
|
||||
| `ceiling(number)` | evaluates a decimal number and returns the smallest integer greater than or equal to the decimal number.
|
||||
| `floor(number)` | evaluates a decimal number and returns the largest integer less than or equal to the decimal number.
|
||||
| `round(decimal)` | returns a number that is the nearest integer to the given number.
|
||||
| `sum(node-set)` | returns a number that is the sum of the numeric values of each node in a given node-set.
|
||||
|
||||
|
||||
### 11.3. Logic.
|
||||
|
||||
| XPath | Description |
|
||||
| -------------- | ----------- |
|
||||
| `true()` | returns a boolean value of true.
|
||||
| `false()` | returns boolean false.
|
||||
| `not(expression)` | evaluates a boolean expression and returns the opposite value.
|
||||
|
||||
|
||||
### 11.4. Node.
|
||||
|
||||
| XPath | Description |
|
||||
| -------------- | ----------- |
|
||||
| `lang(string)` | determines whether the context node matches the given language and returns boolean true or false.
|
||||
| `name([node-set])` | returns a string representing the QName of the first node in a given node-set.
|
||||
| `namespace-uri([node-set])` | returns a string representing the namespace URI of the first node in a given node-set.
|
||||
|
||||
|
||||
### 11.5. Context.
|
||||
|
||||
|
||||
| XPath | Description |
|
||||
| -------------- | ----------- |
|
||||
| `count(node-set)` | counts the number of nodes in a node-set and returns an integer.
|
||||
| `function-available(name)` | determines if a given function is available and returns boolean true or false.
|
||||
| `last()` | returns a number equal to the context size from the expression evaluation context.
|
||||
| `position()` | returns a number equal to the context position from the expression evaluation context.
|
||||
|
||||
|
||||
### 11.6. String.
|
||||
|
||||
|
||||
| XPath | Description |
|
||||
| -------------- | ----------- |
|
||||
| `contains(haystack-string, needle-string)` | determines whether the first argument string contains the second argument string and returns boolean true or false.
|
||||
| `concat(string1, string2 [stringn]*)` | concatenates two or more strings and returns the resulting string.
|
||||
| `normalize-space(string)` | strips leading and trailing white-space from a string, replaces sequences of whitespace characters by a single space, and returns the resulting string.
|
||||
| `starts-with(haystack, needle)` | checks whether the first string starts with the second string and returns true or false.
|
||||
| `string-length([string])` | returns a number equal to the number of characters in a given string.
|
||||
| `substring(string, start [length])` | returns a part of a given string.
|
||||
| `substring-after(haystack, needle)` | returns a string that is the rest of a given string after a given substring.
|
||||
| `substring-before(haystack, needle)` | returns a string that is the rest of a given string before a given substring.
|
||||
| `translate(string, abc, XYZ)` | evaluates a string and a set of characters to translate and returns the translated string.
|
||||
|
||||
|
||||
## 12. XPath Axes.
|
||||
|
||||
| XPath | Description |
|
||||
| -------------- | ----------- |
|
||||
| `ancestor` | indicates all the ancestors of the context node beginning with the parent node and traveling through to the root node.
|
||||
| `ancestor-or-self` | indicates the context node and all of its ancestors, including the root node.
|
||||
| `attribute (@)` | indicates the attributes of the context node. Only elements have attributes. This axis can be abbreviated with the at sign (@).
|
||||
| `child (/)` | indicates the children of the context node. If an XPath expression does not specify an axis, this is understood by default. Since only the root node or element nodes have children, any other use will select nothing.
|
||||
| `descendant (//)` | indicates all of the children of the context node, and all of their children, and so forth. Attribute and namespace nodes are not included - the parent of an attribute node is an element node, but attribute nodes are not the children of their parents.
|
||||
| `descendant-or-self` | indicates the context node and all of its descendants. Attribute and namespace nodes are not included - the parent of an attribute node is an element node, but attribute nodes are not the children of their parents.
|
||||
| `following` | indicates all the nodes that appear after the context node, except any descendant, attribute, and namespace nodes.
|
||||
| `following-sibling` | indicates all the nodes that have the same parent as the context node and appear after the context node in the source document.
|
||||
| `parent(..)` | indicates the single node that is the parent of the context node. It can be abbreviated as two periods (..).
|
||||
| `preceding` | indicates all the nodes that precede the context node in the document except any ancestor, attribute and namespace nodes.
|
||||
| `preceding-sibling` | indicates all the nodes that have the same parent as the context node and appear before the context node in the source document.
|
||||
| `self (.)` | indicates the context node itself. It can be abbreviated as a single period (.).
|
||||
**XPath** navigates an XML or HTML document as a tree. Every step has the form `axis::node-test[predicate]`, and most of it has a short form: `//div[@class="card"]/a/@href` reads "the `href` of every `a` inside any `div` with class `card`".
|
||||
|
||||
> ⚠️ Browsers, Selenium and most scrapers implement **XPath 1.0**. `ends-with()`, `matches()`, `lower-case()` and `replace()` are 2.0+ and only work in tools like `xmllint`, Saxon or `lxml` with an XSLT 2.0 processor. Workarounds for 1.0 are given below.
|
||||
|
||||
## 🧭 Path Syntax
|
||||
|
||||
| XPath | Selects |
|
||||
|------------|---------------------------------------------|
|
||||
| `/` | From the document root — an absolute path |
|
||||
| `//` | Anywhere in the document, at any depth |
|
||||
| `.` | The current (context) node |
|
||||
| `..` | The parent of the context node |
|
||||
| `@attr` | An attribute |
|
||||
| `*` | Any element |
|
||||
| `@*` | Any attribute |
|
||||
| `text()` | The text nodes of an element |
|
||||
| `node()` | Any node at all, text and comments included |
|
||||
| `[n]` | The nth match — **1-based**, not 0-based |
|
||||
| `[last()]` | The last match |
|
||||
| `a \| b` | The union of two expressions |
|
||||
|
||||
```xpath
|
||||
/html/body (: absolute: the body of the page :)
|
||||
//text() (: every text node :)
|
||||
./div/b (: relative to the context node :)
|
||||
(//p)[2] (: the second p in the whole document :)
|
||||
//p[2] (: every p that is the second p of its parent :)
|
||||
```
|
||||
|
||||
## 🏷 Elements & Attributes
|
||||
|
||||
| XPath | Description | CSS |
|
||||
|-----------------------------------------------|-------------------------------------------|----------------|
|
||||
| `//E` | Every `<E>` element | `E` |
|
||||
| `(//E)[2]` | The second `<E>` in the document | — |
|
||||
| `//E[@A]` | `<E>` carrying attribute `A` | `E[A]` |
|
||||
| `//E[@A="t"]` | `<E>` where `A` is exactly `t` | `E[A='t']` |
|
||||
| `//E[contains(@A,"t")]` | `<E>` where `A` contains `t` | `E[A*='t']` |
|
||||
| `//E[starts-with(@A,"t")]` | `<E>` where `A` starts with `t` | `E[A^='t']` |
|
||||
| `//E[substring(@A, string-length(@A)-2)="t"]` | `<E>` where `A` ends with `t` (1.0 trick) | `E[A$='t']` |
|
||||
| `//E[ends-with(@A,"t")]` | The same, XPath 2.0+ only | `E[A$='t']` |
|
||||
| `//E[matches(@A,"r")]` | `A` matches regex `r` (2.0+) | — |
|
||||
| `//E/@A` | The value of attribute `A` | — |
|
||||
| `//*/@A` | Attribute `A` of any element | — |
|
||||
| `//E[@A2="t"]/@A1` | `A1` where `A2` is `t` | — |
|
||||
| `//E1[@id="I1"] \| //E2[@id="I2"]` | Either element | `E1#I1, E2#I2` |
|
||||
| `//E[@id="I1" or @id="I2"]` | `<E>` with either id | `E#I1, E#I2` |
|
||||
|
||||
## 🆔 Id, Name, Class & Language
|
||||
|
||||
| XPath | Description | CSS |
|
||||
|-------------------------------------------------------------------|-----------------------------------------------------------|-------------------------|
|
||||
| `//*[@id="I"]` | The element with id `I` | `#I` |
|
||||
| `//E[@id="I"]` | `<E>` with id `I` | `E#I` |
|
||||
| `//*[@name="N"]` | Any element named `N` | `[name='N']` |
|
||||
| `//*[@name="N"][@value="v"]` | Named `N` with value `v` | `[name='N'][value='v']` |
|
||||
| `//*[contains(concat(" ", normalize-space(@class), " "), " C ")]` | Element with class `C` — the padding avoids matching `C2` | `.C` |
|
||||
| `//E[contains(concat(" ", normalize-space(@class), " "), " C ")]` | `<E>` with class `C` | `E.C` |
|
||||
| `//E[@lang="L" or starts-with(@lang, concat("L","-"))]` | `<E>` in language `L` or a subcode | `E[lang\|='L']` |
|
||||
|
||||
## 📝 Text & Links
|
||||
|
||||
| XPath | Description | CSS |
|
||||
|-----------------------------------|------------------------------------------------------|--------------------|
|
||||
| `//*[.="t"]` | Element whose whole text is exactly `t` | — |
|
||||
| `//E[contains(text(),"t")]` | `<E>` with a text node containing `t` | — |
|
||||
| `//E[contains(.,"t")]` | `<E>` whose text, descendants included, contains `t` | — |
|
||||
| `//E[normalize-space()="t"]` | Exact text, ignoring surrounding whitespace | — |
|
||||
| `//a` | Every link | `a` |
|
||||
| `//a[.="t"]` | The link labelled exactly `t` | — |
|
||||
| `//a[@href="url"]` | The link pointing at `url` | `a[href='url']` |
|
||||
| `//a[.="t"]/@href` | The URL behind the label `t` | — |
|
||||
| `//a[starts-with(@href,"https")]` | Secure links only | `a[href^='https']` |
|
||||
|
||||
## 👪 Parents & Children
|
||||
|
||||
| XPath | Description | CSS |
|
||||
|------------------------------------------------------------------|--------------------------------------------------------|-------------------------|
|
||||
| `//E/*[1]` | First child of `<E>` | `E > *:first-child` |
|
||||
| `//E/*[last()]` | Last child of `<E>` | `E > *:last-child` |
|
||||
| `//E[1]` | Each `<E>` that is the first of its type in its parent | `E:first-of-type` |
|
||||
| `//E[last()]` | Last `<E>` of its type | `E:last-of-type` |
|
||||
| `//E[2]` | Second `<E>` of its type | `E:nth-of-type(2)` |
|
||||
| `//*[2][name()="E"]` | Second child, if it is an `<E>` | `E:nth-child(2)` |
|
||||
| `//E[last()-1]` | Second-to-last `<E>` of its type | `E:nth-last-of-type(2)` |
|
||||
| `//E/..` | The parent of `<E>` | — |
|
||||
| `//div/form/parent::*` | Every element that has a form child | `div:has(> form)` |
|
||||
| `//*[@id="I"]//E` | `<E>` anywhere under id `I` | `#I E` |
|
||||
| `//E[count(*)=0]` | `<E>` with no element children | `E:empty` |
|
||||
| `//E[count(*)=1]` | `<E>` with exactly one child | — |
|
||||
| `//E[count(preceding-sibling::*)+count(following-sibling::*)=0]` | `<E>` that is an only child | `E:only-child` |
|
||||
| `//E[count(../E)=1]` | `<E>` with no `<E>` siblings | `E:only-of-type` |
|
||||
| `//E1[E2 and not(*[not(self::E2)])]` | `<E1>` whose only children are `<E2>` | — |
|
||||
| `//E[position() mod N = M + 1]` | Every Nth element, offset by M | `E:nth-child(Nn+M)` |
|
||||
|
||||
## 👥 Siblings
|
||||
|
||||
| XPath | Description | CSS |
|
||||
|---------------------------------------------|--------------------------------------|---------------|
|
||||
| `//E2/following-sibling::E1` | Every `<E1>` after a sibling `<E2>` | `E2 ~ E1` |
|
||||
| `//E2/following-sibling::*[1][name()="E1"]` | `<E1>` immediately after `<E2>` | `E2 + E1` |
|
||||
| `//E2/following-sibling::*[2][name()="E1"]` | `<E1>` two siblings after `<E2>` | `E2 + * + E1` |
|
||||
| `//E/following-sibling::*[1]` | Whatever follows `<E>` directly | `E + *` |
|
||||
| `//E2/preceding-sibling::E1` | Every `<E1>` before a sibling `<E2>` | — |
|
||||
| `//E2/preceding-sibling::*[1][name()="E1"]` | `<E1>` immediately before `<E2>` | — |
|
||||
| `//E/preceding-sibling::*[1]` | Whatever precedes `<E>` directly | — |
|
||||
|
||||
## 📊 Tables & Form State
|
||||
|
||||
| XPath | Description | CSS |
|
||||
|------------------------------------------------|------------------------------------------|--------------------------------------|
|
||||
| `//*[@id="T"]//tr[3]//td[2]` | Third row, second cell | `#T tr:nth-child(3) td:nth-child(2)` |
|
||||
| `//td[preceding-sibling::td="t"]` | The cell right after the one reading `t` | — |
|
||||
| `//td[preceding-sibling::td[contains(.,"t")]]` | Cells following one containing `t` | `td:has(~ td)` |
|
||||
| `//table[count(tr)=1 and count(tr/td)=2]` | Tables with one row and two columns | — |
|
||||
| `//table/tr[last()]` | The last row of a table | `table tr:last-child` |
|
||||
| `//E[@disabled]` | A disabled control | `E:disabled` |
|
||||
| `//E[not(@disabled)]` | A control that is not disabled | `E:enabled` |
|
||||
| `//*[@checked]` | A checked checkbox or radio | `:checked` |
|
||||
|
||||
## 🧮 Functions
|
||||
|
||||
### Strings
|
||||
|
||||
| Function | Returns |
|
||||
|--------------------------------------|----------------------------------------------------------|
|
||||
| `contains(haystack, needle)` | True when the first string contains the second |
|
||||
| `starts-with(haystack, needle)` | True when it starts with it |
|
||||
| `concat(s1, s2, …)` | The strings joined together |
|
||||
| `normalize-space(s)` | The string with runs of whitespace collapsed and trimmed |
|
||||
| `string-length(s)` | The number of characters |
|
||||
| `substring(s, start [, length])` | A slice — **1-based** |
|
||||
| `substring-before(haystack, needle)` | Everything before the first occurrence |
|
||||
| `substring-after(haystack, needle)` | Everything after it |
|
||||
| `translate(s, from, to)` | Characters mapped one by one — the 1.0 way to fold case |
|
||||
|
||||
### Numbers, logic & nodes
|
||||
|
||||
| Function | Returns |
|
||||
|---------------------------------------------|-----------------------------------------------------------|
|
||||
| `count(node-set)` | How many nodes matched |
|
||||
| `sum(node-set)` | The sum of their numeric values |
|
||||
| `position()` | The index of the context node within its step |
|
||||
| `last()` | The size of the context — so `[last()]` is the final node |
|
||||
| `name([node-set])` | The qualified name of the first node |
|
||||
| `namespace-uri([node-set])` | Its namespace URI |
|
||||
| `lang(s)` | True when the node is in that language |
|
||||
| `not(expr)` | The negation |
|
||||
| `true()` / `false()` | Boolean constants |
|
||||
| `boolean(expr)` / `string(o)` / `number(o)` | Type conversions |
|
||||
| `ceiling(n)` / `floor(n)` / `round(n)` | Rounding |
|
||||
|
||||
> 💡 Case-insensitive matching in XPath 1.0:
|
||||
> `//E[contains(translate(@A,"ABCDEFGHIJKLMNOPQRSTUVWXYZ","abcdefghijklmnopqrstuvwxyz"), "t")]`
|
||||
|
||||
## 🧭 Axes
|
||||
|
||||
| Axis | Short form | Selects |
|
||||
|----------------------|-------------|-----------------------------------------------------------------|
|
||||
| `child` | *(default)* | The direct children of the context node |
|
||||
| `descendant` | `//` | Every node below the context node |
|
||||
| `descendant-or-self` | `//` | The context node and everything below it |
|
||||
| `parent` | `..` | The single parent node |
|
||||
| `ancestor` | — | Parent, grandparent, up to the root |
|
||||
| `ancestor-or-self` | — | The same, including the context node |
|
||||
| `following` | — | Everything after the node in document order, except descendants |
|
||||
| `following-sibling` | — | Later siblings under the same parent |
|
||||
| `preceding` | — | Everything before it, except its ancestors |
|
||||
| `preceding-sibling` | — | Earlier siblings under the same parent |
|
||||
| `attribute` | `@` | The attributes of the node |
|
||||
| `self` | `.` | The context node itself |
|
||||
|
||||
```xpath
|
||||
//td[.="Total"]/ancestor::table (: the table a cell belongs to :)
|
||||
//h2[.="Specs"]/following-sibling::ul[1] (: the list after a heading :)
|
||||
//input[@name="q"]/ancestor-or-self::form (: the form wrapping a field :)
|
||||
```
|
||||
|
||||
## 🧠 Worked Examples
|
||||
|
||||
```xpath
|
||||
//hr[@class="edge" and position()=1]
|
||||
(: the first hr of class 'edge' under each parent :)
|
||||
|
||||
//table[parent::div[@class="pad"] and not(@id)]//a
|
||||
(: links in unnamed tables inside a padded div :)
|
||||
|
||||
/html/body/div/*[preceding-sibling::h4]
|
||||
(: whatever comes after an h4 :)
|
||||
|
||||
//tr/td[font[@class="head" and text()="TRACK"]]
|
||||
(: cells whose font child says TRACK :)
|
||||
|
||||
//*[count(*)=3]
|
||||
(: elements with exactly three children :)
|
||||
|
||||
//rdf:Seq/rdf:li/em:id
|
||||
(: namespaced elements — the prefixes must be bound by the host tool :)
|
||||
|
||||
//var | //acronym
|
||||
(: a union of two node sets :)
|
||||
```
|
||||
|
||||
## 🛠 Running XPath
|
||||
|
||||
```javascript
|
||||
// browser devtools console
|
||||
$x('//a[contains(@href, "docs")]')
|
||||
|
||||
// plain DOM API
|
||||
document.evaluate('//h1', document, null, XPathResult.ANY_TYPE, null).iterateNext()
|
||||
```
|
||||
|
||||
```bash
|
||||
# from the shell, over XML or HTML
|
||||
xmllint --xpath '//book/title/text()' catalog.xml
|
||||
xmllint --html --xpath '//a/@href' page.html 2>/dev/null
|
||||
```
|
||||
|
||||
```python
|
||||
from lxml import html
|
||||
tree = html.parse('page.html')
|
||||
tree.xpath('//div[@class="card"]/a/@href')
|
||||
```
|
||||
|
||||
```python
|
||||
# Selenium and Playwright both accept XPath locators
|
||||
driver.find_element(By.XPATH, '//button[normalize-space()="Save"]')
|
||||
page.locator('xpath=//button[normalize-space()="Save"]').click()
|
||||
```
|
||||
|
||||
## ⚠️ Notes
|
||||
|
||||
- **Indexes start at 1.** `[0]` never matches anything.
|
||||
- **`//E[1]` is per-parent**, `(//E)[1]` is document-wide — the single most common XPath bug.
|
||||
- **Namespaced documents** need prefixes registered with the host library; in XPath 1.0 a common workaround is `//*[local-name()="title"]`.
|
||||
- **`//` is expensive** on large documents; anchor the path when you can.
|
||||
- **Prefer text matching with `normalize-space()`** — HTML is full of stray whitespace and line breaks.
|
||||
|
||||
## 📚 Resources
|
||||
|
||||
- [MDN — XPath](https://developer.mozilla.org/en-US/docs/Web/XML/XPath)
|
||||
- [MDN — XPath functions](https://developer.mozilla.org/en-US/docs/Web/XML/XPath/Reference/Functions)
|
||||
- [XPath 1.0 specification](https://www.w3.org/TR/xpath-10/)
|
||||
- [XPath cheat sheet (alephzarro)](http://xpath.alephzarro.com/content/cheatsheet.html)
|
||||
|
||||
Credit to the original author: https://gist.github.com/jmaccabee/550a0b9fcfdc7e6b170cd34c6ec7bc56
|
||||
@@ -1,67 +1,135 @@
|
||||
Zellij is a **terminal workspace and multiplexer** with panes, tabs, layouts, and plugin support.
|
||||
**Zellij** is a terminal workspace and multiplexer with panes, tabs, layouts, session persistence and WebAssembly plugins. It is modal: a `Ctrl` chord enters a mode, then single keys act inside it.
|
||||
|
||||
> ℹ️ Press `Ctrl+p` for **pane** mode, `Ctrl+t` for **tab** mode, `Ctrl+n` for **resize**, `Ctrl+h` for **move**, `Ctrl+s` for **search/scroll**, `Ctrl+o` for **session**. `Esc` or `Ctrl+c` returns to normal mode.
|
||||
|
||||
## 📦 Installation
|
||||
|
||||
```bash
|
||||
# Install via Cargo
|
||||
# from crates.io
|
||||
cargo install --locked zellij
|
||||
|
||||
# Try without installing
|
||||
# from a package manager
|
||||
brew install zellij
|
||||
|
||||
# try it without installing anything
|
||||
bash <(curl -L https://zellij.dev/launch)
|
||||
```
|
||||
|
||||
## 🚀 Starting & Managing Sessions
|
||||
## 🚀 Sessions
|
||||
|
||||
```bash
|
||||
# Start a new Zellij session
|
||||
# start a session
|
||||
zellij
|
||||
|
||||
# Start with a specific session name
|
||||
zellij --session mysession
|
||||
# start a named session
|
||||
zellij --session <name>
|
||||
|
||||
# Attach to an existing session
|
||||
zellij attach mysession
|
||||
# attach to a session
|
||||
zellij attach <name>
|
||||
|
||||
# List all sessions
|
||||
# attach, creating the session if it does not exist
|
||||
zellij attach --create <name>
|
||||
|
||||
# list sessions
|
||||
zellij list-sessions
|
||||
|
||||
# Kill a session
|
||||
zellij kill-session mysession
|
||||
# kill a session (it stays resurrectable)
|
||||
zellij kill-session <name>
|
||||
|
||||
# kill every session
|
||||
zellij kill-all-sessions
|
||||
|
||||
# delete a session for good
|
||||
zellij delete-session <name>
|
||||
```
|
||||
|
||||
## 🪟 Pane Management
|
||||
## 🪟 Pane Mode — `Ctrl+p`
|
||||
|
||||
| Action | Command / Shortcut |
|
||||
|--------|--------------------|
|
||||
| Split vertically | `Ctrl + p` → `v` |
|
||||
| Split horizontally | `Ctrl + p` → `h` |
|
||||
| Close pane | `Ctrl + p` → `x` |
|
||||
| Move focus | `Ctrl + p` → Arrow keys |
|
||||
| Resize pane | `Ctrl + p` → `r` then arrows |
|
||||
| Toggle floating pane | `Ctrl + p` → `f` |
|
||||
| Toggle stacked panes | `Ctrl + p` → `s` |
|
||||
| Key | Action |
|
||||
|-----------------|-------------------------------------------------|
|
||||
| `n` | New pane |
|
||||
| `d` | Split down (new pane below) |
|
||||
| `r` | Split right (new pane to the right) |
|
||||
| `x` | Close the focused pane |
|
||||
| `f` | Toggle fullscreen for the focused pane |
|
||||
| `w` | Toggle a floating pane |
|
||||
| `e` | Embed a floating pane, or float an embedded one |
|
||||
| `c` | Rename the pane |
|
||||
| `z` | Toggle the pane frames |
|
||||
| `←↑↓→` / `hjkl` | Move the focus |
|
||||
|
||||
## 📑 Tab Management
|
||||
## 📑 Tab Mode — `Ctrl+t`
|
||||
|
||||
| Action | Command / Shortcut |
|
||||
|--------|--------------------|
|
||||
| New tab | `Ctrl + p` → `t` |
|
||||
| Close tab | `Ctrl + p` → `q` |
|
||||
| Rename tab | `Ctrl + p` → `n` |
|
||||
| Move between tabs | `Ctrl + p` → Left/Right arrows |
|
||||
| Key | Action |
|
||||
|--------------|---------------------------------------------|
|
||||
| `n` | New tab |
|
||||
| `x` | Close the tab |
|
||||
| `r` | Rename the tab |
|
||||
| `s` | Toggle sync — type into every pane at once |
|
||||
| `b` | Break the focused pane out into its own tab |
|
||||
| `[` / `]` | Move the pane to the previous / next tab |
|
||||
| `←→` / `1…9` | Switch tabs |
|
||||
|
||||
## 🛠️ Layouts
|
||||
## 📐 Resize & Move
|
||||
|
||||
| Mode | Keys | Action |
|
||||
|-------------------|-----------------|---------------------------------|
|
||||
| Resize — `Ctrl+n` | `←↑↓→` / `hjkl` | Grow the pane in that direction |
|
||||
| Resize — `Ctrl+n` | `+` / `-` | Grow or shrink the pane |
|
||||
| Move — `Ctrl+h` | `←↑↓→` / `hjkl` | Move the pane in the layout |
|
||||
|
||||
## 🔍 Search & Scrollback — `Ctrl+s`
|
||||
|
||||
| Key | Action |
|
||||
|-----------------|---------------------------------------|
|
||||
| `s` | Search the scrollback |
|
||||
| `e` | Open the scrollback in your `$EDITOR` |
|
||||
| `PgUp` / `PgDn` | Scroll a page at a time |
|
||||
| `n` / `p` | Next / previous search hit |
|
||||
| `c` | Clear the search |
|
||||
|
||||
## 🗄 Session Mode — `Ctrl+o`
|
||||
|
||||
| Key | Action |
|
||||
|-----|---------------------------------------------|
|
||||
| `d` | Detach — the session keeps running |
|
||||
| `w` | Session manager (switch, resurrect, delete) |
|
||||
| `p` | Plugin manager |
|
||||
| `c` | Configuration |
|
||||
|
||||
> 💡 `Ctrl+g` locks the interface so every key goes straight to the program in the pane (useful when the app wants `Ctrl+p`). Press `Ctrl+g` again to unlock. `Ctrl+q` quits Zellij.
|
||||
|
||||
## ⚡ Shortcuts Without a Mode
|
||||
|
||||
| Keys | Action |
|
||||
|-------------------------|---------------------------|
|
||||
| `Alt+n` | New pane |
|
||||
| `Alt+←↑↓→` / `Alt+hjkl` | Move the focus |
|
||||
| `Alt+=` / `Alt+-` | Resize the focused pane |
|
||||
| `Alt+[` / `Alt+]` | Cycle through the layouts |
|
||||
| `Alt+f` | Toggle a floating pane |
|
||||
| `Alt+i` / `Alt+o` | Move the tab left / right |
|
||||
|
||||
## 🛠 Layouts
|
||||
|
||||
Layouts are KDL files that describe the panes and tabs a session starts with.
|
||||
|
||||
```bash
|
||||
# Start with a predefined layout
|
||||
zellij --layout path/to/layout.kdl
|
||||
# start with a layout
|
||||
zellij --layout <path>/layout.kdl
|
||||
|
||||
# Example layout file (KDL format)
|
||||
# dump the built-in default to start from
|
||||
zellij setup --dump-layout default > layout.kdl
|
||||
```
|
||||
|
||||
```kdl
|
||||
layout {
|
||||
tab {
|
||||
tab name="editor" {
|
||||
pane split_direction="vertical" {
|
||||
pane
|
||||
pane
|
||||
pane command="cargo" {
|
||||
args "watch" "-x" "test"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -69,38 +137,47 @@ layout {
|
||||
|
||||
## ⚙️ Configuration
|
||||
|
||||
- Config file: `~/.config/zellij/config.kdl`
|
||||
- Common options:
|
||||
```kdl
|
||||
keybinds {
|
||||
normal {
|
||||
bind "Ctrl g" { SwitchToMode "locked"; }
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## 🔌 Plugins
|
||||
|
||||
- Zellij supports **WebAssembly plugins**.
|
||||
- Example usage:
|
||||
```bash
|
||||
zellij --plugin path/to/plugin.wasm
|
||||
```
|
||||
|
||||
## 🧭 Useful Flags
|
||||
The config file is `~/.config/zellij/config.kdl`.
|
||||
|
||||
```bash
|
||||
--session <name> # Name the session
|
||||
--layout <file> # Use a layout file
|
||||
--help # Show help
|
||||
--version # Show version
|
||||
# write the default configuration out to edit it
|
||||
zellij setup --dump-config > ~/.config/zellij/config.kdl
|
||||
|
||||
# check where zellij looks for its files
|
||||
zellij setup --check
|
||||
```
|
||||
|
||||
## 🗂️ Quick Reference
|
||||
```kdl
|
||||
theme "gruvbox-dark"
|
||||
default_shell "fish"
|
||||
pane_frames false
|
||||
|
||||
- **Session** → `zellij`, `attach`, `list-sessions`, `kill-session`
|
||||
- **Pane** → split, move, resize, float, stack
|
||||
- **Tab** → new, close, rename, switch
|
||||
- **Layout** → `--layout file.kdl`
|
||||
- **Config** → `~/.config/zellij/config.kdl`
|
||||
- **Plugins** → `.wasm` modules
|
||||
keybinds {
|
||||
normal {
|
||||
bind "Ctrl g" { SwitchToMode "locked"; }
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## 🔌 Running Commands & Plugins
|
||||
|
||||
```bash
|
||||
# open a new pane running a command
|
||||
zellij run -- htop
|
||||
|
||||
# open a file in a new pane, in your editor
|
||||
zellij edit <file>
|
||||
|
||||
# drive a running session from a script
|
||||
zellij action new-tab --name deploy
|
||||
zellij action write-chars "make deploy"
|
||||
|
||||
# load a WebAssembly plugin
|
||||
zellij --plugin <path>/plugin.wasm
|
||||
```
|
||||
|
||||
## 📚 Resources
|
||||
|
||||
- [Documentation](https://zellij.dev/documentation/)
|
||||
- [Layouts reference](https://zellij.dev/documentation/layouts)
|
||||
- [Configuration reference](https://zellij.dev/documentation/configuration)
|
||||
@@ -1,72 +1,224 @@
|
||||
## 📦 ZFS Pool Basics
|
||||
**OpenZFS** combines the volume manager and the filesystem. A **pool** (`zpool`) is built from **vdevs**, and vdevs are built from disks; **datasets** (`zfs`) are the filesystems and volumes that live inside a pool.
|
||||
|
||||
> ⚠️ Redundancy lives at the **vdev** level, not the pool level. Lose one vdev and you lose the whole pool, however healthy the others are.
|
||||
|
||||
## 📦 Pool Basics
|
||||
|
||||
```bash
|
||||
# Create a new pool
|
||||
zpool create mypool /dev/sdX
|
||||
# create a pool from a single disk (no redundancy)
|
||||
zpool create mypool /dev/disk/by-id/<disk>
|
||||
|
||||
# List all pools
|
||||
# a two-disk mirror, 4K-aligned
|
||||
zpool create -o ashift=12 mypool mirror /dev/disk/by-id/<disk1> /dev/disk/by-id/<disk2>
|
||||
|
||||
# raidz2 across six disks
|
||||
zpool create -o ashift=12 tank raidz2 <disk1> <disk2> <disk3> <disk4> <disk5> <disk6>
|
||||
|
||||
# mount the pool somewhere other than /<poolname>
|
||||
zpool create -m /srv/data mypool <disk>
|
||||
|
||||
# add another vdev (this widens the pool, it does not add redundancy)
|
||||
zpool add mypool mirror <disk3> <disk4>
|
||||
|
||||
# add a hot spare, a read cache (L2ARC) and a separate intent log
|
||||
zpool add mypool spare <disk>
|
||||
zpool add mypool cache <ssd>
|
||||
zpool add mypool log mirror <ssd1> <ssd2>
|
||||
|
||||
# destroy a pool — irreversible
|
||||
zpool destroy mypool
|
||||
```
|
||||
|
||||
## 🔍 Inspecting
|
||||
|
||||
```bash
|
||||
# pools, their size and health
|
||||
zpool list
|
||||
|
||||
# Show detailed pool status
|
||||
# per-vdev breakdown
|
||||
zpool list -v
|
||||
|
||||
# layout, errors and any running scrub or resilver
|
||||
zpool status mypool
|
||||
|
||||
# Add a new vdev to an existing pool
|
||||
zpool add mypool /dev/sdY
|
||||
# only the pools that need attention
|
||||
zpool status -x
|
||||
|
||||
# Destroy a pool (⚠️ irreversible!)
|
||||
zpool destroy mypool
|
||||
# live throughput, refreshed every 5 seconds
|
||||
zpool iostat -v 5
|
||||
|
||||
# Export (unmount) a pool
|
||||
zpool export mypool
|
||||
# every property of the pool
|
||||
zpool get all mypool
|
||||
|
||||
# Import (mount) a pool
|
||||
zpool import mypool
|
||||
|
||||
# Scrub a pool (check for errors)
|
||||
zpool scrub mypool
|
||||
|
||||
# Check scrub status
|
||||
zpool status mypool
|
||||
# what has been done to this pool, ever
|
||||
zpool history mypool
|
||||
```
|
||||
|
||||
## 🛠️ Vdev Types & RAID Equivalents
|
||||
## 🛠 Vdev Types
|
||||
|
||||
| ZFS Vdev Type | Description | RAID Equivalent | Min Disks | Fault Tolerance |
|
||||
|-----------------------|--------------------------------------------------|----------------------|-----------|------------------|
|
||||
| `stripe` | Data striped across disks, no redundancy | RAID 0 | 1 | 0 |
|
||||
| `mirror` | Identical copies on multiple disks | RAID 1 | 2+ | 1 per mirror |
|
||||
| `raidz1` | Single parity | RAID 5 | 3+ | 1 |
|
||||
| `raidz2` | Double parity | RAID 6 | 4+ | 2 |
|
||||
| `raidz3` | Triple parity | RAID 7 (uncommon) | 5+ | 3 |
|
||||
| Vdev type | Description | RAID equivalent | Min. disks | Survives |
|
||||
|-----------|-------------------------------------|---------------------|------------|--------------------------------|
|
||||
| `stripe` | Data spread across disks, no parity | RAID 0 | 1 | nothing |
|
||||
| `mirror` | A full copy on every disk | RAID 1 | 2 | all but one disk |
|
||||
| `raidz1` | Single parity | RAID 5 | 3 | 1 disk |
|
||||
| `raidz2` | Double parity | RAID 6 | 4 | 2 disks |
|
||||
| `raidz3` | Triple parity | — (no common level) | 5 | 3 disks |
|
||||
| `dRAID` | Distributed parity with spare space | — | 6+ | as configured, rebuilds faster |
|
||||
|
||||
> ⚠️ ZFS pools are built from one or more vdevs. If any vdev fails, the entire pool fails—even if other vdevs are healthy.
|
||||
Support vdevs: `cache` (L2ARC read cache), `log` (SLOG, for synchronous writes), `special` (metadata and small blocks), `spare` (hot spare).
|
||||
|
||||
## 📁 Dataset & Filesystem Management
|
||||
## 🩺 Health & Maintenance
|
||||
|
||||
```bash
|
||||
# Create a dataset
|
||||
zfs create mypool/mydataset
|
||||
# verify every block against its checksum
|
||||
zpool scrub mypool
|
||||
|
||||
# List datasets
|
||||
zfs list
|
||||
# stop or pause a running scrub
|
||||
zpool scrub -s mypool
|
||||
zpool scrub -p mypool
|
||||
|
||||
# Set quota on a dataset
|
||||
zfs set quota=10G mypool/mydataset
|
||||
# forget errors that have been dealt with
|
||||
zpool clear mypool
|
||||
|
||||
# Snapshot a dataset
|
||||
zfs snapshot mypool/mydataset@snap1
|
||||
# swap a failing disk for a new one
|
||||
zpool replace mypool <old-disk> <new-disk>
|
||||
|
||||
# Roll back to a snapshot
|
||||
zfs rollback mypool/mydataset@snap1
|
||||
# turn a single disk into a mirror, or break one up
|
||||
zpool attach mypool <existing-disk> <new-disk>
|
||||
zpool detach mypool <disk>
|
||||
|
||||
# Send snapshot to another system
|
||||
zfs send mypool/mydataset@snap1 | ssh user@host zfs receive backup/mirror
|
||||
# take a disk out of service and bring it back
|
||||
zpool offline mypool <disk>
|
||||
zpool online mypool <disk>
|
||||
|
||||
# tell the SSDs which blocks are free
|
||||
zpool trim mypool
|
||||
|
||||
# enable the on-disk features of a newer OpenZFS release
|
||||
zpool upgrade mypool
|
||||
```
|
||||
|
||||
## 🧮 Tips & Best Practices
|
||||
> 💡 Scrub monthly on consumer disks, weekly if the data matters. `zpool status` reports what the last scrub found.
|
||||
|
||||
- ✅ Always use whole disks (not partitions) for ZFS.
|
||||
- ✅ Use ECC RAM if possible—ZFS relies heavily on memory integrity.
|
||||
- ✅ Avoid mixing vdev types in a single pool.
|
||||
- ✅ Use `zpool scrub` regularly to detect and fix silent corruption.
|
||||
- ✅ Remember: redundancy is at the vdev level, not the pool level.
|
||||
## 📤 Import & Export
|
||||
|
||||
```bash
|
||||
# release a pool so it can be moved to another machine
|
||||
zpool export mypool
|
||||
|
||||
# list the pools that could be imported
|
||||
zpool import
|
||||
|
||||
# import by name, preferring the stable device names
|
||||
zpool import -d /dev/disk/by-id mypool
|
||||
|
||||
# import every pool it can find
|
||||
zpool import -a
|
||||
|
||||
# import a pool that was not exported cleanly
|
||||
zpool import -f mypool
|
||||
|
||||
# import under a different name, or under an alternate root
|
||||
zpool import mypool newname
|
||||
zpool import -R /mnt mypool
|
||||
```
|
||||
|
||||
## ⚙️ Pool Properties
|
||||
|
||||
```bash
|
||||
# trim automatically as blocks are freed
|
||||
zpool set autotrim=on mypool
|
||||
|
||||
# grow the pool when the underlying disks grow
|
||||
zpool set autoexpand=on mypool
|
||||
|
||||
# replace a failed disk with a spare automatically
|
||||
zpool set autoreplace=on mypool
|
||||
|
||||
# what to do when the pool faults: wait | continue | panic
|
||||
zpool set failmode=continue mypool
|
||||
|
||||
# read one property
|
||||
zpool get health,capacity,fragmentation mypool
|
||||
```
|
||||
|
||||
## 📁 Datasets & Filesystems
|
||||
|
||||
```bash
|
||||
# create a dataset
|
||||
zfs create mypool/data
|
||||
|
||||
# create a nested dataset, making the parents as needed
|
||||
zfs create -p mypool/data/projects/web
|
||||
|
||||
# create a block device (zvol) for a VM or iSCSI target
|
||||
zfs create -V 50G mypool/vm-disk
|
||||
|
||||
# list datasets, snapshots and volumes
|
||||
zfs list
|
||||
zfs list -t all -r mypool
|
||||
|
||||
# common properties
|
||||
zfs set compression=lz4 mypool/data
|
||||
zfs set quota=100G mypool/data
|
||||
zfs set reservation=10G mypool/data
|
||||
zfs set atime=off mypool/data
|
||||
zfs set recordsize=1M mypool/media
|
||||
|
||||
# read properties, including where they were inherited from
|
||||
zfs get -r compression mypool
|
||||
zfs get all mypool/data
|
||||
|
||||
# mount, unmount, and destroy
|
||||
zfs mount mypool/data
|
||||
zfs unmount mypool/data
|
||||
zfs destroy -r mypool/data
|
||||
```
|
||||
|
||||
## 📸 Snapshots & Replication
|
||||
|
||||
```bash
|
||||
# take a snapshot (instant, and free until the data diverges)
|
||||
zfs snapshot mypool/data@2026-08-22
|
||||
|
||||
# snapshot a dataset and everything under it
|
||||
zfs snapshot -r mypool/data@nightly
|
||||
|
||||
# list snapshots and what they cost
|
||||
zfs list -t snapshot -o name,used,refer
|
||||
|
||||
# go back in time — discards everything written since
|
||||
zfs rollback mypool/data@2026-08-22
|
||||
|
||||
# read a single file out of a snapshot instead
|
||||
ls /mypool/data/.zfs/snapshot/2026-08-22/
|
||||
|
||||
# a writable copy of a snapshot
|
||||
zfs clone mypool/data@2026-08-22 mypool/data-copy
|
||||
|
||||
# send a full snapshot to another machine
|
||||
zfs send mypool/data@snap1 | ssh user@host zfs receive backup/data
|
||||
|
||||
# send only what changed since the previous snapshot
|
||||
zfs send -i mypool/data@snap1 mypool/data@snap2 | ssh user@host zfs receive backup/data
|
||||
|
||||
# delete a snapshot
|
||||
zfs destroy mypool/data@snap1
|
||||
```
|
||||
|
||||
## 🧠 Tips & Best Practices
|
||||
|
||||
- Address disks by `/dev/disk/by-id/...`, never `/dev/sdX` — kernel names move between boots.
|
||||
- Set `ashift=12` at creation time for any modern drive; it cannot be changed afterwards.
|
||||
- Give ZFS whole disks, not partitions, so it can manage the write cache itself.
|
||||
- Keep pools below ~80% full — performance falls off a cliff above that, and fragmentation is permanent.
|
||||
- Use ECC RAM where you can: ZFS trusts what is in memory.
|
||||
- Do not mix vdev types or widths in one pool; the pool is only as good as its weakest vdev.
|
||||
- A snapshot is not a backup — replicate it somewhere else with `zfs send`.
|
||||
|
||||
## 📚 Resources
|
||||
|
||||
- [OpenZFS documentation](https://openzfs.github.io/openzfs-docs/)
|
||||
- [`zpool` manual page](https://openzfs.github.io/openzfs-docs/man/master/8/zpool.8.html)
|
||||
- [`zfs` manual page](https://openzfs.github.io/openzfs-docs/man/master/8/zfs.8.html)
|
||||
- [Workload tuning guide](https://openzfs.github.io/openzfs-docs/Performance%20and%20Tuning/Workload%20Tuning.html)
|
||||
Reference in new issue
Block a user