From 65070645dd430478a7c55919d4f8faf78ff41f11 Mon Sep 17 00:00:00 2001 From: FireIsGood Date: Thu, 16 May 2024 02:59:56 -0700 Subject: [PATCH] fix: awk format Fixes some modifier spacing --- source/_posts/awk.md | 316 +++++++++++++++++++++++-------------------- 1 file changed, 167 insertions(+), 149 deletions(-) diff --git a/source/_posts/awk.md b/source/_posts/awk.md index 702e72f..8f8a9c6 100644 --- a/source/_posts/awk.md +++ b/source/_posts/awk.md @@ -3,46 +3,50 @@ title: Awk date: 2020-12-31 15:18:34 background: bg-slate-600 tags: - - bash - - text - - script + - bash + - text + - script categories: - - Linux Command + - Linux Command intro: | - This is a one page quick reference cheat sheet to the [GNU awk](https://www.gnu.org/software/gawk/manual/gawk.html), which covers commonly used awk expressions and commands. + This is a one page quick reference cheat sheet to the [GNU awk](https://www.gnu.org/software/gawk/manual/gawk.html), which covers commonly used awk expressions and commands. plugins: - - copyCode + - copyCode --- -Getting Started ---------------- +## Getting Started ### Have a try + ```shell script {.wrap} $ awk -F: '{print $1, $NF}' /etc/passwd ``` ----- -| - | - | - | -|---|---------------|---------------------------| -| | `-F:` | Colon as a separator | -| | `{...}` | Awk program | -| | `print` | Prints the current record | -| | `$1` | First field | -| | `$NF` | Last field | -| | `/etc/passwd` | Input data file | + +--- + +| - | - | +| ------------- | ------------------------- | +| `-F:` | Colon as a separator | +| `{...}` | Awk program | +| `print` | Prints the current record | +| `$1` | First field | +| `$NF` | Last field | +| `/etc/passwd` | Input data file | + {.left-text} +### Awk program - -### Awk program ``` -BEGIN {} - {} - {} +BEGIN {} + {} + {} ... END {< final actions >} ``` + #### Example + ``` awk ' BEGIN { print "\n>>>Start" } @@ -51,11 +55,11 @@ awk ' ' /etc/passwd ``` - ### Variables {.row-span-2} + ```bash $1 $2/$(NF-1) $3/$NF - ▼ ▼ ▼ + ▼ ▼ ▼ ┌──────┬──────────────────┬───────┐ $0/NR ▶ │ ID │ WEBSITE │ URI │ ├──────┼──────────────────┼───────┤ @@ -64,6 +68,7 @@ $0/NR ▶ │ 1 │ cheatsheets.zip │ awk │ $0/NR ▶ │ 2 │ google.com │ 25 │ └──────┴──────────────────┴───────┘ ``` + --- ``` @@ -76,38 +81,38 @@ awk -F: '{print NR, $0}' /etc/passwd # Second last field awk -F: '{print $(NF-1)}' /etc/passwd -# Custom string +# Custom string awk -F: '{print $1 "=" $6}' /etc/passwd ``` + See: [Variables](#awk-variables) - - - ### Awk program examples {.col-span-2 .row-span-2} + ``` awk 'BEGIN {print "hello world"}' # Prints "hello world" awk -F: '{print $1}' /etc/passwd # -F: Specify field separator # /pattern/ Execute actions only for matched pattern -awk -F: '/root/ {print $1}' /etc/passwd +awk -F: '/root/ {print $1}' /etc/passwd # BEGIN block is executed once at the start -awk -F: 'BEGIN { print "uid"} { print $1 }' /etc/passwd +awk -F: 'BEGIN { print "uid"} { print $1 }' /etc/passwd # END block is executed once at the end awk -F: '{print $1} END { print "-done-"}' /etc/passwd ``` - ### Conditions + ``` awk -F: '$3>30 {print $1}' /etc/passwd ``` + See: [Conditions](#awk-conditions) - ### Generate 1000 spaces + ``` awk 'BEGIN{ while (a++ < 1000) @@ -115,11 +120,11 @@ awk 'BEGIN{ print s }' ``` + See: [Loops](#awk-loops) - - ### Arrays + ``` awk 'BEGIN { fruits["mango"] = "yellow"; @@ -129,11 +134,11 @@ awk 'BEGIN { } }' ``` + See: [Arrays](#awk-arrays) - - ### Functions + ``` # => 5 awk 'BEGIN{print length("hello")}' @@ -142,33 +147,29 @@ awk 'BEGIN{print toupper("hello")}' # => hel awk 'BEGIN{print substr("hello", 1, 3)}' ``` + See: [Functions](#awk-functions) - - - -Awk Variables ---------- - +## Awk Variables ### Build-in variables + | - | - | -|----------------|-----------------------------------------------------| +| -------------- | --------------------------------------------------- | | `$0` | Whole line | | `$1, $2...$NF` | First, second… last field | | `NR` | `N`umber of `R`ecords | | `NF` | `N`umber of `F`ields | -| `OFS` | `O`utput `F`ield `S`eparator
_(default " ")_ | +| `OFS` | `O`utput `F`ield `S`eparator
_(default " ")_ | | `FS` | input `F`ield `S`eparator
_(default " ")_ | | `ORS` | `O`utput `R`ecord `S`eparator
_(default "\n")_ | | `RS` | input `R`ecord `S`eparator
_(default "\n")_ | | `FILENAME` | Name of the file | - - ### Expressions + | - | - | -|---------------------|------------------------------------| +| ------------------- | ---------------------------------- | | `$1 == "root"` | First field equals root | | `{print $(NF-1)}` | Second last field | | `NR!=1{print $0}` | From 2nd record | @@ -177,7 +178,7 @@ Awk Variables | `END{print NR}` | Total records | | `BEGIN{print OFMT}` | Output format | | `{print NR, $0}` | Line number | -| `{print NR " " $0}` | Line number (tab) | +| `{print NR " " $0}` | Line number (tab) | | `{$1 = NR; print}` | Replace 1st field with line number | | `$NF > 4` | Last field > 4 | | `NR % 2 == 0` | Even records | @@ -185,70 +186,72 @@ Awk Variables | `BEGIN{print ARGC}` | Total arguments | | `ORS=NR%5?",":"\n"` | Concatenate records | - - - ### Examples + Print sum and average + ``` awk -F: '{sum += $3} END { print sum, sum/NR } ' /etc/passwd ``` + Printing parameters + ``` awk 'BEGIN { for (i = 1; i < ARGC; i++) print ARGV[i] }' a b c ``` + Output field separator as a comma + ``` awk 'BEGIN { FS=":";OFS=","} {print $1,$2,$3,$4}' /etc/passwd ``` + Position of match + ``` awk 'BEGIN { if (match("One Two Three", "Tw")) print RSTART }' ``` + Length of match + ``` awk 'BEGIN { if (match("One Two Three", "re")) print RLENGTH }' ``` - - - - ### Environment Variables + | - | - | -|-----------|-----------------------------------------------------------| +| --------- | --------------------------------------------------------- | | `ARGC` | Number or arguments | | `ARGV` | Array of arguments | | `FNR` | `F`ile `N`umber of `R`ecords | -| `OFMT` | Format for numbers
_(default "%.6g")_ | +| `OFMT` | Format for numbers
_(default "%.6g")_ | | `RSTART` | Location in the string | | `RLENGTH` | Length of match | | `SUBSEP` | Multi-dimensional array separator
_(default "\034")_ | | `ARGIND` | Argument Index | - - ### GNU awk only + | - | - | -|---------------|-----------------------| +| ------------- | --------------------- | | `ENVIRON` | Environment variables | | `IGNORECASE` | Ignore case | | `CONVFMT` | Conversion format | | `ERRNO` | System errors | | `FIELDWIDTHS` | Fixed width fields | - - ### Defining variable + ``` awk -v var1="Hello" -v var2="Wold" ' END {print var1, var2} @@ -256,80 +259,94 @@ awk -v var1="Hello" -v var2="Wold" ' ``` #### Use shell variables + ``` awk -v varName="$PWD" ' END {print varName}' ` - `<=` - `>=` + {.cols-3 .marker-none} - - ### Examples + ``` awk 'BEGIN { if ("foo" ~ "^fo+$") print "Fooey!"; }' ``` + #### Not match + ``` awk 'BEGIN { if ("boo" !~ "^fo+$") print "Boo!"; }' ``` + #### if in array + ``` awk 'BEGIN { assoc["foo"] = "bar"; @@ -339,14 +356,12 @@ awk 'BEGIN { }' ``` +## Awk Functions - - -Awk Functions ----------- ### Common functions {.col-span-2} + | Function | Description | -|-----------------------|---------------------------------------------------------------------------------| +| --------------------- | ------------------------------------------------------------------------------- | | `index(s,t)` | Position in string s where string t occurs, 0 if not found | | `length(s)` | Length of string s (or $0 if no arg) | | `rand` | Random number between 0 and 1 | @@ -362,8 +377,8 @@ Awk Functions | `toupper(s)` | String s to uppercase | | `getline` | Set $0 to next input record from current input file. | - ### User defined function + ``` awk ' # Returns minimum number @@ -382,7 +397,7 @@ awk ' function main(num1, num2){ result = find_min(num1, num2) print "Minimum =", result - + result = find_max(num1, num2) print "Maximum =", result } @@ -393,15 +408,10 @@ awk ' ' ``` - - - -Awk Arrays ---------- - - +## Awk Arrays ### Array with index + ``` awk 'BEGIN { arr[0] = "foo"; @@ -413,6 +423,7 @@ awk 'BEGIN { ``` ### Array with key + ``` awk 'BEGIN { assoc["foo"] = "bar"; @@ -422,8 +433,8 @@ awk 'BEGIN { }' ``` - ### Array with split + ``` awk 'BEGIN { split("foo:bar:baz", arr, ":"); @@ -433,6 +444,7 @@ awk 'BEGIN { ``` ### Array with asort + ``` awk 'BEGIN { arr[0] = 3 @@ -444,9 +456,8 @@ awk 'BEGIN { }' ``` - - ### Multi-dimensional + ``` awk 'BEGIN { multidim[0,0] = "foo"; @@ -457,24 +468,23 @@ awk 'BEGIN { ``` ### Multi-dimensional iteration + ``` awk 'BEGIN { array[1,2]=3; array[2,3]=5; for (comb in array) { split(comb,sep,SUBSEP); - print sep[1], sep[2], + print sep[1], sep[2], array[sep[1],sep[2]] } }' ``` - - -Awk Conditions ----------- +## Awk Conditions ### if-else statement + ``` awk -v count=2 'BEGIN { if (count == 1) @@ -483,15 +493,17 @@ awk -v count=2 'BEGIN { print "Huh?"; }' ``` + #### Ternary operator + ``` awk -v count=2 'BEGIN { print (count==1) ? "Yes" : "Huh?"; }' ``` - ### Exists + ``` awk 'BEGIN { assoc["foo"] = "bar"; @@ -500,7 +512,9 @@ awk 'BEGIN { print "Fooey!"; }' ``` + #### Not exists + ``` awk 'BEGIN { assoc["foo"] = "bar"; @@ -510,8 +524,8 @@ awk 'BEGIN { }' ``` - ### switch + ``` awk -F: '{ switch (NR * 2 + 1) { @@ -519,31 +533,32 @@ awk -F: '{ case "11": print NR - 1 break - + case /2[[:digit:]]+/: print NR - + default: print NR + 1 - + case -1: print NR * -1 } }' /etc/passwd ``` - -Awk Loops ----------- +## Awk Loops ### for...i + ``` awk 'BEGIN { for (i = 0; i < 10; i++) print "i=" i; }' ``` -#### Powers of two between 1 and 100 + +#### Powers of two between 1 and 100 + ``` awk 'BEGIN { for (i = 1; i <= 100; i *= 2) @@ -551,8 +566,8 @@ awk 'BEGIN { }' ``` - ### for...in + ``` awk 'BEGIN { assoc["key1"] = "val1" @@ -561,7 +576,9 @@ awk 'BEGIN { print assoc[key]; }' ``` + #### Arguments + ``` awk 'BEGIN { for (argnum in ARGV) @@ -569,10 +586,10 @@ awk 'BEGIN { }' a b c ``` - - ### Examples {.row-span-3} + #### Reverse records + ``` awk -F: '{ x[NR] = $0 } END { @@ -583,6 +600,7 @@ awk -F: '{ x[NR] = $0 } ``` #### Reverse fields + ``` awk -F: '{ for (i = NF; i > 0; i--) @@ -592,6 +610,7 @@ awk -F: '{ ``` #### Sum by record + ``` awk -F: '{ s=0; @@ -601,8 +620,8 @@ awk -F: '{ }' /etc/passwd ``` - #### Sum whole file + ``` awk -F: ' {for (i = 1; i <= NF; i++) @@ -612,9 +631,8 @@ awk -F: ' ' /etc/passwd ``` - - ### while {.row-span-2} + ``` awk 'BEGIN { while (a < 10) { @@ -623,7 +641,9 @@ awk 'BEGIN { } }' ``` + #### do...while + ``` awk '{ i = 1 @@ -634,9 +654,8 @@ awk '{ }' /etc/passwd ``` - - ### Break + ``` awk 'BEGIN { break_num = 5 @@ -648,9 +667,8 @@ awk 'BEGIN { }' ``` - - ### Continue + ``` awk 'BEGIN { for (x = 0; x <= 10; x++) { @@ -662,28 +680,30 @@ awk 'BEGIN { }' ``` - - -Awk Formatted Printing ---------- +## Awk Formatted Printing ### Usage + #### Right align + ``` awk 'BEGIN{printf "|%10s|\n", "hello"}' | hello| ``` + #### Left align + ``` awk 'BEGIN{printf "|%-10s|\n", "hello"}' |hello | ``` -### Common specifiers +### Common specifiers + | Character | Description | -|---------------|-----------------------| +| ------------- | --------------------- | | `c` | ASCII character | | `d` | Decimal integer | | `e`, `E`, `f` | Floating-point format | @@ -691,25 +711,24 @@ awk 'BEGIN{printf "|%-10s|\n", "hello"}' | `s` | String | | `%` | Literal % | - - - ### Space + ``` awk -F: '{ printf "%-10s %s\n", $1, $(NF-1) }' /etc/passwd | head -n 3 ``` + Outputs + ```shell script root /root bin /bin daemon /sbin ``` - - ### Header + ``` awk -F: 'BEGIN { printf "%-10s %s\n", "User", "Home" @@ -717,7 +736,9 @@ awk -F: 'BEGIN { { printf "%-10s %s\n", $1, $(NF-1) } ' /etc/passwd | head -n 5 ``` + Outputs + ``` User Home ---- ---- @@ -726,12 +747,10 @@ bin /bin daemon /sbin ``` - - -Miscellaneous -------------- +## Miscellaneous ### Regex Metacharacters + - `\` - `^` - `$` @@ -744,11 +763,13 @@ Miscellaneous - `*` - `+` - `?` + {.cols-3 .marker-none} ### Escape Sequences + | - | - | -|------|---------------------| +| ---- | ------------------- | | `\b` | Backspace | | `\f` | Form feed | | `\n` | Newline (line feed) | @@ -756,8 +777,8 @@ Miscellaneous | `\t` | Horizontal tab | | `\v` | Vertical tab | - ### Run script + ```shell script $ cat demo.awk #!/usr/bin/awk -f @@ -768,11 +789,8 @@ $ awk -f demo.awk /etc/passwd 69 ``` +## Also see -Also see --------- - -- [The GNU Awk User's Guide](https://www-zeuthen.desy.de/dv/documentation/unixguide/infohtml/gawk/gawk.html) _(www-zeuthen.desy.de)_ +- [The GNU Awk User's Guide](https://www-zeuthen.desy.de/dv/documentation/unixguide/infohtml/gawk/gawk.html) + _(www-zeuthen.desy.de)_ - [AWK cheatsheet](https://gist.github.com/Rafe/3102414) _(gist.github.com)_ - -