fix: awk format

Fixes some modifier spacing
This commit is contained in:
FireIsGood committed 2024-05-16 14:39:03 -07:00
1 parent be7fee8a83
commit 65070645dd
1 file changed
+167 -149
+167 -149
View File
@@ -3,46 +3,50 @@ title: Awk
date: 2020-12-31 15:18:34
background: bg-slate-600
tags:
- bash
- text
- script
- bash
- text
- script
categories:
- Linux Command
- Linux Command
intro: |
This is a one page quick reference cheat sheet to the [GNU awk](https://www.gnu.org/software/gawk/manual/gawk.html), which covers commonly used awk expressions and commands.
This is a one page quick reference cheat sheet to the [GNU awk](https://www.gnu.org/software/gawk/manual/gawk.html), which covers commonly used awk expressions and commands.
plugins:
- copyCode
- copyCode
---
Getting Started
---------------
## Getting Started
### Have a try
```shell script {.wrap}
$ awk -F: '{print $1, $NF}' /etc/passwd
```
----
| - | - | - |
|---|---------------|---------------------------|
| | `-F:` | Colon as a separator |
| | `{...}` | Awk program |
| | `print` | Prints the current record |
| | `$1` | First field |
| | `$NF` | Last field |
| | `/etc/passwd` | Input data file |
---
| - | - |
| ------------- | ------------------------- |
| `-F:` | Colon as a separator |
| `{...}` | Awk program |
| `print` | Prints the current record |
| `$1` | First field |
| `$NF` | Last field |
| `/etc/passwd` | Input data file |
{.left-text}
### Awk program
### Awk program
```
BEGIN {<initializations>}
<pattern 1> {<program actions>}
<pattern 2> {<program actions>}
BEGIN {<initializations>}
<pattern 1> {<program actions>}
<pattern 2> {<program actions>}
...
END {< final actions >}
```
#### Example
```
awk '
BEGIN { print "\n>>>Start" }
@@ -51,11 +55,11 @@ awk '
' /etc/passwd
```
### Variables {.row-span-2}
```bash
$1 $2/$(NF-1) $3/$NF
▼ ▼ ▼
▼ ▼ ▼
┌──────┬──────────────────┬───────┐
$0/NR ▶ │ ID │ WEBSITE │ URI │
├──────┼──────────────────┼───────┤
@@ -64,6 +68,7 @@ $0/NR ▶ │ 1 │ cheatsheets.zip │ awk │
$0/NR ▶ │ 2 │ google.com │ 25 │
└──────┴──────────────────┴───────┘
```
---
```
@@ -76,38 +81,38 @@ awk -F: '{print NR, $0}' /etc/passwd
# Second last field
awk -F: '{print $(NF-1)}' /etc/passwd
# Custom string
# Custom string
awk -F: '{print $1 "=" $6}' /etc/passwd
```
See: [Variables](#awk-variables)
### Awk program examples {.col-span-2 .row-span-2}
```
awk 'BEGIN {print "hello world"}' # Prints "hello world"
awk -F: '{print $1}' /etc/passwd # -F: Specify field separator
# /pattern/ Execute actions only for matched pattern
awk -F: '/root/ {print $1}' /etc/passwd
awk -F: '/root/ {print $1}' /etc/passwd
# BEGIN block is executed once at the start
awk -F: 'BEGIN { print "uid"} { print $1 }' /etc/passwd
awk -F: 'BEGIN { print "uid"} { print $1 }' /etc/passwd
# END block is executed once at the end
awk -F: '{print $1} END { print "-done-"}' /etc/passwd
```
### Conditions
```
awk -F: '$3>30 {print $1}' /etc/passwd
```
See: [Conditions](#awk-conditions)
### Generate 1000 spaces
```
awk 'BEGIN{
while (a++ < 1000)
@@ -115,11 +120,11 @@ awk 'BEGIN{
print s
}'
```
See: [Loops](#awk-loops)
### Arrays
```
awk 'BEGIN {
fruits["mango"] = "yellow";
@@ -129,11 +134,11 @@ awk 'BEGIN {
}
}'
```
See: [Arrays](#awk-arrays)
### Functions
```
# => 5
awk 'BEGIN{print length("hello")}'
@@ -142,33 +147,29 @@ awk 'BEGIN{print toupper("hello")}'
# => hel
awk 'BEGIN{print substr("hello", 1, 3)}'
```
See: [Functions](#awk-functions)
Awk Variables
---------
## Awk Variables
### Build-in variables
| - | - |
|----------------|-----------------------------------------------------|
| -------------- | --------------------------------------------------- |
| `$0` | Whole line |
| `$1, $2...$NF` | First, second… last field |
| `NR` | `N`umber of `R`ecords |
| `NF` | `N`umber of `F`ields |
| `OFS` | `O`utput `F`ield `S`eparator <br> _(default " ")_ |
| `OFS` | `O`utput `F`ield `S`eparator <br> _(default " ")_ |
| `FS` | input `F`ield `S`eparator <br> _(default " ")_ |
| `ORS` | `O`utput `R`ecord `S`eparator <br> _(default "\n")_ |
| `RS` | input `R`ecord `S`eparator <br> _(default "\n")_ |
| `FILENAME` | Name of the file |
### Expressions
| - | - |
|---------------------|------------------------------------|
| ------------------- | ---------------------------------- |
| `$1 == "root"` | First field equals root |
| `{print $(NF-1)}` | Second last field |
| `NR!=1{print $0}` | From 2nd record |
@@ -177,7 +178,7 @@ Awk Variables
| `END{print NR}` | Total records |
| `BEGIN{print OFMT}` | Output format |
| `{print NR, $0}` | Line number |
| `{print NR " " $0}` | Line number (tab) |
| `{print NR " " $0}` | Line number (tab) |
| `{$1 = NR; print}` | Replace 1st field with line number |
| `$NF > 4` | Last field > 4 |
| `NR % 2 == 0` | Even records |
@@ -185,70 +186,72 @@ Awk Variables
| `BEGIN{print ARGC}` | Total arguments |
| `ORS=NR%5?",":"\n"` | Concatenate records |
### Examples
Print sum and average
```
awk -F: '{sum += $3}
END { print sum, sum/NR }
' /etc/passwd
```
Printing parameters
```
awk 'BEGIN {
for (i = 1; i < ARGC; i++)
print ARGV[i] }' a b c
```
Output field separator as a comma
```
awk 'BEGIN { FS=":";OFS=","}
{print $1,$2,$3,$4}' /etc/passwd
```
Position of match
```
awk 'BEGIN {
if (match("One Two Three", "Tw"))
print RSTART }'
```
Length of match
```
awk 'BEGIN {
if (match("One Two Three", "re"))
print RLENGTH }'
```
### Environment Variables
| - | - |
|-----------|-----------------------------------------------------------|
| --------- | --------------------------------------------------------- |
| `ARGC` | Number or arguments |
| `ARGV` | Array of arguments |
| `FNR` | `F`ile `N`umber of `R`ecords |
| `OFMT` | Format for numbers <br> _(default "%.6g")_ |
| `OFMT` | Format for numbers <br> _(default "%.6g")_ |
| `RSTART` | Location in the string |
| `RLENGTH` | Length of match |
| `SUBSEP` | Multi-dimensional array separator <br> _(default "\034")_ |
| `ARGIND` | Argument Index |
### GNU awk only
| - | - |
|---------------|-----------------------|
| ------------- | --------------------- |
| `ENVIRON` | Environment variables |
| `IGNORECASE` | Ignore case |
| `CONVFMT` | Conversion format |
| `ERRNO` | System errors |
| `FIELDWIDTHS` | Fixed width fields |
### Defining variable
```
awk -v var1="Hello" -v var2="Wold" '
END {print var1, var2}
@@ -256,80 +259,94 @@ awk -v var1="Hello" -v var2="Wold" '
```
#### Use shell variables
```
awk -v varName="$PWD" '
END {print varName}' </dev/null
```
Awk Operators
---------
## Awk Operators
### Operators
| - | - |
|------------------|-------------|
| ---------------- | ----------- |
| `{print $1}` | First field |
| `$2 == "foo"` | Equals |
| `$2 != "foo"` | Not equals |
| `"foo" in array` | In array |
#### Regular expression
| - | - |
|-----------------|-------------------|
| --------------- | ----------------- |
| `/regex/` | Line matches |
| `!/regex/` | Line not matches |
| `$1 ~ /regex/` | Field matches |
| `$1 !~ /regex/` | Field not matches |
#### More conditions
| - | - |
|------------------------|-----|
| `($2 <= 4 || $3 < 20)` | Or |
| `($1 == 4 && $3 < 20)` | And |
| - | - |
| ------------------------ | --- |
| `($2 <= 4 \|\| $3 < 20)` | Or |
| `($1 == 4 && $3 < 20)` | And |
### Operations
#### Arithmetic operations
- `+`
- `-`
- `*`
- `/`
- `%`
- `++`
- `--`
- `-`
- `*`
- `/`
- `%`
- `++`
- `--`
{.cols-3 .marker-none}
#### Shorthand assignments
- `+=`
- `-=`
- `*=`
- `/=`
- `%=`
- `+=`
- `-=`
- `*=`
- `/=`
- `%=`
{.cols-3 .marker-none}
#### Comparison operators
- `==`
- `!=`
- `<`
- `>`
- `<=`
- `>=`
{.cols-3 .marker-none}
### Examples
```
awk 'BEGIN {
if ("foo" ~ "^fo+$")
print "Fooey!";
}'
```
#### Not match
```
awk 'BEGIN {
if ("boo" !~ "^fo+$")
print "Boo!";
}'
```
#### if in array
```
awk 'BEGIN {
assoc["foo"] = "bar";
@@ -339,14 +356,12 @@ awk 'BEGIN {
}'
```
## Awk Functions
Awk Functions
----------
### Common functions {.col-span-2}
| Function | Description |
|-----------------------|---------------------------------------------------------------------------------|
| --------------------- | ------------------------------------------------------------------------------- |
| `index(s,t)` | Position in string s where string t occurs, 0 if not found |
| `length(s)` | Length of string s (or $0 if no arg) |
| `rand` | Random number between 0 and 1 |
@@ -362,8 +377,8 @@ Awk Functions
| `toupper(s)` | String s to uppercase |
| `getline` | Set $0 to next input record from current input file. |
### User defined function
```
awk '
# Returns minimum number
@@ -382,7 +397,7 @@ awk '
function main(num1, num2){
result = find_min(num1, num2)
print "Minimum =", result
result = find_max(num1, num2)
print "Maximum =", result
}
@@ -393,15 +408,10 @@ awk '
'
```
Awk Arrays
---------
## Awk Arrays
### Array with index
```
awk 'BEGIN {
arr[0] = "foo";
@@ -413,6 +423,7 @@ awk 'BEGIN {
```
### Array with key
```
awk 'BEGIN {
assoc["foo"] = "bar";
@@ -422,8 +433,8 @@ awk 'BEGIN {
}'
```
### Array with split
```
awk 'BEGIN {
split("foo:bar:baz", arr, ":");
@@ -433,6 +444,7 @@ awk 'BEGIN {
```
### Array with asort
```
awk 'BEGIN {
arr[0] = 3
@@ -444,9 +456,8 @@ awk 'BEGIN {
}'
```
### Multi-dimensional
```
awk 'BEGIN {
multidim[0,0] = "foo";
@@ -457,24 +468,23 @@ awk 'BEGIN {
```
### Multi-dimensional iteration
```
awk 'BEGIN {
array[1,2]=3;
array[2,3]=5;
for (comb in array) {
split(comb,sep,SUBSEP);
print sep[1], sep[2],
print sep[1], sep[2],
array[sep[1],sep[2]]
}
}'
```
Awk Conditions
----------
## Awk Conditions
### if-else statement
```
awk -v count=2 'BEGIN {
if (count == 1)
@@ -483,15 +493,17 @@ awk -v count=2 'BEGIN {
print "Huh?";
}'
```
#### Ternary operator
```
awk -v count=2 'BEGIN {
print (count==1) ? "Yes" : "Huh?";
}'
```
### Exists
```
awk 'BEGIN {
assoc["foo"] = "bar";
@@ -500,7 +512,9 @@ awk 'BEGIN {
print "Fooey!";
}'
```
#### Not exists
```
awk 'BEGIN {
assoc["foo"] = "bar";
@@ -510,8 +524,8 @@ awk 'BEGIN {
}'
```
### switch
```
awk -F: '{
switch (NR * 2 + 1) {
@@ -519,31 +533,32 @@ awk -F: '{
case "11":
print NR - 1
break
case /2[[:digit:]]+/:
print NR
default:
print NR + 1
case -1:
print NR * -1
}
}' /etc/passwd
```
Awk Loops
----------
## Awk Loops
### for...i
```
awk 'BEGIN {
for (i = 0; i < 10; i++)
print "i=" i;
}'
```
#### Powers of two between 1 and 100
#### Powers of two between 1 and 100
```
awk 'BEGIN {
for (i = 1; i <= 100; i *= 2)
@@ -551,8 +566,8 @@ awk 'BEGIN {
}'
```
### for...in
```
awk 'BEGIN {
assoc["key1"] = "val1"
@@ -561,7 +576,9 @@ awk 'BEGIN {
print assoc[key];
}'
```
#### Arguments
```
awk 'BEGIN {
for (argnum in ARGV)
@@ -569,10 +586,10 @@ awk 'BEGIN {
}' a b c
```
### Examples {.row-span-3}
#### Reverse records
```
awk -F: '{ x[NR] = $0 }
END {
@@ -583,6 +600,7 @@ awk -F: '{ x[NR] = $0 }
```
#### Reverse fields
```
awk -F: '{
for (i = NF; i > 0; i--)
@@ -592,6 +610,7 @@ awk -F: '{
```
#### Sum by record
```
awk -F: '{
s=0;
@@ -601,8 +620,8 @@ awk -F: '{
}' /etc/passwd
```
#### Sum whole file
```
awk -F: '
{for (i = 1; i <= NF; i++)
@@ -612,9 +631,8 @@ awk -F: '
' /etc/passwd
```
### while {.row-span-2}
```
awk 'BEGIN {
while (a < 10) {
@@ -623,7 +641,9 @@ awk 'BEGIN {
}
}'
```
#### do...while
```
awk '{
i = 1
@@ -634,9 +654,8 @@ awk '{
}' /etc/passwd
```
### Break
```
awk 'BEGIN {
break_num = 5
@@ -648,9 +667,8 @@ awk 'BEGIN {
}'
```
### Continue
```
awk 'BEGIN {
for (x = 0; x <= 10; x++) {
@@ -662,28 +680,30 @@ awk 'BEGIN {
}'
```
Awk Formatted Printing
---------
## Awk Formatted Printing
### Usage
#### Right align
```
awk 'BEGIN{printf "|%10s|\n", "hello"}'
| hello|
```
#### Left align
```
awk 'BEGIN{printf "|%-10s|\n", "hello"}'
|hello |
```
### Common specifiers
### Common specifiers
| Character | Description |
|---------------|-----------------------|
| ------------- | --------------------- |
| `c` | ASCII character |
| `d` | Decimal integer |
| `e`, `E`, `f` | Floating-point format |
@@ -691,25 +711,24 @@ awk 'BEGIN{printf "|%-10s|\n", "hello"}'
| `s` | String |
| `%` | Literal % |
### Space
```
awk -F: '{
printf "%-10s %s\n", $1, $(NF-1)
}' /etc/passwd | head -n 3
```
Outputs
```shell script
root /root
bin /bin
daemon /sbin
```
### Header
```
awk -F: 'BEGIN {
printf "%-10s %s\n", "User", "Home"
@@ -717,7 +736,9 @@ awk -F: 'BEGIN {
{ printf "%-10s %s\n", $1, $(NF-1) }
' /etc/passwd | head -n 5
```
Outputs
```
User Home
---- ----
@@ -726,12 +747,10 @@ bin /bin
daemon /sbin
```
Miscellaneous
-------------
## Miscellaneous
### Regex Metacharacters
- `\`
- `^`
- `$`
@@ -744,11 +763,13 @@ Miscellaneous
- `*`
- `+`
- `?`
{.cols-3 .marker-none}
### Escape Sequences
| - | - |
|------|---------------------|
| ---- | ------------------- |
| `\b` | Backspace |
| `\f` | Form feed |
| `\n` | Newline (line feed) |
@@ -756,8 +777,8 @@ Miscellaneous
| `\t` | Horizontal tab |
| `\v` | Vertical tab |
### Run script
```shell script
$ cat demo.awk
#!/usr/bin/awk -f
@@ -768,11 +789,8 @@ $ awk -f demo.awk /etc/passwd
69
```
## Also see
Also see
--------
- [The GNU Awk User's Guide](https://www-zeuthen.desy.de/dv/documentation/unixguide/infohtml/gawk/gawk.html) _(www-zeuthen.desy.de)_
- [The GNU Awk User's Guide](https://www-zeuthen.desy.de/dv/documentation/unixguide/infohtml/gawk/gawk.html)
_(www-zeuthen.desy.de)_
- [AWK cheatsheet](https://gist.github.com/Rafe/3102414) _(gist.github.com)_