AWK Count Occurrences: Lines, Words, Characters & Values

Dev Talk

6 min read

·

5 days ago

--

Listen

Share

Press enter or click to view image in full size

“How many times does this appear?” can mean four different things: how many lines contain it, how many times it appears in total, how many times each value shows up in a column, or how many distinct values there are. AWK handles all four with a few short commands. This guide shows each one with sample input and the exact output.

The quick answers

awk '/error/ {c++} END {print c+0}' file.txt                 # lines containing "error"
awk '{n += gsub(/error/, "&")} END {print n+0}' file.txt     # every occurrence of "error"
awk '{c[$1]++} END {for (k in c) print c[k], k}' file.txt    # how often each value of column 1 appears

The first counts lines, the second counts every occurrence even when one line has several, and the third counts per value. If AWK itself is new to you, start with the beginner’s guide.

Sample files

cat > words.txt <<'EOF'
error: disk full
ok
Error: timeout, error again
warning: low memory
error error error
ERROR: failed
EOF

cat > sales.csv <<'EOF'
date,region,product,qty,price
2026-10-01,north,pen,10,1.50
2026-10-01,south,book,3,12.00
2026-10-02,north,book,2,12.00
2026-10-02,east,pen,25,1.50
2026-10-03,south,pen,5,1.50
EOF

cat > access.log <<'EOF'
10.0.0.1 - - [24/Oct/2026:10:00:01 +0000] "GET /index.php HTTP/1.1" 200 512
10.0.0.2 - - [24/Oct/2026:10:00:02 +0000] "GET /about HTTP/1.1" 404 128
10.0.0.1 - - [24/Oct/2026:10:00:03 +0000] "POST /login HTTP/1.1" 302 0
10.0.0.3 - - [24/Oct/2026:10:00:04 +0000] "GET /index.php HTTP/1.1" 200 512
10.0.0.2 - - [24/Oct/2026:10:00:05 +0000] "GET /missing HTTP/1.1" 404 128
10.0.0.1 - - [24/Oct/2026:10:00:06 +0000] "GET /index.php HTTP/1.1" 500 64
EOF

Count lines that match

awk '/error/ {c++} END {print c+0}' words.txt 
# 3

Each matching line adds one to c, and the END block prints the total. The +0 matters. If nothing matches, c was never set and plain print c prints an empty line instead of a number. With c+0 you get 0.

Only lowercase error matched here, so the lines with Error and ERROR were not counted. The case-insensitive version is below.

grep -c error words.txt gives the same 3. AWK earns its place when the count needs a condition on a column. In the sample web server log, the status code is the second to last field, so the number of error responses (status 400 or higher) is:

awk '$(NF-1) >= 400 {c++} END {print c+0}' access.log 
# 3

NF is the number of fields, so $(NF-1) is the one before the last. The parentheses are required.

Count every occurrence, not just lines

grep -c and the first command above count lines. The line error error error counts once. To count every occurrence, use the return value of gsub(), which is the number of replacements it made:

awk '{t = $0; n += gsub(/error/, "&", t)} END {print n}' words.txt 
# 5

Replacing a match with & means "put back the text that matched", so nothing changes, but you still get the count. Working on a copy ( t) leaves the original line untouched. The same count from the shell is grep -o error words.txt | wc -l, which also gives 5.

To see the count per line:

awk '{n = gsub(/error/, "&"); print NR": "n}' words.txt
# 1: 1
# 2: 0
# 3: 1
# 4: 0
# 5: 3
# 6: 0

Ignore case

Lowercase a copy of the line first. This works in every awk:

awk '{t = tolower($0); n += gsub(/error/, "&", t)} END {print n}' words.txt
# 7

gawk also has an IGNORECASE variable, but mawk silently ignores it, so tolower() is the safer habit.

Whole words only

A regex match also finds error inside errors or error:. To count only fields that are exactly the word, compare the fields:

awk '{for (i = 1; i <= NF; i++) if ($i == "error") n++} END {print n+0}' words.txt
# 4

This gives 4, not 5, because error: on the first line has a colon attached and is not equal to error.

Count one character

echo 'banana' | awk '{print gsub(/a/, "&")}' 
# 3

Two traps. A quoted string is a regular expression, so gsub(".", ...) matches every character; on a.b.c it returned 5, while gsub(/\./, ...) returned 2. And matches do not overlap: gsub(/aa/, ...) on aaaa returned 2, not 3.

Count how often each value appears

This is the most useful pattern. Use the value as an array key and add one every time you see it:

awk -F, 'NR>1 {c[$2]++} END {for (k in c) print k, c[k]}' sales.csv | sort
# east 1
# north 2
# south 2

c[$2]++ reads as "increase the counter for this region". A key that has not been seen yet starts at zero, so there is no setup. NR>1 skips the header, and -F, sets the comma as the separator (see AWK field separators).

The order of for (k in c) is not defined and differs between awks, so pipe the result to sort as above.

Sort by count, highest first

Print the count first, then sort numerically in reverse:

awk -F, 'NR>1 {c[$3]++} END {for (k in c) print c[k], k}' sales.csv | sort -k1,1nr -k2
# 3 pen
# 2 book

The second sort key, -k2, puts names with equal counts in a stable order.

Top visitors in a web server log

awk '{c[$1]++} END {for (k in c) print c[k], k}' access.log | sort -k1,1nr -k2
# 3 10.0.0.1
# 2 10.0.0.2
# 1 10.0.0.3

Add | head -10 for a top 10. Change $1 to $7 to count requested paths instead of IP addresses, or to $(NF-1) to count status codes:

awk '{c[$(NF-1)]++} END {for (k in c) print k, c[k]}' access.log | sort
# 200 2
# 302 1
# 404 2
# 500 1

The column numbers assume the common access log layout shown in the sample, where the quoted request is split by spaces into method, path and protocol. If your log format differs, check with awk '{print $7}' access.log | head first.

Add percentages

awk -F, 'NR>1 {c[$2]++; t++} END {for (k in c) printf "%-6s %d %.0f%%\n", k, c[k], c[k]/t*100}' sales.csv | sort
# east   1 20%
# north  2 40%
# south  2 40%

Count distinct values

To know how many different regions exist, not how often each appears:

awk -F, 'NR>1 && !seen[$2]++ {n++} END {print n}' sales.csv
# 3

awk -F, 'NR>1 {s[$2]} END {print length(s)}' sales.csv
# 3

The first counts a value the first time it is seen. The second only touches each key, then asks for the size of the array. length(array) works in gawk and mawk.

Count lines and words

awk 'END {print NR}' words.txt          # 6   (lines)
awk '{w += NF} END {print w}' words.txt # 16  (words)
awk '!NF {c++} END {print c+0}' file    # blank lines

These match wc -l and wc -w. NR is the line number of the last line read, and NF is the field count of each line.

Count words across a file

awk '{for (i = 1; i <= NF; i++) {w = tolower($i); gsub(/[^a-z]/, "", w); if (w != "") f[w]++}}
     END {for (k in f) print f[k], k}' words.txt | sort -k1,1nr -k2 | head -4
# 7 error
# 1 again
# 1 disk
# 1 failed

For each word, lowercase it, strip everything that is not a letter so error: and Error both become error, and count it. Because of the [^a-z] filter this removes digits and non-English letters too, so adjust the pattern for your text.

Use a shell variable as the search term

awk -v w=error '$0 ~ w {c++} END {print c+0}' words.txt 
# 3

With ~ the variable is used as a regular expression, so special characters such as . keep their regex meaning. Do not put the shell variable inside the program text; pass it with -v.

Common problems

• A blank line instead of 0. You printed a counter that was never set. Print c+0.
• The count is lower than expected. Counting lines (c++ on a match) and counting occurrences (gsub) give different totals when a line has the term more than once.
• Header row counted. Add NR>1 to the rule.
• Case variants counted separately. Error, error and ERROR are different keys. Lowercase the key with c[tolower($1)]++.
• Trailing spaces or Windows line endings. A value with a hidden \r or a trailing space is a different key from the same value without it. Clean it first, for example with sub(/\r$/, "").
• Output order changes between machines. It comes from the for-in loop. Sort the output.

Originally published at https://dev-talk.com on October 30, 2026.