How to count occurrences of text in a file? The Next CEO of Stack OverflowCount duplicated words in a text fileuniq --count command is yields incorrect result?How to compare two (vague) file lists and print the duplicates?How to count occurrences of each character?How do I count text lines?How long it will take to sort uniq a 62GB file?How does uniq work?Count number of data points in fileWrong sorting a text fileHow to count number of partial occurrences of a string in a file

Customer Requests (Sometimes) Drive Me Bonkers!

WOW air has ceased operation, can I get my tickets refunded?

Is a stroke of luck acceptable after a series of unfavorable events?

How should I support this large drywall patch?

How do we know the LHC results are robust?

What makes a siege story/plot interesting?

How do scammers retract money, while you can’t?

Why here is plural "We went to the movies last night."

Can the Reverse Gravity spell affect the Meteor Swarm spell?

Is it a good idea to use COLUMN AS (left([Another_Column],(4)) instead of LEFT in the select?

If the heap is initialized for security, then why is the stack uninitialized?

Horror movie/show or scene where a horse creature opens its mouth really wide and devours a man in a stables

If I blow insulation everywhere in my attic except the door trap, will heat escape through it?

Why does standard notation not preserve intervals (visually)

Where to find order of arguments for default functions

Does the Brexit deal have to be agreed by both Houses?

Are there languages with no euphemisms?

How can I open an app using Terminal?

How long to clear the 'suck zone' of a turbofan after start is initiated?

% symbol leads to superlong (forever?) compilations

Only print output after finding pattern

What's the point of interval inversion?

Fastest way to shutdown Ubuntu Mate 18.10

How easy is it to start Magic from scratch?



How to count occurrences of text in a file?



The Next CEO of Stack OverflowCount duplicated words in a text fileuniq --count command is yields incorrect result?How to compare two (vague) file lists and print the duplicates?How to count occurrences of each character?How do I count text lines?How long it will take to sort uniq a 62GB file?How does uniq work?Count number of data points in fileWrong sorting a text fileHow to count number of partial occurrences of a string in a file










5

















I have a log file sorted by IP addresses,
I want to find the number of occurrences of each unique IP address.
How can I do this with bash? Possibly listing the number of occurrences next to an ip, such as:



5.135.134.16 count: 5
13.57.220.172: count 30
18.206.226 count:2


and so on.



Here’s a sample of the log:



5.135.134.16 - - [23/Mar/2019:08:42:54 -0400] "GET /wp-login.php HTTP/1.1" 200 2988 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:62.0) Gecko/20100101 Firefox/62.0"
5.135.134.16 - - [23/Mar/2019:08:42:55 -0400] "GET /wp-login.php HTTP/1.1" 200 2988 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:62.0) Gecko/20100101 Firefox/62.0"
5.135.134.16 - - [23/Mar/2019:08:42:55 -0400] "POST /wp-login.php HTTP/1.1" 200 3836 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:62.0) Gecko/20100101 Firefox/62.0"
5.135.134.16 - - [23/Mar/2019:08:42:55 -0400] "POST /wp-login.php HTTP/1.1" 200 3988 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:62.0) Gecko/20100101 Firefox/62.0"
5.135.134.16 - - [23/Mar/2019:08:42:56 -0400] "POST /xmlrpc.php HTTP/1.1" 200 413 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:62.0) Gecko/20100101 Firefox/62.0"
13.57.220.172 - - [23/Mar/2019:11:01:05 -0400] "GET /wp-login.php HTTP/1.1" 200 2988 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:62.0) Gecko/20100101 Firefox/62.0"
13.57.220.172 - - [23/Mar/2019:11:01:06 -0400] "POST /wp-login.php HTTP/1.1" 200 3985 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:62.0) Gecko/20100101 Firefox/62.0"
13.57.220.172 - - [23/Mar/2019:11:01:07 -0400] "GET /wp-login.php HTTP/1.1" 200 2988 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:62.0) Gecko/20100101 Firefox/62.0"
13.57.220.172 - - [23/Mar/2019:11:01:08 -0400] "POST /wp-login.php HTTP/1.1" 200 3833 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:62.0) Gecko/20100101 Firefox/62.0"
13.57.220.172 - - [23/Mar/2019:11:01:09 -0400] "GET /wp-login.php HTTP/1.1" 200 2988 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:62.0) Gecko/20100101 Firefox/62.0"
13.57.220.172 - - [23/Mar/2019:11:01:11 -0400] "POST /wp-login.php HTTP/1.1" 200 3836 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:62.0) Gecko/20100101 Firefox/62.0"
13.57.220.172 - - [23/Mar/2019:11:01:12 -0400] "GET /wp-login.php HTTP/1.1" 200 2988 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:62.0) Gecko/20100101 Firefox/62.0"
13.57.220.172 - - [23/Mar/2019:11:01:15 -0400] "POST /wp-login.php HTTP/1.1" 200 3837 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:62.0) Gecko/20100101 Firefox/62.0"
13.57.220.172 - - [23/Mar/2019:11:01:17 -0400] "POST /xmlrpc.php HTTP/1.1" 200 413 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:62.0) Gecko/20100101 Firefox/62.0"
13.57.233.99 - - [23/Mar/2019:04:17:45 -0400] "GET / HTTP/1.1" 200 25160 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36"
18.206.226.75 - - [23/Mar/2019:21:58:07 -0400] "GET /wp-login.php HTTP/1.1" 200 2988 "https://www.google.com/url?3a622303df89920683e4421b2cf28977" "Mozilla/5.0 (Windows NT 6.2; rv:33.0) Gecko/20100101 Firefox/33.0"
18.206.226.75 - - [23/Mar/2019:21:58:07 -0400] "POST /wp-login.php HTTP/1.1" 200 3988 "https://www.google.com/url?3a622303df89920683e4421b2cf28977" "Mozilla/5.0 (Windows NT 6.2; rv:33.0) Gecko/20100101 Firefox/33.0"
18.213.10.181 - - [23/Mar/2019:14:45:42 -0400] "GET /wp-login.php HTTP/1.1" 200 2988 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:62.0) Gecko/20100101 Firefox/62.0"
18.213.10.181 - - [23/Mar/2019:14:45:42 -0400] "GET /wp-login.php HTTP/1.1" 200 2988 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:62.0) Gecko/20100101 Firefox/62.0"
18.213.10.181 - - [23/Mar/2019:14:45:42 -0400] "GET /wp-login.php HTTP/1.1" 200 2988 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:62.0) Gecko/20100101 Firefox/62.0"









share|improve this question
























  • With “bash”, do you mean the plain shell or the command line in general?

    – dessert
    2 hours ago















5

















I have a log file sorted by IP addresses,
I want to find the number of occurrences of each unique IP address.
How can I do this with bash? Possibly listing the number of occurrences next to an ip, such as:



5.135.134.16 count: 5
13.57.220.172: count 30
18.206.226 count:2


and so on.



Here’s a sample of the log:



5.135.134.16 - - [23/Mar/2019:08:42:54 -0400] "GET /wp-login.php HTTP/1.1" 200 2988 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:62.0) Gecko/20100101 Firefox/62.0"
5.135.134.16 - - [23/Mar/2019:08:42:55 -0400] "GET /wp-login.php HTTP/1.1" 200 2988 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:62.0) Gecko/20100101 Firefox/62.0"
5.135.134.16 - - [23/Mar/2019:08:42:55 -0400] "POST /wp-login.php HTTP/1.1" 200 3836 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:62.0) Gecko/20100101 Firefox/62.0"
5.135.134.16 - - [23/Mar/2019:08:42:55 -0400] "POST /wp-login.php HTTP/1.1" 200 3988 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:62.0) Gecko/20100101 Firefox/62.0"
5.135.134.16 - - [23/Mar/2019:08:42:56 -0400] "POST /xmlrpc.php HTTP/1.1" 200 413 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:62.0) Gecko/20100101 Firefox/62.0"
13.57.220.172 - - [23/Mar/2019:11:01:05 -0400] "GET /wp-login.php HTTP/1.1" 200 2988 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:62.0) Gecko/20100101 Firefox/62.0"
13.57.220.172 - - [23/Mar/2019:11:01:06 -0400] "POST /wp-login.php HTTP/1.1" 200 3985 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:62.0) Gecko/20100101 Firefox/62.0"
13.57.220.172 - - [23/Mar/2019:11:01:07 -0400] "GET /wp-login.php HTTP/1.1" 200 2988 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:62.0) Gecko/20100101 Firefox/62.0"
13.57.220.172 - - [23/Mar/2019:11:01:08 -0400] "POST /wp-login.php HTTP/1.1" 200 3833 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:62.0) Gecko/20100101 Firefox/62.0"
13.57.220.172 - - [23/Mar/2019:11:01:09 -0400] "GET /wp-login.php HTTP/1.1" 200 2988 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:62.0) Gecko/20100101 Firefox/62.0"
13.57.220.172 - - [23/Mar/2019:11:01:11 -0400] "POST /wp-login.php HTTP/1.1" 200 3836 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:62.0) Gecko/20100101 Firefox/62.0"
13.57.220.172 - - [23/Mar/2019:11:01:12 -0400] "GET /wp-login.php HTTP/1.1" 200 2988 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:62.0) Gecko/20100101 Firefox/62.0"
13.57.220.172 - - [23/Mar/2019:11:01:15 -0400] "POST /wp-login.php HTTP/1.1" 200 3837 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:62.0) Gecko/20100101 Firefox/62.0"
13.57.220.172 - - [23/Mar/2019:11:01:17 -0400] "POST /xmlrpc.php HTTP/1.1" 200 413 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:62.0) Gecko/20100101 Firefox/62.0"
13.57.233.99 - - [23/Mar/2019:04:17:45 -0400] "GET / HTTP/1.1" 200 25160 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36"
18.206.226.75 - - [23/Mar/2019:21:58:07 -0400] "GET /wp-login.php HTTP/1.1" 200 2988 "https://www.google.com/url?3a622303df89920683e4421b2cf28977" "Mozilla/5.0 (Windows NT 6.2; rv:33.0) Gecko/20100101 Firefox/33.0"
18.206.226.75 - - [23/Mar/2019:21:58:07 -0400] "POST /wp-login.php HTTP/1.1" 200 3988 "https://www.google.com/url?3a622303df89920683e4421b2cf28977" "Mozilla/5.0 (Windows NT 6.2; rv:33.0) Gecko/20100101 Firefox/33.0"
18.213.10.181 - - [23/Mar/2019:14:45:42 -0400] "GET /wp-login.php HTTP/1.1" 200 2988 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:62.0) Gecko/20100101 Firefox/62.0"
18.213.10.181 - - [23/Mar/2019:14:45:42 -0400] "GET /wp-login.php HTTP/1.1" 200 2988 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:62.0) Gecko/20100101 Firefox/62.0"
18.213.10.181 - - [23/Mar/2019:14:45:42 -0400] "GET /wp-login.php HTTP/1.1" 200 2988 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:62.0) Gecko/20100101 Firefox/62.0"









share|improve this question
























  • With “bash”, do you mean the plain shell or the command line in general?

    – dessert
    2 hours ago













5












5








5


3








I have a log file sorted by IP addresses,
I want to find the number of occurrences of each unique IP address.
How can I do this with bash? Possibly listing the number of occurrences next to an ip, such as:



5.135.134.16 count: 5
13.57.220.172: count 30
18.206.226 count:2


and so on.



Here’s a sample of the log:



5.135.134.16 - - [23/Mar/2019:08:42:54 -0400] "GET /wp-login.php HTTP/1.1" 200 2988 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:62.0) Gecko/20100101 Firefox/62.0"
5.135.134.16 - - [23/Mar/2019:08:42:55 -0400] "GET /wp-login.php HTTP/1.1" 200 2988 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:62.0) Gecko/20100101 Firefox/62.0"
5.135.134.16 - - [23/Mar/2019:08:42:55 -0400] "POST /wp-login.php HTTP/1.1" 200 3836 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:62.0) Gecko/20100101 Firefox/62.0"
5.135.134.16 - - [23/Mar/2019:08:42:55 -0400] "POST /wp-login.php HTTP/1.1" 200 3988 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:62.0) Gecko/20100101 Firefox/62.0"
5.135.134.16 - - [23/Mar/2019:08:42:56 -0400] "POST /xmlrpc.php HTTP/1.1" 200 413 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:62.0) Gecko/20100101 Firefox/62.0"
13.57.220.172 - - [23/Mar/2019:11:01:05 -0400] "GET /wp-login.php HTTP/1.1" 200 2988 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:62.0) Gecko/20100101 Firefox/62.0"
13.57.220.172 - - [23/Mar/2019:11:01:06 -0400] "POST /wp-login.php HTTP/1.1" 200 3985 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:62.0) Gecko/20100101 Firefox/62.0"
13.57.220.172 - - [23/Mar/2019:11:01:07 -0400] "GET /wp-login.php HTTP/1.1" 200 2988 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:62.0) Gecko/20100101 Firefox/62.0"
13.57.220.172 - - [23/Mar/2019:11:01:08 -0400] "POST /wp-login.php HTTP/1.1" 200 3833 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:62.0) Gecko/20100101 Firefox/62.0"
13.57.220.172 - - [23/Mar/2019:11:01:09 -0400] "GET /wp-login.php HTTP/1.1" 200 2988 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:62.0) Gecko/20100101 Firefox/62.0"
13.57.220.172 - - [23/Mar/2019:11:01:11 -0400] "POST /wp-login.php HTTP/1.1" 200 3836 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:62.0) Gecko/20100101 Firefox/62.0"
13.57.220.172 - - [23/Mar/2019:11:01:12 -0400] "GET /wp-login.php HTTP/1.1" 200 2988 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:62.0) Gecko/20100101 Firefox/62.0"
13.57.220.172 - - [23/Mar/2019:11:01:15 -0400] "POST /wp-login.php HTTP/1.1" 200 3837 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:62.0) Gecko/20100101 Firefox/62.0"
13.57.220.172 - - [23/Mar/2019:11:01:17 -0400] "POST /xmlrpc.php HTTP/1.1" 200 413 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:62.0) Gecko/20100101 Firefox/62.0"
13.57.233.99 - - [23/Mar/2019:04:17:45 -0400] "GET / HTTP/1.1" 200 25160 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36"
18.206.226.75 - - [23/Mar/2019:21:58:07 -0400] "GET /wp-login.php HTTP/1.1" 200 2988 "https://www.google.com/url?3a622303df89920683e4421b2cf28977" "Mozilla/5.0 (Windows NT 6.2; rv:33.0) Gecko/20100101 Firefox/33.0"
18.206.226.75 - - [23/Mar/2019:21:58:07 -0400] "POST /wp-login.php HTTP/1.1" 200 3988 "https://www.google.com/url?3a622303df89920683e4421b2cf28977" "Mozilla/5.0 (Windows NT 6.2; rv:33.0) Gecko/20100101 Firefox/33.0"
18.213.10.181 - - [23/Mar/2019:14:45:42 -0400] "GET /wp-login.php HTTP/1.1" 200 2988 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:62.0) Gecko/20100101 Firefox/62.0"
18.213.10.181 - - [23/Mar/2019:14:45:42 -0400] "GET /wp-login.php HTTP/1.1" 200 2988 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:62.0) Gecko/20100101 Firefox/62.0"
18.213.10.181 - - [23/Mar/2019:14:45:42 -0400] "GET /wp-login.php HTTP/1.1" 200 2988 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:62.0) Gecko/20100101 Firefox/62.0"









share|improve this question


















I have a log file sorted by IP addresses,
I want to find the number of occurrences of each unique IP address.
How can I do this with bash? Possibly listing the number of occurrences next to an ip, such as:



5.135.134.16 count: 5
13.57.220.172: count 30
18.206.226 count:2


and so on.



Here’s a sample of the log:



5.135.134.16 - - [23/Mar/2019:08:42:54 -0400] "GET /wp-login.php HTTP/1.1" 200 2988 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:62.0) Gecko/20100101 Firefox/62.0"
5.135.134.16 - - [23/Mar/2019:08:42:55 -0400] "GET /wp-login.php HTTP/1.1" 200 2988 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:62.0) Gecko/20100101 Firefox/62.0"
5.135.134.16 - - [23/Mar/2019:08:42:55 -0400] "POST /wp-login.php HTTP/1.1" 200 3836 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:62.0) Gecko/20100101 Firefox/62.0"
5.135.134.16 - - [23/Mar/2019:08:42:55 -0400] "POST /wp-login.php HTTP/1.1" 200 3988 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:62.0) Gecko/20100101 Firefox/62.0"
5.135.134.16 - - [23/Mar/2019:08:42:56 -0400] "POST /xmlrpc.php HTTP/1.1" 200 413 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:62.0) Gecko/20100101 Firefox/62.0"
13.57.220.172 - - [23/Mar/2019:11:01:05 -0400] "GET /wp-login.php HTTP/1.1" 200 2988 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:62.0) Gecko/20100101 Firefox/62.0"
13.57.220.172 - - [23/Mar/2019:11:01:06 -0400] "POST /wp-login.php HTTP/1.1" 200 3985 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:62.0) Gecko/20100101 Firefox/62.0"
13.57.220.172 - - [23/Mar/2019:11:01:07 -0400] "GET /wp-login.php HTTP/1.1" 200 2988 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:62.0) Gecko/20100101 Firefox/62.0"
13.57.220.172 - - [23/Mar/2019:11:01:08 -0400] "POST /wp-login.php HTTP/1.1" 200 3833 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:62.0) Gecko/20100101 Firefox/62.0"
13.57.220.172 - - [23/Mar/2019:11:01:09 -0400] "GET /wp-login.php HTTP/1.1" 200 2988 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:62.0) Gecko/20100101 Firefox/62.0"
13.57.220.172 - - [23/Mar/2019:11:01:11 -0400] "POST /wp-login.php HTTP/1.1" 200 3836 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:62.0) Gecko/20100101 Firefox/62.0"
13.57.220.172 - - [23/Mar/2019:11:01:12 -0400] "GET /wp-login.php HTTP/1.1" 200 2988 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:62.0) Gecko/20100101 Firefox/62.0"
13.57.220.172 - - [23/Mar/2019:11:01:15 -0400] "POST /wp-login.php HTTP/1.1" 200 3837 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:62.0) Gecko/20100101 Firefox/62.0"
13.57.220.172 - - [23/Mar/2019:11:01:17 -0400] "POST /xmlrpc.php HTTP/1.1" 200 413 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:62.0) Gecko/20100101 Firefox/62.0"
13.57.233.99 - - [23/Mar/2019:04:17:45 -0400] "GET / HTTP/1.1" 200 25160 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36"
18.206.226.75 - - [23/Mar/2019:21:58:07 -0400] "GET /wp-login.php HTTP/1.1" 200 2988 "https://www.google.com/url?3a622303df89920683e4421b2cf28977" "Mozilla/5.0 (Windows NT 6.2; rv:33.0) Gecko/20100101 Firefox/33.0"
18.206.226.75 - - [23/Mar/2019:21:58:07 -0400] "POST /wp-login.php HTTP/1.1" 200 3988 "https://www.google.com/url?3a622303df89920683e4421b2cf28977" "Mozilla/5.0 (Windows NT 6.2; rv:33.0) Gecko/20100101 Firefox/33.0"
18.213.10.181 - - [23/Mar/2019:14:45:42 -0400] "GET /wp-login.php HTTP/1.1" 200 2988 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:62.0) Gecko/20100101 Firefox/62.0"
18.213.10.181 - - [23/Mar/2019:14:45:42 -0400] "GET /wp-login.php HTTP/1.1" 200 2988 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:62.0) Gecko/20100101 Firefox/62.0"
18.213.10.181 - - [23/Mar/2019:14:45:42 -0400] "GET /wp-login.php HTTP/1.1" 200 2988 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:62.0) Gecko/20100101 Firefox/62.0"






command-line bash sort uniq






share|improve this question















share|improve this question













share|improve this question




share|improve this question








edited 1 hour ago









dessert

25.2k673106




25.2k673106










asked 2 hours ago









j0hj0h

6,4971657119




6,4971657119












  • With “bash”, do you mean the plain shell or the command line in general?

    – dessert
    2 hours ago

















  • With “bash”, do you mean the plain shell or the command line in general?

    – dessert
    2 hours ago
















With “bash”, do you mean the plain shell or the command line in general?

– dessert
2 hours ago





With “bash”, do you mean the plain shell or the command line in general?

– dessert
2 hours ago










4 Answers
4






active

oldest

votes


















5














You can use grep and uniq for the list of addresses, loop over them and grep again for the count:



for i in $(<log grep -o '^[^ ]*' | uniq); do
printf '%s count %dn' "$i" $(<log grep -c "$i")
done


grep -o '^[^ ]*' outputs every character from the beginning (^) until the first space of each line, uniq removes repeated lines, thus leaving you with a list of IP addresses. Thanks to command substitution, the for loop loops over this list printing the currently processed IP followed by “ count ” and the count. The latter is computed by grep -c, which counts the number of lines with at least one match.



Example run



$ for i in $(<log grep -o '^[^ ]*'|uniq);do printf '%s count %dn' "$i" $(<log grep -c "$i");done
5.135.134.16 count 5
13.57.220.172 count 9
13.57.233.99 count 1
18.206.226.75 count 2
18.213.10.181 count 3





share|improve this answer
































    9














    You can use cut and uniq tools:



    cut -d ' ' -f1 test.txt | uniq -c
    5 5.135.134.16
    9 13.57.220.172
    1 13.57.233.99
    2 18.206.226.75
    3 18.213.10.181


    Explanation :




    • cut -d ' ' -f1 : extract first field (ip address)


    • uniq -c : report repeated lines and display the number of occurences





    share|improve this answer










    New contributor




    Mikael Flora is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
    Check out our Code of Conduct.















    • 1





      One could use sed, e.g. sed -E 's/ *(S*) *(S*)/2 count: 1/' to get the output exactly like OP wanted.

      – dessert
      2 hours ago


















    5














    If you don't specifically require the given output format, then I would recommend the already posted cut + uniq based answer



    If you really need the given output format, a single-pass way to do it in Awk would be



    awk 'c[$1]++ ENDfor(i in c) print i, "count: " c[i]' log


    This is somewhat non-ideal when the input is already sorted since it unnecessarily stores all the IPs into memory - a better, though more complicated, way to do it in the pre-sorted case (more directly equivalent to uniq -c) would be:



    awk '
    NR==1 last=$1
    $1 != last print last, "count: " c[last]; last = $1
    c[$1]++
    END print last, "count: " c[last]
    '


    Ex.



    $ awk 'NR==1 last=$1 $1 != last print last, "count: " c[last]; last = $1 c[$1]++ ENDprint last, "count: " c[last]' log
    5.135.134.16 count: 5
    13.57.220.172 count: 9
    13.57.233.99 count: 1
    18.206.226.75 count: 2
    18.213.10.181 count: 3





    share|improve this answer
































      4














      Here is one possible solution:





      IN_FILE="file.log"
      for IP in $(awk 'print $1' "$IN_FILE" | sort -u)
      do
      echo -en "$IPtcount: "
      grep -c "$IP" "$IN_FILE"
      done


      • replace file.log with the actual file name.

      • the command substitution expression $(awk 'print $1' "$IN_FILE" | sort -u) will provide a list of the unique values of the first column.

      • then grep -c will count each of these values within the file.


      $ IN_FILE="file.log"; for IP in $(awk 'print $1' "$IN_FILE" | sort -u); do echo -en "$IPtcount: "; grep -c "$IP" "$IN_FILE"; done
      13.57.220.172 count: 9
      13.57.233.99 count: 1
      18.206.226.75 count: 2
      18.213.10.181 count: 3
      5.135.134.16 count: 5





      share|improve this answer

























        Your Answer








        StackExchange.ready(function()
        var channelOptions =
        tags: "".split(" "),
        id: "89"
        ;
        initTagRenderer("".split(" "), "".split(" "), channelOptions);

        StackExchange.using("externalEditor", function()
        // Have to fire editor after snippets, if snippets enabled
        if (StackExchange.settings.snippets.snippetsEnabled)
        StackExchange.using("snippets", function()
        createEditor();
        );

        else
        createEditor();

        );

        function createEditor()
        StackExchange.prepareEditor(
        heartbeatType: 'answer',
        autoActivateHeartbeat: false,
        convertImagesToLinks: true,
        noModals: true,
        showLowRepImageUploadWarning: true,
        reputationToPostImages: 10,
        bindNavPrevention: true,
        postfix: "",
        imageUploader:
        brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
        contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
        allowUrls: true
        ,
        onDemand: true,
        discardSelector: ".discard-answer"
        ,immediatelyShowMarkdownHelp:true
        );



        );













        draft saved

        draft discarded


















        StackExchange.ready(
        function ()
        StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2faskubuntu.com%2fquestions%2f1129521%2fhow-to-count-occurrences-of-text-in-a-file%23new-answer', 'question_page');

        );

        Post as a guest















        Required, but never shown

























        4 Answers
        4






        active

        oldest

        votes








        4 Answers
        4






        active

        oldest

        votes









        active

        oldest

        votes






        active

        oldest

        votes









        5














        You can use grep and uniq for the list of addresses, loop over them and grep again for the count:



        for i in $(<log grep -o '^[^ ]*' | uniq); do
        printf '%s count %dn' "$i" $(<log grep -c "$i")
        done


        grep -o '^[^ ]*' outputs every character from the beginning (^) until the first space of each line, uniq removes repeated lines, thus leaving you with a list of IP addresses. Thanks to command substitution, the for loop loops over this list printing the currently processed IP followed by “ count ” and the count. The latter is computed by grep -c, which counts the number of lines with at least one match.



        Example run



        $ for i in $(<log grep -o '^[^ ]*'|uniq);do printf '%s count %dn' "$i" $(<log grep -c "$i");done
        5.135.134.16 count 5
        13.57.220.172 count 9
        13.57.233.99 count 1
        18.206.226.75 count 2
        18.213.10.181 count 3





        share|improve this answer





























          5














          You can use grep and uniq for the list of addresses, loop over them and grep again for the count:



          for i in $(<log grep -o '^[^ ]*' | uniq); do
          printf '%s count %dn' "$i" $(<log grep -c "$i")
          done


          grep -o '^[^ ]*' outputs every character from the beginning (^) until the first space of each line, uniq removes repeated lines, thus leaving you with a list of IP addresses. Thanks to command substitution, the for loop loops over this list printing the currently processed IP followed by “ count ” and the count. The latter is computed by grep -c, which counts the number of lines with at least one match.



          Example run



          $ for i in $(<log grep -o '^[^ ]*'|uniq);do printf '%s count %dn' "$i" $(<log grep -c "$i");done
          5.135.134.16 count 5
          13.57.220.172 count 9
          13.57.233.99 count 1
          18.206.226.75 count 2
          18.213.10.181 count 3





          share|improve this answer



























            5












            5








            5







            You can use grep and uniq for the list of addresses, loop over them and grep again for the count:



            for i in $(<log grep -o '^[^ ]*' | uniq); do
            printf '%s count %dn' "$i" $(<log grep -c "$i")
            done


            grep -o '^[^ ]*' outputs every character from the beginning (^) until the first space of each line, uniq removes repeated lines, thus leaving you with a list of IP addresses. Thanks to command substitution, the for loop loops over this list printing the currently processed IP followed by “ count ” and the count. The latter is computed by grep -c, which counts the number of lines with at least one match.



            Example run



            $ for i in $(<log grep -o '^[^ ]*'|uniq);do printf '%s count %dn' "$i" $(<log grep -c "$i");done
            5.135.134.16 count 5
            13.57.220.172 count 9
            13.57.233.99 count 1
            18.206.226.75 count 2
            18.213.10.181 count 3





            share|improve this answer















            You can use grep and uniq for the list of addresses, loop over them and grep again for the count:



            for i in $(<log grep -o '^[^ ]*' | uniq); do
            printf '%s count %dn' "$i" $(<log grep -c "$i")
            done


            grep -o '^[^ ]*' outputs every character from the beginning (^) until the first space of each line, uniq removes repeated lines, thus leaving you with a list of IP addresses. Thanks to command substitution, the for loop loops over this list printing the currently processed IP followed by “ count ” and the count. The latter is computed by grep -c, which counts the number of lines with at least one match.



            Example run



            $ for i in $(<log grep -o '^[^ ]*'|uniq);do printf '%s count %dn' "$i" $(<log grep -c "$i");done
            5.135.134.16 count 5
            13.57.220.172 count 9
            13.57.233.99 count 1
            18.206.226.75 count 2
            18.213.10.181 count 3






            share|improve this answer














            share|improve this answer



            share|improve this answer








            edited 1 hour ago

























            answered 2 hours ago









            dessertdessert

            25.2k673106




            25.2k673106























                9














                You can use cut and uniq tools:



                cut -d ' ' -f1 test.txt | uniq -c
                5 5.135.134.16
                9 13.57.220.172
                1 13.57.233.99
                2 18.206.226.75
                3 18.213.10.181


                Explanation :




                • cut -d ' ' -f1 : extract first field (ip address)


                • uniq -c : report repeated lines and display the number of occurences





                share|improve this answer










                New contributor




                Mikael Flora is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
                Check out our Code of Conduct.















                • 1





                  One could use sed, e.g. sed -E 's/ *(S*) *(S*)/2 count: 1/' to get the output exactly like OP wanted.

                  – dessert
                  2 hours ago















                9














                You can use cut and uniq tools:



                cut -d ' ' -f1 test.txt | uniq -c
                5 5.135.134.16
                9 13.57.220.172
                1 13.57.233.99
                2 18.206.226.75
                3 18.213.10.181


                Explanation :




                • cut -d ' ' -f1 : extract first field (ip address)


                • uniq -c : report repeated lines and display the number of occurences





                share|improve this answer










                New contributor




                Mikael Flora is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
                Check out our Code of Conduct.















                • 1





                  One could use sed, e.g. sed -E 's/ *(S*) *(S*)/2 count: 1/' to get the output exactly like OP wanted.

                  – dessert
                  2 hours ago













                9












                9








                9







                You can use cut and uniq tools:



                cut -d ' ' -f1 test.txt | uniq -c
                5 5.135.134.16
                9 13.57.220.172
                1 13.57.233.99
                2 18.206.226.75
                3 18.213.10.181


                Explanation :




                • cut -d ' ' -f1 : extract first field (ip address)


                • uniq -c : report repeated lines and display the number of occurences





                share|improve this answer










                New contributor




                Mikael Flora is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
                Check out our Code of Conduct.










                You can use cut and uniq tools:



                cut -d ' ' -f1 test.txt | uniq -c
                5 5.135.134.16
                9 13.57.220.172
                1 13.57.233.99
                2 18.206.226.75
                3 18.213.10.181


                Explanation :




                • cut -d ' ' -f1 : extract first field (ip address)


                • uniq -c : report repeated lines and display the number of occurences






                share|improve this answer










                New contributor




                Mikael Flora is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
                Check out our Code of Conduct.









                share|improve this answer



                share|improve this answer








                edited 1 hour ago





















                New contributor




                Mikael Flora is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
                Check out our Code of Conduct.









                answered 2 hours ago









                Mikael FloraMikael Flora

                915




                915




                New contributor




                Mikael Flora is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
                Check out our Code of Conduct.





                New contributor





                Mikael Flora is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
                Check out our Code of Conduct.






                Mikael Flora is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
                Check out our Code of Conduct.







                • 1





                  One could use sed, e.g. sed -E 's/ *(S*) *(S*)/2 count: 1/' to get the output exactly like OP wanted.

                  – dessert
                  2 hours ago












                • 1





                  One could use sed, e.g. sed -E 's/ *(S*) *(S*)/2 count: 1/' to get the output exactly like OP wanted.

                  – dessert
                  2 hours ago







                1




                1





                One could use sed, e.g. sed -E 's/ *(S*) *(S*)/2 count: 1/' to get the output exactly like OP wanted.

                – dessert
                2 hours ago





                One could use sed, e.g. sed -E 's/ *(S*) *(S*)/2 count: 1/' to get the output exactly like OP wanted.

                – dessert
                2 hours ago











                5














                If you don't specifically require the given output format, then I would recommend the already posted cut + uniq based answer



                If you really need the given output format, a single-pass way to do it in Awk would be



                awk 'c[$1]++ ENDfor(i in c) print i, "count: " c[i]' log


                This is somewhat non-ideal when the input is already sorted since it unnecessarily stores all the IPs into memory - a better, though more complicated, way to do it in the pre-sorted case (more directly equivalent to uniq -c) would be:



                awk '
                NR==1 last=$1
                $1 != last print last, "count: " c[last]; last = $1
                c[$1]++
                END print last, "count: " c[last]
                '


                Ex.



                $ awk 'NR==1 last=$1 $1 != last print last, "count: " c[last]; last = $1 c[$1]++ ENDprint last, "count: " c[last]' log
                5.135.134.16 count: 5
                13.57.220.172 count: 9
                13.57.233.99 count: 1
                18.206.226.75 count: 2
                18.213.10.181 count: 3





                share|improve this answer





























                  5














                  If you don't specifically require the given output format, then I would recommend the already posted cut + uniq based answer



                  If you really need the given output format, a single-pass way to do it in Awk would be



                  awk 'c[$1]++ ENDfor(i in c) print i, "count: " c[i]' log


                  This is somewhat non-ideal when the input is already sorted since it unnecessarily stores all the IPs into memory - a better, though more complicated, way to do it in the pre-sorted case (more directly equivalent to uniq -c) would be:



                  awk '
                  NR==1 last=$1
                  $1 != last print last, "count: " c[last]; last = $1
                  c[$1]++
                  END print last, "count: " c[last]
                  '


                  Ex.



                  $ awk 'NR==1 last=$1 $1 != last print last, "count: " c[last]; last = $1 c[$1]++ ENDprint last, "count: " c[last]' log
                  5.135.134.16 count: 5
                  13.57.220.172 count: 9
                  13.57.233.99 count: 1
                  18.206.226.75 count: 2
                  18.213.10.181 count: 3





                  share|improve this answer



























                    5












                    5








                    5







                    If you don't specifically require the given output format, then I would recommend the already posted cut + uniq based answer



                    If you really need the given output format, a single-pass way to do it in Awk would be



                    awk 'c[$1]++ ENDfor(i in c) print i, "count: " c[i]' log


                    This is somewhat non-ideal when the input is already sorted since it unnecessarily stores all the IPs into memory - a better, though more complicated, way to do it in the pre-sorted case (more directly equivalent to uniq -c) would be:



                    awk '
                    NR==1 last=$1
                    $1 != last print last, "count: " c[last]; last = $1
                    c[$1]++
                    END print last, "count: " c[last]
                    '


                    Ex.



                    $ awk 'NR==1 last=$1 $1 != last print last, "count: " c[last]; last = $1 c[$1]++ ENDprint last, "count: " c[last]' log
                    5.135.134.16 count: 5
                    13.57.220.172 count: 9
                    13.57.233.99 count: 1
                    18.206.226.75 count: 2
                    18.213.10.181 count: 3





                    share|improve this answer















                    If you don't specifically require the given output format, then I would recommend the already posted cut + uniq based answer



                    If you really need the given output format, a single-pass way to do it in Awk would be



                    awk 'c[$1]++ ENDfor(i in c) print i, "count: " c[i]' log


                    This is somewhat non-ideal when the input is already sorted since it unnecessarily stores all the IPs into memory - a better, though more complicated, way to do it in the pre-sorted case (more directly equivalent to uniq -c) would be:



                    awk '
                    NR==1 last=$1
                    $1 != last print last, "count: " c[last]; last = $1
                    c[$1]++
                    END print last, "count: " c[last]
                    '


                    Ex.



                    $ awk 'NR==1 last=$1 $1 != last print last, "count: " c[last]; last = $1 c[$1]++ ENDprint last, "count: " c[last]' log
                    5.135.134.16 count: 5
                    13.57.220.172 count: 9
                    13.57.233.99 count: 1
                    18.206.226.75 count: 2
                    18.213.10.181 count: 3






                    share|improve this answer














                    share|improve this answer



                    share|improve this answer








                    edited 1 hour ago

























                    answered 2 hours ago









                    steeldriversteeldriver

                    70.3k11114186




                    70.3k11114186





















                        4














                        Here is one possible solution:





                        IN_FILE="file.log"
                        for IP in $(awk 'print $1' "$IN_FILE" | sort -u)
                        do
                        echo -en "$IPtcount: "
                        grep -c "$IP" "$IN_FILE"
                        done


                        • replace file.log with the actual file name.

                        • the command substitution expression $(awk 'print $1' "$IN_FILE" | sort -u) will provide a list of the unique values of the first column.

                        • then grep -c will count each of these values within the file.


                        $ IN_FILE="file.log"; for IP in $(awk 'print $1' "$IN_FILE" | sort -u); do echo -en "$IPtcount: "; grep -c "$IP" "$IN_FILE"; done
                        13.57.220.172 count: 9
                        13.57.233.99 count: 1
                        18.206.226.75 count: 2
                        18.213.10.181 count: 3
                        5.135.134.16 count: 5





                        share|improve this answer





























                          4














                          Here is one possible solution:





                          IN_FILE="file.log"
                          for IP in $(awk 'print $1' "$IN_FILE" | sort -u)
                          do
                          echo -en "$IPtcount: "
                          grep -c "$IP" "$IN_FILE"
                          done


                          • replace file.log with the actual file name.

                          • the command substitution expression $(awk 'print $1' "$IN_FILE" | sort -u) will provide a list of the unique values of the first column.

                          • then grep -c will count each of these values within the file.


                          $ IN_FILE="file.log"; for IP in $(awk 'print $1' "$IN_FILE" | sort -u); do echo -en "$IPtcount: "; grep -c "$IP" "$IN_FILE"; done
                          13.57.220.172 count: 9
                          13.57.233.99 count: 1
                          18.206.226.75 count: 2
                          18.213.10.181 count: 3
                          5.135.134.16 count: 5





                          share|improve this answer



























                            4












                            4








                            4







                            Here is one possible solution:





                            IN_FILE="file.log"
                            for IP in $(awk 'print $1' "$IN_FILE" | sort -u)
                            do
                            echo -en "$IPtcount: "
                            grep -c "$IP" "$IN_FILE"
                            done


                            • replace file.log with the actual file name.

                            • the command substitution expression $(awk 'print $1' "$IN_FILE" | sort -u) will provide a list of the unique values of the first column.

                            • then grep -c will count each of these values within the file.


                            $ IN_FILE="file.log"; for IP in $(awk 'print $1' "$IN_FILE" | sort -u); do echo -en "$IPtcount: "; grep -c "$IP" "$IN_FILE"; done
                            13.57.220.172 count: 9
                            13.57.233.99 count: 1
                            18.206.226.75 count: 2
                            18.213.10.181 count: 3
                            5.135.134.16 count: 5





                            share|improve this answer















                            Here is one possible solution:





                            IN_FILE="file.log"
                            for IP in $(awk 'print $1' "$IN_FILE" | sort -u)
                            do
                            echo -en "$IPtcount: "
                            grep -c "$IP" "$IN_FILE"
                            done


                            • replace file.log with the actual file name.

                            • the command substitution expression $(awk 'print $1' "$IN_FILE" | sort -u) will provide a list of the unique values of the first column.

                            • then grep -c will count each of these values within the file.


                            $ IN_FILE="file.log"; for IP in $(awk 'print $1' "$IN_FILE" | sort -u); do echo -en "$IPtcount: "; grep -c "$IP" "$IN_FILE"; done
                            13.57.220.172 count: 9
                            13.57.233.99 count: 1
                            18.206.226.75 count: 2
                            18.213.10.181 count: 3
                            5.135.134.16 count: 5






                            share|improve this answer














                            share|improve this answer



                            share|improve this answer








                            edited 2 hours ago

























                            answered 2 hours ago









                            pa4080pa4080

                            14.7k52872




                            14.7k52872



























                                draft saved

                                draft discarded
















































                                Thanks for contributing an answer to Ask Ubuntu!


                                • Please be sure to answer the question. Provide details and share your research!

                                But avoid


                                • Asking for help, clarification, or responding to other answers.

                                • Making statements based on opinion; back them up with references or personal experience.

                                To learn more, see our tips on writing great answers.




                                draft saved


                                draft discarded














                                StackExchange.ready(
                                function ()
                                StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2faskubuntu.com%2fquestions%2f1129521%2fhow-to-count-occurrences-of-text-in-a-file%23new-answer', 'question_page');

                                );

                                Post as a guest















                                Required, but never shown





















































                                Required, but never shown














                                Required, but never shown












                                Required, but never shown







                                Required, but never shown

































                                Required, but never shown














                                Required, but never shown












                                Required, but never shown







                                Required, but never shown







                                Popular posts from this blog

                                How does Billy Russo acquire his 'Jigsaw' mask? Unicorn Meta Zoo #1: Why another podcast? Announcing the arrival of Valued Associate #679: Cesar Manara Favourite questions and answers from the 1st quarter of 2019Why does Bane wear the mask?Why does Kylo Ren wear a mask?Why did Captain America remove his mask while fighting Batroc the Leaper?How did the OA acquire her wisdom?Is Billy Breckenridge gay?How does Adrian Toomes hide his earnings from the IRS?What is the state of affairs on Nootka Sound by the end of season 1?How did Tia Dalma acquire Captain Barbossa's body?How is one “Deemed Worthy”, to acquire the Greatsword “Dawn”?How did Karen acquire the handgun?

                                Личност Атрибути на личността | Литература и източници | НавигацияРаждането на личносттаредактиратередактирате

                                A sequel to Domino's tragic life Why Christmas is for Friends Cold comfort at Charles' padSad farewell for Lady JanePS Most watched News videos