Geek Answers Handbook

Count the unique regex appearance

I have a maillog server, and I want to calculate how many email messages are sent to each user for each hour.

At the moment I’m deleting all the information that I don’t need, but I can’t calculate how many letters each unique user sends.

What I wrote so far:

 awk '{print $3, $7;}' ./maillog | sed '/from/!d' | sed 's/:[0-9][0-9]:[0-9][0-9] /:00 /g' | sed 's/from=<//g' | egrep '[a-zA-Z0-9]+\@[a-zA-Z0-9.-]+(org|net|com)' | uniq -c > output.txt

The main problem is that (I believe) that I find the same user several times in the same hour (which I don’t want).

Here is what I need to get. Keep in mind that what I need to get is just an example, this is not the right result that I should have. If you run the script that I wrote in the file I gave you, you will get user25 2 times in the same hour, which does not satisfy the requirements.

Here is an example of the output someone suggested (very long):

Jan 16 08:33:04 mail.knurledwidgets.example.org sendmail[3539]: q5c1SrFqkAZq9b: Milter: connect to filters
Jan 16 08:33:06 mail.knurledwidgets.example.org sendmail[3539]: q5c1SrFqkAZq9b: from=<user1@dont-cross-the-memes.example.com>, size=38065260, class=-30, nrcpts=1, msgid=<gnDSaYSEaP4Yk/.F0EhYbIYcihGO8Vd.dont-cross-the-memes.example.com>, proto=ESMTP, daemon=MTA-v6, relay=proton.dont-cross-the-memes.example.com [192.168.98.234]
Jan 16 08:33:06 mail.knurledwidgets.example.org sendmail[7734]: qqGjhufuNY5UJ: Milter: connect to filters
Jan 16 08:33:07 mail.knurledwidgets.example.org sendmail[8780]: qkwEbHuoJi40Lj: Milter: connect to filters
Jan 16 08:33:07 mail.knurledwidgets.example.org sendmail[8780]: qkwEbHuoJi40Lj: from=<user25@knurledwidgets.example.org>, size=36412443, class=-30, nrcpts=1, msgid=<w/7AIsHSy6.gkNTPlyyE55u.knurledwidgets.example.org>, proto=ESMTP, daemon=MTA-v6, relay=mail.knurledwidgets.example.org [10.0.0.20]
Jan 16 08:33:08 mail.knurledwidgets.example.org sendmail[7734]: qqGjhufuNY5UJ: from=<user6@stellar-patrol.example.com>, size=33411319, class=-30, nrcpts=1, msgid=<il/5SxUES9XwRhX.KfO6ywkQROALbnz.stellar-patrol.example.com>, proto=ESMTP, daemon=MTA-v6, relay=feinstein.stellar-patrol.example.com [192.168.73.3]
Jan 16 08:33:09 mail.knurledwidgets.example.org sendmail[3539]: q5c1SrFqkAZq9b: Milter accept: message
Jan 16 08:33:09 mail.knurledwidgets.example.org sendmail[8780]: qkwEbHuoJi40Lj: Milter accept: message
Jan 16 08:33:10 mail.knurledwidgets.example.org sendmail[7734]: qqGjhufuNY5UJ: Milter accept: message
Jan 16 08:33:12 mail.knurledwidgets.example.org sendmail[1618]: qhgKT0cN80gSX: Milter: connect to filters
Jan 16 08:33:13 mail.knurledwidgets.example.org sendmail[1618]: qhgKT0cN80gSX: from=<user25@knurledwidgets.example.org>, size=780642, class=-30, nrcpts=1, msgid=<hX49btAurMDDZlhWo.5RpGEJxQQilElvDgRpc3sw.knurledwidgets.example.org>, proto=ESMTP, daemon=MTA-v6, relay=mail.knurledwidgets.example.org [10.0.0.20]

And here is a sample output:

1 08:00 user10@yuhoo.example.com
1 08:00 user19@knurledwidgets.example.org
1 08:00 user1@beshonk.example.com
5 08:00 user27@knurledwidgets.example.org
1 09:00 user12@knurledwidgets.example.org
1 09:00 user17@knurledwidgets.example.org
1 09:00 user26@knurledwidgets.example.org
7 09:00 user27@knurledwidgets.example.org
2 09:00 user33@knurledwidgets.example.org
1 09:00 user42@knurledwidgets.example.org

Please also explain the answer you gave, as the goal is to learn not to do the exercise.

thank you for your time

+4
source share
1 answer

A sortto uniqgive you the calculations:

awk '{print $3, $7;}' ./maillog | sed '/from/!d' | sed 's/:[0-9][0-9]:[0-9][0-9] /:00 /g' | sed 's/from=<//g' | egrep '[a-zA-Z0-9]+\@[a-zA-Z0-9.-]+(org|net|com)' | sort | uniq -c`

  1 08:00 user1@dont-cross-the-memes.example.com>,
  2 08:00 user25@knurledwidgets.example.org>,
  1 08:00 user6@stellar-patrol.example.com>,

See uniq --help:

: uniq , . sort -u uniq. , , LC_COLLATE.

+2

More articles:


All Articles