How often you bumped into a .gz file where you need to check the contents? 
I know I do quite often. A gz file is a compressed file created with gzip and I didn’t knew better, I would copy the file into another folder, uncompress it and then look into the contents. Is there a way to avoid these unnecessary decompression and waste time? Of course there is. In Linux, you can view contents of a compressed .gz file without uncompressing (uncompress on the fly actually or in temp directory) which makes perfect sense for those who deal with large log files and does forensic stuffs. The way it’s done is by using Z commands.
Z commands for example: zcat allows you to view contents of a compressed file, zless or zmore allows you to do paging (page by page viewing of a file), zgrep or zegrep allows you to search within a compressed file, zdiff or zcmp allows you to compare two files… Sounds good? I bet it does. Welcome to Z commands that you can use to view, search, compare and page compressed files without wasting time uncompressing.
First of all, lets find some compressed files to see what it actually looks like.
Compressed files:
Open a terminal and browse to /var/log. /var/log is where most of your logs files will go by default unless otherwise specified by an application/system. Perform a list (ls) command to see contents of that directory. As you can see, many .gz files in there.
root@kali:~# cd /var/log root@kali:/var/log# ls alternatives.logĀ Ā Ā debugĀ Ā Ā Ā Ā Ā Ā Ā Ā Ā kern.log.1Ā Ā Ā Ā Ā mysql.log.4.gzĀ Ā Ā Ā Ā syslog.6.gz alternatives.log.1Ā debug.1Ā Ā Ā Ā Ā Ā Ā Ā kern.log.2.gzĀ Ā mysql.log.5.gzĀ Ā Ā Ā Ā syslog.7.gz apache2Ā Ā Ā Ā Ā Ā Ā Ā Ā Ā Ā Ā debug.2.gzĀ Ā Ā Ā Ā kern.log.3.gzĀ Ā mysql.log.6.gzĀ Ā Ā Ā Ā sysstat aptĀ Ā Ā Ā Ā Ā Ā Ā Ā Ā Ā Ā Ā Ā Ā Ā debug.3.gzĀ Ā Ā Ā Ā kern.log.4.gzĀ Ā mysql.log.7.gzĀ Ā Ā Ā Ā tor aptitudeĀ Ā Ā Ā Ā Ā Ā Ā Ā Ā Ā dmesgĀ Ā Ā Ā Ā Ā Ā Ā Ā Ā lastlogĀ Ā Ā Ā Ā Ā Ā Ā newsĀ Ā Ā Ā Ā Ā Ā Ā Ā Ā Ā Ā Ā Ā Ā unattended-upgrades aptitude.1.gzĀ Ā Ā Ā Ā Ā dmesg.0Ā Ā Ā Ā Ā Ā Ā Ā lpr.logĀ Ā Ā Ā Ā Ā Ā Ā nginxĀ Ā Ā Ā Ā Ā Ā Ā Ā Ā Ā Ā Ā Ā user.log auth.logĀ Ā Ā Ā Ā Ā Ā Ā Ā Ā Ā dmesg.1.gzĀ Ā Ā Ā Ā mail.errĀ Ā Ā Ā Ā Ā Ā ntpstatsĀ Ā Ā Ā Ā Ā Ā Ā Ā Ā Ā user.log.1 auth.log.1Ā Ā Ā Ā Ā Ā Ā Ā Ā dmesg.2.gzĀ Ā Ā Ā Ā mail.infoĀ Ā Ā Ā Ā Ā openvasĀ Ā Ā Ā Ā Ā Ā Ā Ā Ā Ā Ā user.log.2.gz auth.log.2.gzĀ Ā Ā Ā Ā Ā dmesg.3.gzĀ Ā Ā Ā Ā mail.logĀ Ā Ā Ā Ā Ā Ā pm-powersave.logĀ Ā Ā user.log.3.gz auth.log.3.gzĀ Ā Ā Ā Ā Ā dmesg.4.gzĀ Ā Ā Ā Ā mail.warnĀ Ā Ā Ā Ā Ā pm-powersave.log.1Ā user.log.4.gz auth.log.4.gzĀ Ā Ā Ā Ā Ā dpkg.logĀ Ā Ā Ā Ā Ā Ā messagesĀ Ā Ā Ā Ā Ā Ā postgresqlĀ Ā Ā Ā Ā Ā Ā Ā Ā wtmp bootstrap.logĀ Ā Ā Ā Ā Ā dpkg.log.1Ā Ā Ā Ā Ā messages.1Ā Ā Ā Ā Ā pycentral.logĀ Ā Ā Ā Ā Ā wtmp.1 btmpĀ Ā Ā Ā Ā Ā Ā Ā Ā Ā Ā Ā Ā Ā Ā dradisĀ Ā Ā Ā Ā Ā Ā Ā Ā messages.2.gzĀ Ā sambaĀ Ā Ā Ā Ā Ā Ā Ā Ā Ā Ā Ā Ā Ā wvdialconf.log btmp.1Ā Ā Ā Ā Ā Ā Ā Ā Ā Ā Ā Ā Ā exim4Ā Ā Ā Ā Ā Ā Ā Ā Ā Ā messages.3.gzĀ Ā speech-dispatcherĀ Ā Xorg.0.log chkrootkitĀ Ā Ā Ā Ā Ā Ā Ā Ā faillogĀ Ā Ā Ā Ā Ā Ā Ā messages.4.gzĀ Ā stunnel4Ā Ā Ā Ā Ā Ā Ā Ā Ā Ā Ā Xorg.0.log.old ConsoleKitĀ Ā Ā Ā Ā Ā Ā Ā Ā fontconfig.logĀ mysqlĀ Ā Ā Ā Ā Ā Ā Ā Ā Ā syslogĀ Ā Ā Ā Ā Ā Ā Ā Ā Ā Ā Ā Ā Xorg.1.log daemon.logĀ Ā Ā Ā Ā Ā Ā Ā Ā fsckĀ Ā Ā Ā Ā Ā Ā Ā Ā Ā Ā mysql.errĀ Ā Ā Ā Ā Ā syslog.1Ā Ā Ā Ā Ā Ā Ā Ā Ā Ā Ā Xorg.1.log.old daemon.log.1Ā Ā Ā Ā Ā Ā Ā gdm3Ā Ā Ā Ā Ā Ā Ā Ā Ā Ā Ā mysql.logĀ Ā Ā Ā Ā Ā syslog.2.gz daemon.log.2.gzĀ Ā Ā Ā inetsimĀ Ā Ā Ā Ā Ā Ā Ā mysql.log.1.gzĀ syslog.3.gz daemon.log.3.gzĀ Ā Ā Ā installerĀ Ā Ā Ā Ā Ā mysql.log.2.gzĀ syslog.4.gz daemon.log.4.gzĀ Ā Ā Ā kern.logĀ Ā Ā Ā Ā Ā Ā mysql.log.3.gzĀ syslog.5.gz
I will use dmesg and dmesg.1.gz files for this exercise.
Now normally if you want to see the contents of dmesg file, you would use cat command and pipe it to either more or less for pagination:
root@kali:/var/log# cat dmesg | more (output - truncated) [Ā Ā Ā 0.000000] Initializing cgroup subsys cpuset [Ā Ā Ā 0.000000] Initializing cgroup subsys cpu [Ā Ā Ā 0.000000] Initializing cgroup subsys cpuacct [Ā Ā Ā 0.000000] Linux version 3.14-kali1-amd64 (debian-kernel@lists.debian.org) (gcc version 4.7.2 (Debian 4.7.2-5) ) #1 SMP Debian 3.14.5-1kali1 (2014-06-07) [Ā Ā Ā 0.000000] Command line: BOOT_IMAGE=/boot/vmlinuz-3.14-kali1-amd64 root=UUID=9f174fa5-0c59-4024-b307-463b7bc1752d ro initrd=/install/gtk/initrd.gz quiet nouveau.modeset=0 [Ā Ā Ā 0.000000] e820: BIOS-provided physical RAM map: [Ā Ā Ā 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009efff] usable [Ā Ā Ā 0.000000] BIOS-e820: [mem 0x000000000009f000-0x000000000009ffff] reserved [Ā Ā Ā 0.000000] BIOS-e820: [mem 0x00000000000e4000-0x00000000000fffff] reserved [Ā Ā Ā 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bd77ffff] usable [Ā Ā Ā 0.000000] BIOS-e820: [mem 0x00000000bd780000-0x00000000bd78dfff] ACPI data [Ā Ā Ā 0.000000] BIOS-e820: [mem 0x00000000bd78e000-0x00000000bd7cffff] ACPI NVS [Ā Ā Ā 0.000000] BIOS-e820: [mem 0x00000000bd7d0000-0x00000000bd7dffff] reserved [Ā Ā Ā 0.000000] BIOS-e820: [mem 0x00000000bd7ed000-0x00000000bdffffff] reserved [Ā Ā Ā 0.000000] BIOS-e820: [mem 0x00000000fed20000-0x00000000fed3ffff] reserved [Ā Ā Ā 0.000000] BIOS-e820: [mem 0x00000000fee00000-0x00000000fee00fff] reserved [Ā Ā Ā 0.000000] BIOS-e820: [mem 0x00000000ffb00000-0x00000000ffffffff] reserved [Ā Ā Ā 0.000000] BIOS-e820: [mem 0x0000000100000000-0x000000023fffffff] usable [Ā Ā Ā 0.000000] NX (Execute Disable) protection: active [Ā Ā Ā 0.000000] SMBIOS 2.6 present. [Ā Ā Ā 0.000000] DMI: Acer Veriton S680GĀ Ā Ā Ā Ā /Veriton S680G, BIOS P01-B0C2Ā Ā Ā Ā Ā Ā 03/25/2011 [Ā Ā Ā 0.000000] e820: update [mem 0x00000000-0x00000fff] usable ==> reserved [Ā Ā Ā 0.000000] e820: remove [mem 0x000a0000-0x000fffff] usable [Ā Ā Ā 0.000000] No AGP bridge found --More--
root@kali:/var/log# cat dmesg | less
But what happens when you try to view contents of a compressed file such as dmesg.1.gz
root@kali:/var/log# cat dmesg.1.gz | more
j_ļæ½Sdmesg.0ļæ½ļæ½$ļæ½,ļæ½×8ļæ½&eļæ½~9ļæ½ļæ½4jļæ½qļæ½ļæ½~ļæ½ļæ½Fļæ½Yļæ½ļæ½QHļæ½.U;Oļæ½^L*ļæ½ ļæ½8ļæ½K7ļæ½ļæ½k;tIļæ½ļæ½ļæ½Gļæ½ļæ½ļæ½ ļæ½ļæ½Iļæ½ftļæ½qEļæ½ļæ½ļæ½5ļæ½Õ³ļæ½hVGļæ½dļæ½ļæ½P$ļæ½ļæ½ļæ½ļæ½ļæ½X8ļæ½ļæ½dČļæ½Kļæ½Ė~ļæ½fv|\fO ļæ½gļæ½ļæ½wļæ½ļæ½ļæ½gĀ Ā Ā ļæ½ļæ½3ļæ½7ļæ½uļæ½2Xļæ½Kļæ½..3eĀ Ā Ā ļæ½ ļæ½ļæ½:q=ļæ½.sIļæ½zM}ļæ½CBļæ½vÜ«
3qļæ½ļæ½ļæ½Ėļæ½ļæ½ļæ½Mļæ½ļæ½:ļæ½Gļæ½<ļæ½+Ł¤ļæ½"`'ļæ½Y^ļæ½ļæ½`Ā Ā Ā ļæ½'yfļæ½IpLmļæ½yļ£ļæ½ļæ½.\ļæ½ļæ½ļæ½5Uļæ½4ļæ½`ļæ½ļæ½ļæ½ļæ½ļæ½éøvfļæ½ļæ½ļæ½xĆ»ļæ½uvļæ½ļæ½ļæ½}ļæ½źļæ½u8ļæ½ļæ½+ļæ½1Wļæ½
Iļæ½ļæ½1ļæ½Ēøļæ½ļæ½
Ā Ā Ā Ā Ā Ā Ā Ā ļæ½[4ļæ½ļæ½wOļæ½6wļæ½ļæ½ļæ½pļæ½fwļæ½ļæ½ļæ½lļæ½ļæ½9ļæ½ļæ½ļæ½ļæ½ļæ½Iļæ½Ā Ā Ā ļæ½ļæ½ļæ½9ļæ½ļæ½ ļæ½ļæ½ļæ½ļæ½ļæ½6ļæ½ļæ½Dļæ½ļæ½@^Lrļæ½Yļæ½N2ļæ½ļæ½{ļæ½ļ°ļæ½ļæ½?:ļæ½Mļæ½ļæ½ļæ½ļæ½ļæ½Ź³
Ā Ā Ā >'2ļæ½hļæ½Jļæ½9
Well, that wasn’t helpful, wasn’t it? It’s in non-human readable format. This is where Z commands comes to rescue.
Use zcat to view compressed files
What you do is pretty much add z infront of your cat command. See example below:
root@kali:/var/log# zcat dmesg.1.gz | more [Ā Ā Ā 0.000000] Initializing cgroup subsys cpuset [Ā Ā Ā 0.000000] Initializing cgroup subsys cpu [Ā Ā Ā 0.000000] Initializing cgroup subsys cpuacct [Ā Ā Ā 0.000000] Linux version 3.14-kali1-amd64 (debian-kernel@lists.debian.org) (gcc version 4.7.2 (Debian 4.7.2-5) ) #1 SMP Debian 3.14.5-1kali1 (2014-06-07) [Ā Ā Ā 0.000000] Command line: BOOT_IMAGE=/boot/vmlinuz-3.14-kali1-amd64 root=UUID=9f174fa5-0c59-4024-b307-463b7bc1752d ro initrd=/install/gtk/initrd.gz quiet nouveau.modeset=0 [Ā Ā Ā 0.000000] e820: BIOS-provided physical RAM map:
That is so easy…
Using zcat to view files instead of cat
By this time, you are already getting the hang of it. But what happens if you want to view a regular (uncompressed file) using zcat?
root@kali:/var/log# zcat dmesg gzip: dmesg: not in gzip format
Well, that wasn’t very helpful. But there’s a way around that, just use a -f flag which will allow you to view any file compressed or not uzing zcat.
root@kali:/var/log# zcat -f dmesg | more [Ā Ā Ā 0.000000] Initializing cgroup subsys cpuset [Ā Ā Ā 0.000000] Initializing cgroup subsys cpu [Ā Ā Ā 0.000000] Initializing cgroup subsys cpuacct [Ā Ā Ā 0.000000] Linux version 3.14-kali1-amd64 (debian-kernel@lists.debian.org) (gcc version 4.7.2 (Debian 4.7.2-5) ) #1 SMP Debian 3.14.5-1kali1 (2014-06-07) [Ā Ā Ā 0.000000] Command line: BOOT_IMAGE=/boot/vmlinuz-3.14-kali1-amd64 root=UUID=9f174fa5-0c59-4024-b307-463b7bc1752d ro initrd=/install/gtk/initrd.gz quiet nouveau.modeset=0
Much better. The reason I am showing this is because now you can write a script to view files, search files irrespective of compression.
