Showing posts with label system load. Show all posts
Showing posts with label system load. Show all posts

Friday, August 19, 2011

Step 2: boot livedvd-amd64-multilib-11.2.iso from hard disk (with grub2)

After a few days experience of the gentoo livedvd run from usb, I think the usb drive should be saved for some other important data; I decided to copy the iso and run it from hard disk; usually a hard disk has large enough space, people could just copy all files inside /image.squashfs to a local partition, like the way on gentoo-wiki, but this computer assigned to me has a SSD drive default, has only 120G, furthermore, it's already partitioned long time ago, I don't have more space to back up all data on it and don't want to risk re-partition it; so I'd still like the way of loop mount iso on the fly, for saving disk space;
http://www.gentoo-wiki.info/HOWTO_Lightning_fast_install_from_LiveCD

/dev/sda3       240277504   250068991     4895744   83  Linux
/dev/sda5       129341440   132855807     1757184   82  Linux swap / Solaris
/dev/sda6       132857856   152387583     9764864   83  Linux LABEL=UBUNTU
/dev/sda7       152389632   210980863    29295616   83  Linux LABEL=HOME
/dev/sda8       210982912   211372031      194560   83  Linux LABEL=DEBIAN

The ubuntu partition already has grub2 default installed and home partition has a lot of remaining space, so I mkdir /gentoo under /home, put livedvd-amd64-multilib-11.2.iso there, rename or hardlink a short name gentoo.iso there; extract kernel/initramfs from the iso and put into /boot of the ubuntu partition, create a new entry there and no need to install another grub;

Notice that grub2 (version 1.9*) has slightly different syntax with grub1 (version 0.9*):
  1. disk partition order, counter from 1, same as Linux partitions;
  2. better not to change (append entries) in grub.cfg directly, use /boot/grub/custom.cfg instead; because every time later ubuntu upgrade or install new kernels would call update-grub2 that would overwrite grub.cfg; you could change /etc/grub/40_custom (that support script smart calculation) or /boot/grub/custom.cfg on that ubuntu partition; here I use custom.cfg;
Gentoo-11 ~ # cat /mnt/sda6/boot/grub/custom.cfg
menuentry "Gentoo11-Live Linux-3.0.0 (on /dev/sda7 {gentoo/gentoo.iso})" {
	  linux (hd0,6)/boot/gentoo/gentoo aufs_mem=2G cdroot=/dev/sda7 cdroot_type=ext4 isoboot=gentoo/gentoo.iso vga=791 splash=silent,theme:livecd-10 console=tty1 quiet nodetect doload=ahci dox
	  initrd (hd0,7)/gentoo/gentoo-initramfs.gz
}

Compared to previous menuentry in syslinux.cfg, here I removed some unused parameters (like "root=/dev/ram0 init=/linuxrc" because that is the default, I think I should recommend the Gentoo-Ten team to remove that, but not sure if any other computer need it, at least my computer doesn't need it, it could boot, as granted; the only useful parameters here are:
  1. aufs_mem=2G, it means how large of tmpfs layered on top of squashfs by aufs; default it's 420M tmpfs, enough if you don't modify/install too many applications in live mode;
  2. cdroot=/dev/sda7, specify a parameter saves parsing during booting; specify cdroot_type the same way;
  3. isoboot=gentoo/gentoo.iso, you need to specify where do you save that iso inside that partition; it's required;
  4. nodetect doload=ahci dox; depends on if you know your hardware, if no idea, autodetect is best, but I know my hardware, specifying it could save booting time;
The default gentoo kernel/initramfs should work this way I thought originally, but it doesn't, I have to look at the init script and change it a little, here is the diff: http://paste.pocoo.org/show/460334/

the initramfs is just cpio (newc format) + gz; extract and reconstruct in this way;
$ mkdir -vp gentoo-initramfs
$ zcat gentoo.igz |(cd gentoo-initramfs; cpio -d -i)
$ emacs -nw gentoo-initramfs/...; modify /init and /etc/init.scripts
$ (cd gentoo-initramfs/; find |cpio -H newc -o) |gzip -v9 >gentoo-initramfs.gz
Or
$ (cd gentoo-initramfs/; find |cpio -H newc -o) |xz -v9 >gentoo-initramfs.xz
Gentoo-11 ~ # (cd /home/gentoo/; ls -lh gentoo-initramfs.*)
-rw-r--r-- 1 root root 9.8M Aug 16 02:25 gentoo-initramfs.gz
-rw-r--r-- 1 root root 7.5M Aug 15 17:50 gentoo-initramfs.xz
The latest xz (aka. lzma2) could save 20% over "gzip -v9";

Update: use "xz --check=crc32" instead, because some latest xz support "none,crc32,crc64,sha256" or more types of integrity check, default as crc64; the kernel xz dec is "xz embedded" that only support "none or crc32",  kernel may report XZ_OPTIONS_ERROR, you could use "xz -vl ..." to check it, read Documentation/xz.txt for more;
/usr/src/linux-3.0-gentoo-r1/lib/decompress_unxz.c:
-=--:%%--F1  decompress_unxz.c   92% L364   (C/l Abbrev)

        case XZ_OPTIONS_ERROR:
                error("Input was encoded with settings that are not "
                                "supported by this XZ decoder");
                break;

# (cd gentoo-initramfs/; find |cpio -H newc -o) |xz --check=crc32 -v9 >../gentoo-initramfs.xz

Friday, August 29, 2008

monitor the system totally with dstat

You must have ever monitored the system with different tools:
  1. "vmstat 2" to get a total system "proc,mem,swap,io,sys,cpu" load;
  2. "iostat 2" to get all block devices with seperated I/O bandwidth data;
  3. "ifstat 2" to get different network interfaces network activity;
  4. "top" to get the most CPU-consuming processes;
Now you can do all these things within dstat:
gektop@tux ~ 0 $ dstat -M proc,cpu,mem,sys,page,net,disk,topcpu,topmem,topio,topbio 5
---procs--- ----total-cpu-usage---- ------memory-usage----- ---system-- ---paging-- -net/total- -dsk/total- -most-expensive- -most-expensive- ----most-expensive---- ----most-expensive----
run blk new|usr sys idl wai hiq siq| used buff cach free| int csw | in out | recv send| read writ| cpu process | memory process | i/o process | block i/o process
0 0 6| 39 9 37 14 0 1| 272M 25M 193M 12M| 760 2178 | 0 21B| 0 0 |5577k 1645k|firefox 10|firefox 125M|mount.ntfs- 10M:8223k|mount.ntfs-2699k: 736k
3 0 8| 93 6 0 0 0 1| 252M 25M 189M 37M| 546 980 | 0 0 |3197B 564B| 109k 675k|firefox 15|firefox 125M|mount.ntfs- 771k:3056B|pdflush 0 : 82k
3 0 71| 77 16 0 7 0 1| 257M 26M 191M 29M| 548 1247 | 0 0 |5445B 689B| 230k 544k|firefox 13|firefox 125M|mount.ntfs- 771k:3056B|pdflush 0 : 94k
2 0 24| 85 14 0 1 0 0| 251M 26M 192M 33M| 550 1046 | 0 0 |1173B 418B| 50k 344k|firefox 12|firefox 125M|mount.ntfs- 771k:3056B|pdflush 0 : 85k
2 0 25| 85 12 0 2 0 0| 275M 26M 194M 8044k| 517 1024 | 0 0 |1206B 448B| 104k 642k|cc1 22|firefox 125M|mount.ntfs- 771k:3056B|pdflush 0 : 47k
2 0 9| 88 9 0 2 0 1| 278M 26M 193M 5568k| 558 1056 | 0 0 |1383B 615B| 110k 451k|cc1 26|firefox 125M|mount.ntfs- 771k:3056B|pdflush 0 : 51k
The meanings of "proc,cpu,mem,sys,page,net,disk,topcpu,topmem,topio,topbio" is obvious, it speaks for itself, the design of dstat is all modular, you can use many other modules than this:
$ (cd /usr/share/dstat/; ls *.py)
dstat_battery.py dstat_gpfs.py dstat_mysql5_com.py dstat_mysql_keys.py dstat_postfix.py dstat_thermal.py dstat_topoom.py dstat_vzcpu.py
dstat_cpufreq.py dstat_helloworld.py dstat_mysql5_conn.py dstat_nfs3op.py dstat.py dstat_topbio.py dstat_utmp.py dstat_vzubc.py
dstat_dbus.py dstat_innodb_buffer.py dstat_mysql5_io.py dstat_nfs3.py dstat_rpcd.py dstat_topcpu.py dstat_vmkhba.py dstat_wifi.py
dstat_freespace.py dstat_innodb_io.py dstat_mysql5_keys.py dstat_nfsd3op.py dstat_rpc.py dstat_topio.py dstat_vmkint.py
dstat_gpfsop.py dstat_innodb_ops.py dstat_mysql_io.py dstat_nfsd3.py dstat_sendmail.py dstat_topmem.py dstat_vmknic.py
So with "dstat -M battery 5", you can get the most battery consuming components when hardware supports ACPI.

Its module is just a simple py file with a simple class derived from dstat, so if there's no one fit for you, you can write your own dstat module.

Note: only CLI-based utils discussed here, I know there are many GUI-based or graph-generating tools can do better graphics, but on ther server side, "
Simple is better than complex.
"