One cool ZFS feature that was introduced in Solaris 10 11/06 is the ability to clone a file system. This is a truly powerful tool, since it allows admins to create copies of file systems prior to performing upgrades, and a way to roll a file system back to a previous state (this provides a simple lightweight method to back out changes if upgrades go awry). To illustrated just how useful this is, let’s create a clone of the file system oradata. To clone oradata, we first need to take a sapshot of the oradata file system:

$ zfs snapshot striped/oradata@ordata_snap

Once a snapshot is created, the zfs utilities’ “clone” option can be used to create a clone of the file system from the snaphost we just created:

$ zfs clone striped/oradata@ordata_snap striped/oradata_clone

After the clone is created, it can be “promoted” to a file system with the zfs utilities’ “promote” option:

$ zfs promote striped/oradata_clone

If everything works correctly (which it should), the clone and the file system the clone was created from will appear in the df output:

$ df -h

Filesystem             size   used  avail capacity  Mounted on
/dev/dsk/c0d0s0         17G   2.9G    14G    18%    /
/devices                 0K     0K     0K     0%    /devices
/dev                     0K     0K     0K     0%    /dev
ctfs                     0K     0K     0K     0%    /system/contract
proc                     0K     0K     0K     0%    /proc
mnttab                   0K     0K     0K     0%    /etc/mnttab
swap                  1016M   768K  1015M     1%    /etc/svc/volatile
objfs                    0K     0K     0K     0%    /system/object
/usr/lib/libc/libc_hwcap1.so.1
                        17G   2.9G    14G    18%    /lib/libc.so.1
fd                       0K     0K     0K     0%    /dev/fd
swap                  1015M     0K  1015M     0%    /tmp
swap                  1015M    24K  1015M     1%    /var/run
striped                 52G    27K    50G     1%    /striped
striped/oradata         52G   300M    50G     1%    /striped/oradata
striped/oradata_clone
                        52G   300M    50G     1%    /striped/oradata_clone

Now let’s pretend that a DBA accidentally deletes the Oracle datafiles in /striped/oradata:

$ rm -f /striped/oradata/*

Oops! Since oradata_clone is a clone of the oradata file system, we can restore oradata by first deleting the original file system (you could also rename it if you wanted to keep it around):

$ zfs destroy -f striped/oradata

And then renaming the clone to take it’s place:

$ zfs rename striped/oradata_clone striped/oradata

Once the rename occurs, everything is as it once was:

$ df -h

Filesystem             size   used  avail capacity  Mounted on
/dev/dsk/c0d0s0         17G   2.9G    14G    18%    /
/devices                 0K     0K     0K     0%    /devices
/dev                     0K     0K     0K     0%    /dev
ctfs                     0K     0K     0K     0%    /system/contract
proc                     0K     0K     0K     0%    /proc
mnttab                   0K     0K     0K     0%    /etc/mnttab
swap                  1015M   768K  1014M     1%    /etc/svc/volatile
objfs                    0K     0K     0K     0%    /system/object
/usr/lib/libc/libc_hwcap1.so.1
                        17G   2.9G    14G    18%    /lib/libc.so.1
fd                       0K     0K     0K     0%    /dev/fd
swap                  1014M     0K  1014M     0%    /tmp
swap                  1014M    24K  1014M     1%    /var/run
striped                 52G    26K    50G     1%    /striped
striped/oradata         52G   300M    50G     1%    /striped/oradata

$ ls -la /striped/oradata

total 614555
drwxr-xr-x   2 root     sys            5 Dec 24 11:49 .
drwxr-xr-x   3 root     sys            3 Dec 24 11:59 ..
-rw-r--r--   1 root     root     104857600 Dec 24 11:47 iklts01.dbf
-rw-r--r--   1 root     root     104857600 Dec 24 11:48 iklts02.dbf
-rw-r--r--   1 root     root     104857600 Dec 24 11:49 iklts03.dbf

This has so many uses, and once ZFS boot is incorporated, using clones to protect yourself from rogue patches will be one of the coolest things ever! Shibby!

Posted by matty, filed under Solaris ZFS. Date: December 29, 2006, 4:34 pm | No Comments »

As with most individuals that manage a blog, I get inundated with comment spam. Based on some research I did, it looks like almost all of the blog spam occurs for older posts, especially those over 30 days old. Based on this information, I decided to disable comments for all old posts older than 30 days. It turns out that you can’t do this directly from wordpress without a plugin, so I decided to adjust the “comment_status” column in the wp_posts table directly. To remove comment spam for all posts older than 30 days, I first used the GNU date utility to find the date 30 days ago:

$ date –date=”30 days ago” “+%Y-%m-%d”
2006-11-29

Once I had the date string from 30 days ago, I connected to the MySQL database that hosts my blog, and ran the following SQL query to disable posts for all entries greater than the date returned form GNU date:

$ mysql -umatty -p -h mysql.prefetch.net
Enter password:
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 16155903 to server version: 4.0.27-standard-log

Type ‘help;’ or ‘\h’ for help. Type ‘\c’ to clear the buffer.

mysql> use matty
Database changed

mysql> UPDATE wp_posts SET comment_status = ‘closed’ WHERE post_date < '2006-11-29' AND post_status = 'publish'

mysql> quit
Bye

Now that old posts don’t have comments enabled, I am curious to see how the comment spammers react.

Posted by matty, filed under Wordpress. Date: December 29, 2006, 4:20 pm | 4 Comments »

When scripts executing from cron write to stdout or stderr, the results are emailed to the user the job is running in the context of. One way to control where the mail is sent is by adding a “MAILTO” line to the top of the crontab entry for the user that is running the cron job. Here is an example:

$ crontab -l
MAILTO=matty
* * * * * echo “hi”

Each time this job runs, the user matty will get emailed with the results of the script. You can also use /etc/aliases to assist with this on a global scale, but sometimes that isn’t an option.

Posted by matty, filed under Linux Utilities. Date: December 29, 2006, 2:40 am | No Comments »

One of my friends recently asked me how to verify a signature that is distributed with an opensource application. Since I didn’t have a machine handy to show him, I thought I would jot this down for him in my blog. The first step in verifying a signature requires locating the public key of the individual who signed the binary. Once you locate the key and validate it’s authenticity, you can use the gpg “–import” option to import the key. Once the key is imported, you can use the gpg “–verify” option to verify that the signature is legit:

$ gpg –verify postfix-2.3.2.tar.gz.sig postfix-2.3.2.tar.gz

gpg: WARNING: using insecure memory!
gpg: please see http://www.gnupg.org/faq.html for more information
gpg: Signature made Thu Jul 27 12:49:07 2006 EDT using RSA key ID C12BCD99
gpg: Good signature from "Wietse Venema ”
gpg: WARNING: This key is not certified with a trusted signature!
gpg:          There is no indication that the signature belongs to the owner.
Primary key fingerprint: FF 96 4A 8C 96 88 7C 6E  A4 EF AD BF 48 34 E1 BB

In the above example, I used Wietse Venema’s public key (which I had previously imported) to verify the signature stored in the file postfix-2.3.2.tar.gz.sig against the source code I downloaded. Gpg is a swell piece of software!

Posted by matty, filed under GNU Privacy Guard. Date: December 27, 2006, 12:38 pm | No Comments »

To ensure that my CentOS machines run as efficiently and securely as possible, I disable a number of services after each installation. The end result is a system that accepts ssh connections on TCP port 22, and on one or more service ports that are in use by the applications hosted on the platform. To get to this state, I go through and disable numerous services that come enabled by default. Here is the default list of services that are enabled after a CentOS 4.4 installation:

$ chkconfig –list | grep on

atd             0:off   1:off   2:off   3:on    4:on    5:on    6:off
messagebus      0:off   1:off   2:off   3:on    4:on    5:on    6:off
smartd          0:off   1:off   2:on    3:on    4:on    5:on    6:off
portmap         0:off   1:off   2:off   3:on    4:on    5:on    6:off
sendmail        0:off   1:off   2:on    3:on    4:on    5:on    6:off
netfs           0:off   1:off   2:off   3:on    4:on    5:on    6:off
cups            0:off   1:off   2:on    3:on    4:on    5:on    6:off
irqbalance      0:off   1:off   2:off   3:on    4:on    5:on    6:off
rpcgssd         0:off   1:off   2:off   3:on    4:on    5:on    6:off
xfs             0:off   1:off   2:on    3:on    4:on    5:on    6:off
isdn            0:off   1:off   2:on    3:on    4:on    5:on    6:off
autofs          0:off   1:off   2:off   3:on    4:on    5:on    6:off
gpm             0:off   1:off   2:on    3:on    4:on    5:on    6:off
apmd            0:off   1:off   2:on    3:on    4:on    5:on    6:off
crond           0:off   1:off   2:on    3:on    4:on    5:on    6:off
acpid           0:off   1:off   2:off   3:on    4:on    5:on    6:off
microcode_ctl   0:off   1:off   2:on    3:on    4:on    5:on    6:off
pcmcia          0:off   1:off   2:on    3:on    4:on    5:on    6:off
cpuspeed        0:off   1:on    2:on    3:on    4:on    5:on    6:off
xinetd          0:off   1:off   2:off   3:on    4:on    5:on    6:off
rpcidmapd       0:off   1:off   2:off   3:on    4:on    5:on    6:off
readahead_early 0:off   1:off   2:off   3:off   4:off   5:on    6:off
readahead       0:off   1:off   2:off   3:off   4:off   5:on    6:off
sshd            0:off   1:off   2:on    3:on    4:on    5:on    6:off
anacron         0:off   1:off   2:on    3:on    4:on    5:on    6:off
network         0:off   1:off   2:on    3:on    4:on    5:on    6:off
kudzu           0:off   1:off   2:off   3:on    4:on    5:on    6:off
syslog          0:off   1:off   2:on    3:on    4:on    5:on    6:off
nfslock         0:off   1:off   2:off   3:on    4:on    5:on    6:off
rawdevices      0:off   1:off   2:off   3:on    4:on    5:on    6:off
mdmonitor       0:off   1:off   2:on    3:on    4:on    5:on    6:off
haldaemon       0:off   1:off   2:off   3:on    4:on    5:on    6:off

Several of these services are required, but several others serve no purpose in my environment, and use CPU and memory resources that would be better allocated to my applications. Since I don’t use RPC services, autofs or NFS, those are the first to get disabled:

$ /sbin/chkconfig –level 0123456 portmap off
$ /sbin/chkconfig –level 0123456 nfslock off
$ /sbin/chkconfig –level 0123456 netfs off
$ /sbin/chkconfig –level 0123456 rpcgssd off
$ /sbin/chkconfig –level 0123456 rpcidmapd off
$ /sbin/chkconfig –level 0123456 autofs off

I also don’t allow individual hosts to receive mail from the outside world, so sendmail gets nixed next:

$ /sbin/chkconfig –level 0123456 sendmail off

On server platforms, who needs printing?:

$ /sbin/chkconfig –level 0123456 cups off

Now we get to the font server, isdn capabilities, console mouse and pcmcia support. I don’t use these services on my servers, so they get disabled as well:

$ /sbin/chkconfig –level 0123456 xfs off
$ /sbin/chkconfig –level 0123456 isdn off
$ /sbin/chkconfig –level 0123456 gpm off
$ /sbin/chkconfig –level 0123456 pcmcia off

Once these services are disabled (and optionally stopped with the service command or a reboot), my netstat output looks nice and clean:

$ netstat –tcp –udp –listening

Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address               Foreign Address             State
tcp        0      0 *:ssh                       *:*                         LISTEN

This has served me well over the years, since it reduces boot time (less rc scripts to run), and frees up additional resources for my applications (while this isn’t substantial, every page of memory helps!).

Posted by matty, filed under Linux Security. Date: December 27, 2006, 12:28 pm | 2 Comments »

While reading up on the Linux parted utility, I came across partprobe. Partprobe can be used to display the number of partitions on a device, and is useful for quickly viewing the layout of a given device:

$ /sbin/partprobe -s /dev/hda
/dev/hda: msdos partitions 1 2

If your looking for details on the layout of a device, I reckon fdisk is a better bet:

$ /sbin/fdisk -l /dev/hda

Disk /dev/hda: 8590 MB, 8590417920 bytes
255 heads, 63 sectors/track, 1044 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

   Device Boot      Start         End      Blocks   Id  System
/dev/hda1   *           1         978     7855753+  83  Linux
/dev/hda2             979        1043      522112+  82  Linux swap

I really dig the fdisk “-l” option!

Posted by matty, filed under Linux Utilities. Date: December 26, 2006, 9:15 pm | No Comments »

I needed to expand a ZFS pool from a single disk to a pair of disks today. To expand my pool named “striped,” I ran zpool with the “add” option, the pool name to add the disk to, and the device to add to the pool:

$ zpool add striped c1d1

Once the disk was added to the pool, it was immediately available for use:

$ zpool status -v

  pool: striped
 state: ONLINE
 scrub: none requested
config:

        NAME        STATE     READ WRITE CKSUM
        striped    ONLINE       0     0     0
          c1d0      ONLINE       0     0     0
          c1d1      ONLINE       0     0     0

errors: No known data errors

I used to think Veritas had the easiest method to expand file systems, but I don’t think that is the case anymore. Now if we can just get Sun to allow us remove devices from a pool, and expand the number of columns in a RAIDZ or RAIDZ2 vdev!

Posted by matty, filed under Solaris ZFS. Date: December 26, 2006, 8:59 pm | 5 Comments »

I periodically need to review netstat data to debug network problems, and prefer to view the deltas between two adjacent runs. The Solaris netstat utility can be passed a time interval, which will cause it to display the difference between two runs:

$ netstat -sP tcp 10

TCP     tcpRtoAlgorithm     =     4     tcpRtoMin           =   400
        tcpRtoMax           = 60000     tcpMaxConn          =    -1
        tcpActiveOpens      =    20     tcpPassiveOpens     =    50
        tcpAttemptFails     =    20     tcpEstabResets      =     0
        tcpCurrEstab        =     4     tcpOutSegs          =859323
        tcpOutDataSegs      =847791     tcpOutDataBytes     =1159286456
        tcpRetransSegs      =    19     tcpRetransBytes     = 13388
        tcpOutAck           = 11532     tcpOutAckDelayed    =   620
        tcpOutUrg           =     0     tcpOutWinUpdate     =    96
        tcpOutWinProbe      =     0     tcpOutControl       =   136
        tcpOutRsts          =    20     tcpOutFastRetrans   =     0
        tcpInSegs           =186702
        tcpInAckSegs        =103393     tcpInAckBytes       =1159286502
        tcpInDupAck         =   229     tcpInAckUnsent      =     0
        tcpInInorderSegs    = 83753     tcpInInorderBytes   =79048288
        tcpInUnorderSegs    =     9     tcpInUnorderBytes   =   432
        tcpInDupSegs        =     8     tcpInDupBytes       =  4664
        tcpInPartDupSegs    =     0     tcpInPartDupBytes   =     0
        tcpInPastWinSegs    =     0     tcpInPastWinBytes   =     0
        tcpInWinProbe       =     0     tcpInWinUpdate      =     0
        tcpInClosed         =     0     tcpRttNoUpdate      =     0
        tcpRttUpdate        =103347     tcpTimRetrans       =   109
        tcpTimRetransDrop   =     0     tcpTimKeepalive     =    61
        tcpTimKeepaliveProbe=    15     tcpTimKeepaliveDrop =     0
        tcpListenDrop       =     0     tcpListenDropQ0     =     0
        tcpHalfOpenDrop     =     0     tcpOutSackRetrans   =     7

TCP     tcpRtoAlgorithm     =     0     tcpRtoMin           =   400
        tcpRtoMax           = 60000     tcpMaxConn          =    -1
        tcpActiveOpens      =     0     tcpPassiveOpens     =     0
        tcpAttemptFails     =     0     tcpEstabResets      =     0
        tcpCurrEstab        =     4     tcpOutSegs          =    83
        tcpOutDataSegs      =    83     tcpOutDataBytes     =  6544
        tcpRetransSegs      =     0     tcpRetransBytes     =     0
        tcpOutAck           =     0     tcpOutAckDelayed    =     0
        tcpOutUrg           =     0     tcpOutWinUpdate     =     0
        tcpOutWinProbe      =     0     tcpOutControl       =     0
        tcpOutRsts          =     0     tcpOutFastRetrans   =     0
        tcpInSegs           =    78
        tcpInAckSegs        =    76     tcpInAckBytes       =  6544
        tcpInDupAck         =     0     tcpInAckUnsent      =     0
        tcpInInorderSegs    =     2     tcpInInorderBytes   =    96
        tcpInUnorderSegs    =     0     tcpInUnorderBytes   =     0
        tcpInDupSegs        =     0     tcpInDupBytes       =     0
        tcpInPartDupSegs    =     0     tcpInPartDupBytes   =     0
        tcpInPastWinSegs    =     0     tcpInPastWinBytes   =     0
        tcpInWinProbe       =     0     tcpInWinUpdate      =     0
        tcpInClosed         =     0     tcpRttNoUpdate      =     0
        tcpRttUpdate        =    76     tcpTimRetrans       =     0
        tcpTimRetransDrop   =     0     tcpTimKeepalive     =     0
        tcpTimKeepaliveProbe=     0     tcpTimKeepaliveDrop =     0
        tcpListenDrop       =     0     tcpListenDropQ0     =     0
        tcpHalfOpenDrop     =     0     tcpOutSackRetrans   =     0

Netstat is some good stuff!

Posted by matty, filed under Solaris Utilities. Date: December 26, 2006, 8:53 pm | No Comments »

« Previous Entries