Blog O' Matty


pbcopy / pbpaste in OS X

This article was posted by Matty on 2009-02-13 13:34:00 -0400 -0400

I came across a nifty utility in OS X that allows you to copy / paste data to/from the clipboard without having to select text and command+c/command+v

Michael-MacBook-Pro:~ (michael)> echo foo | pbcopy

Michael-MacBook-Pro:~ (michael)> pbpaste foo

Thats kind of neat.  What about connecting to a remote machine, executing some command, and then having that output in the clipboard of your local machine?

Michael-MacBook-Pro:~ (michael)> ssh somehost.com uptime | pbcopy

Michael-MacBook-Pro:~ (michael)> pbpaste 5:26pm  up 64 day(s),  8:30,  4 users,  load average: 0.04, 0.04, 0.04

Easily encoding documents prior to publishing them to the web

This article was posted by Matty on 2009-02-13 12:13:00 -0400 -0400

While reviewing the command lines on commandlinefu, I came across this nifty little gem:

$ perl -MHTML::Entities -ne 'print encode_entities(_)' FILETOENCODE

This command line snippet takes a file as an argument, and escapes all of the characters that can’t be directly published on the web (i.e., it will convert a right bracket to & gt). This is super useful, and is a welcome addition to the nifty tidy utility!

ZFS dataset and volume properties

This article was posted by Matty on 2009-02-10 09:46:00 -0400 -0400

A  few chapters of the upcoming OpenSolaris Bible have been released.  Specifically, looking through chapter 8 on ZFS, I came across this handy list of properties and their descriptions.

***File System Properties***
aclinherit-- Inheritance of ACL entries
aclmode-- Modification of ACLs in a chmod(2) operation
atime-- Whether access times of files are updated when read
available-- Space available to the file system
canmount-- Whether the file system is mountable
casesensitivity-- Case sensitivity of filename matching
checksum-- Checksum algorithm for data integrity
compression-- Compression algorithm
compressratio-- Compression ratio achieved
copies-- Number of data copies stored
creation-- Time the file system was created
devices-- Whether device nodes can be opened
exec-- Whether processes can be executed
mounted-- Whether the file system is mounted
mountpoint-- Mount point for the file system
nbmand-- Use of nonblocking mandatory locks with CIFS
normalization-- Use Unicode-normalized filenames in name comparisons
origin-- Snapshot on which a clone is based
primarycache-- Controls whether ZFS data and metadata are cached in the
primary cache
quota-- Limit on space that the file system can consume
readonly-- Whether the file system can be modified
recordsize-- Suggested block size for files
referenced-- Amount of data accessible within the file system
refquota-- Space limit for this file system
refreservation-- Minimum space guaranteed to the file system
reservation-- Minimum space guaranteed to the file system and
descendants
secondarycache-- Controls whether ZFS data and metadata are cached in
the secondary cache
setuid-- Allow setuid file execution
shareiscsi-- Export volumes within the file system as iSCSI targets
sharenfs-- Share the file system via NFS
sharesmb-- Share the file system via CIFS
snapdir-- Whether the .zfs directory is visible
type-- Type of dataset
used-- Space consumed by the file system and descendants
usedbychildren-- Space freed if children of the file system were
destroyed
usedbydataset-- Space freed if snapshots and refreservation were
destroyed, and contents of the file system were deleted
usedbyrefreservation-- Space freed if the refreservation was removed
usedbysnapshots-- Space freed if all snapshots of the file system were
destroyed
utf8only-- Use only UTF-8 character set for filenames
version-- On-disk version of the file system
vscan-- Whether to scan regular files for viruses
xattr-- Whether extended attributes are enabled
zoned-- Whether the file system is managed from a nonglobal zone

***Volume Properties***
available-- Space available to the volume
checksum-- Checksum algorithm for data integrity
compression-- Compression algorithm
compressratio-- Compression ratio achieved
copies-- Number of data copies stored
creation-- Time the volume was created
origin-- Snapshot on which the clone is based
primarycache-- Controls whether ZFS data and metadata are cached in the
primary cache
readonly-- Whether the volume can be modified
referenced-- Amount of data accessible within the volume
refreservation-- Minimum space guaranteed to the volume
reservation-- Minimum space guaranteed to the volume and descendants
secondarycache-- Controls whether ZFS data and metadata are cached in
the secondary cache
shareiscsi-- Export the volume as an iSCSI target
type-- Type of dataset
used-- Space consumed by the volume and descendants
usedbychildren-- Space freed if children of the volume were destroyed
usedbydataset-- Space freed if snapshots and refreservation were
destroyed, and contents of the volume were deleted
usedbyrefreservation-- Space freed if the refreservation was removed
usedbysnapshots-- Space freed if all snapshots of the volume were
destroyed
volblocksize-- Block size of the volume
volsize-- Logical size of the volume

Some of these properties were new to me, as they probably only exist in later versions of ZFS in OpenSolaris.   Specifically, vscan, to scan files for viruses is interesting.  I’m wondering where virus definations are stored and updated.  This is actually a pretty nifty feature if you plan on using Solaris’ new in-kernel SMB server to share data with Microsoft Windows based clients.

UPDATE:  Richard provided via a commentan awesome link that shows how to administrate the new CIFS server within OpenSolaris, as well as how to execute the virus scan using the vscanadm utility.  Take a look at the slides on that link for an in-depth administrative tour of these features.

I’d like to learn more about the primarycache and secondarycache settings – exactly what gets tuned when fiddling around with these.

Also, a property called “copies” which allows you to specifiy how many copies of the data should be kept on disk.   I’m not sure exactly why you would want to increase the number of copies of data instead of using raidz, raidz2, mirroring, hot spares, etc. but its neat that the option is there.

Adding environment variables to an SMF service

This article was posted by Matty on 2009-02-06 11:26:00 -0400 -0400

I create an SMF manifest last week for a new service, and needed two environment variables to be set prior to the services’ start method being invoked. After poking around the svccfg manual page, I came across the setenv subcommand, which can be used to set an environment variable:

$ svccfg -s tinydns setenv FOO bar

Since I was creating a manifest from scratch, I ran this against a temporary service and then used the svccfg “export” command to display the entry I needed to add to the manifest I was creating:

$ svccfg export tinydns

Here is the XML that was displayed for the environment variable addition:

<exec_method name='start' type='method' exec='/lib/svc/method/tinydns
start' timeout_seconds='60'>
<method_context>
<method_environment>
<envvar name='FOO' value='bar'/>
</method_environment>
</method_context>
/exec_method>

I like SMF, but sometimes wonder if it is over-engineered for what it actually needs to do. If you have opinions on this, please let me know by submitting a comment.

Figuring out if a dedicated ZFS intent log will help

This article was posted by Matty on 2009-02-06 10:31:00 -0400 -0400

ZFS uses the ZFS intent log (also referred to as a ZIL) to store synchronous writes. This has the advantage that a full transaction group doesn’t need to be written when a synchronous write occurs, and maximizes the use of I/O bandwidth. For some applications (databases come to mind), placing the ZIL on a dedicated device can be extremely helpful. But how can you tell if placing the intent log on a separate device would be useful? To answer this question, you can run Richard Elling’s super handy zilstat DTrace script. As SSD becomes cheaper and cheaper, this script will be a useful tool for folks who want to applications and database performance.