Last year I did a number of concert reviews, but not nearly as many as I would have liked to. This year I plan to review 2 - 4 shows per month, and the acts will range from small local bands to large national headliners. I am also hoping to get peoples feedback from the shows, and take pictures if the venue allows them. So to start off 2009, I thought I would share my experiences from the Drivin’ N Cryin’ show I attended a few weeks back.
The show was scheduled for a Friday night at a relatively small local venue, and I decided to venture out by myself at the last minute to attend. Once I arrived at the venue, I grabbed a drink from the bar and then proceeded to chat with a bunch of folks who were attending the show. While in the middle of a conversation with a hard core Drivin’ n Cryin’ fan, I heard music off in the distance and realized it was time to make my way into the venue to take in the show.
The show started off with a bang, and I was amazed with Kevn Kinney’s vocals. He had such a way with his delivery, and I am still in awe at how well the guys in the band jived with each other musically. Since this was my first DNC show, I’m not 100% certain what the set list was for the night. I can tell you that my favorite tunes were their huge hit “Straight To Hell”, “Fly Me Courageous” as well as their rendition of Neil Young’s “Rockin’ In The Free World”.
I have to say this was one of the better concert experiences I have ever had, and the folks who were in attendance LOVE this band (seriously, they LOVE them!). Everyone was crazy nice at the show, and I made a couple of new new friends while chatting with random folks at the venue (hi Kelly!). Good times were had, and here’s to some killer shows in 2009! Happy New Years everyone!
Most *NIX implementations ship with a syslog daemon, which is responsible for reading messages from clients and writing these messages to one or more log destinations. Most daemons perform this operation synchronously using the fsync() system call, which can lead to some headaches on super busy log servers. Linux has a nifty option to disable synchronous updates, which can really aide in scaling logging servers to handle lots and lots of clients. To disable synchronous updates (this can result in data loss, so make sure to evaluate all options prior to using this feature), you can prefix the log destination with a hyphen like so:
$ grep daemon /etc/syslog.conf
daemon.* -/var/log/messages
This is super useful, and I wish there was a similar feature available out of the box for Solaris (syslog-ng can do this, but you shouldn’t have to install another syslog daemon to get this functionality IMHO).
Slashdot picked up a youtube video of Brendan Gregg, shouting at a Sun 7410. Its pretty halarious, but it does showcase just how cool the DTrace Analytics visual GUI is on Sun’s new line of Amber Road products.
Brendan is one of the co-authors of Solaris Internals, 2nd Edition. He also wrote the DTraceToolKit, which has allowed millions of SysAdmins who don’t have the skills to deep dive into DTrace to harness its powa!
I came across this neat little tidbit on page 27 while reading through the pdf article *UNDERSTANDING THE SECURITY CAPABILITIES OF SOLARIS™ ZONES SOFTWARE
As a test, I’m going to set this resource control on a zone and execute a fork bombto see what appears in system logs. This is pretty cool stuff!
Miscellaneous Controls
One well-known method to over-consume system resources is a fork-bomb.
This method does not necessarily consume a great deal of memory or CPU
resources, but rather seeks to use up all of the process slots in the
kernel’s process table. In the Solaris OS, a running process starts with
just one thread of execution, also called a Light Weight Process (LWP).
Many programs generate new threads, becoming multithreaded processes. By
default, Solaris systems with a 64-bit kernel can run over 85,000 LWPs
simultaneously. A booted zone that is not yet running any applications
has approximately 100 to 150 LWPs. To prevent a zone from using too many
LWPs, a limit can be set on their use. The following command sets a
limit of 300 LWPs for a zone.
global# zonecfg -z web
zonecfg:web> set max-lwps=300
zonecfg:web> exit
global# zoneadm -z web reboot
This parameter can be used, but should not be set so low that it impacts normal application operation. An accurate baseline for the number of LWPs for a given zone should be determined in order to set this valuable at an appropriate level. The number of LWPs used by a zone can be monitored using the following prstat command. In this example, the web zone currently has 108 LWPs. This value changes as processes are created or exit. It should be inspected over a period of time in order to establish a more reliable baseline, and updated when the software, requirements, or workload change. Using the max-lwps resource control successfully usually requires the use of a CPU control, such as the FSS or pools to ensure that there is enough CPU power in the global zone for the platform administrator to fix any problems that might arise.
global# prstat -LZ […] ZONEID NLWP SWAP RSS MEMORY TIME CPU ZONE 0 248 468M 521M 8.6% 0:14:18 0.0% global 37 108 76M 61M 1.0% 0:00:00 0.0% web Total: 122 processes, 356 lwps, load averages: 0.00, 0.00, 0.01
The inability to remove devices from ZFS Zpools has been one of the most annoying / inflexiable things about ZFS. I once read a blogpost about somenone who added a USB flash stick into the root ZFS pool, and now the USB stick became a perminent fixture of the machine! There was no simple way to fix this issue other than backup / network dump / rebuild machine / network restore.
Matthew Ahrens workedon changing the ZFS scrub code in SNV 94 (which made its way into the Fishwork’s Amber Road NAS Appliance), but this work will also lay the groundwork for “zpool evict” or the such. This is a great read on the internal working of how blockpointers are used within ZFS.