All good things (opensolaris) must come to an end

This past weekend I unsubscribed from my last opensolaris mailing list. While reflecting on where technology is heading, I had to take a few minutes to reflect on where things were just a few years back. I remember vividly the day that the website came online. After the announcement came out, I spent 24 straight hours signing up for mailing lists, reading documentation and reviewing the source code for a number of utilities. This had been all too easy with Linux, since all of the code and documentation was available in the public domain. But when the Solaris source came online, I felt like a 4-year old in a HUGE candy store.

Over the next few months I saw the community start to grow at a decent pace. The first opensolaris books (OpenSolaris Bible and Pro Opensolaris) were published, Solaris internals was updated to take Solaris 10 and opensolaris into account and every major trade magazine was writing something about opensolaris. Additionally, our local OpenSolaris users group was starting to grow in size, and I was beginning to make a number of good friends in the community. All of these things got me crazy excited about the opensolaris community, and I wanted to jump in and start helping out any way I could.

After pondering all of the things I wanted in opensolaris, I came up with a simple change that would allow me to get familiar with the development model. The change I proposed and coded up would allow dd to print the status of the copy operation if a SIGUSR1 signal was received. This feature was available on my FreeBSD and Linux boxes, so I wanted to see it on my Solaris hosts as well. I went through the hassle of filling out a form to submit code and then I sent my changes over to my sponsor. He replied stating that he would look things over and get back to me. That was the last time I heard from him and my follow up e-mail didn’t receive a response either.

I am not the type of person to pester someone to do something, so I didn’t sent another e-mail to request status. This of course let to my proposal dying a silent death. :( This was the first thing that led me to wonder if opensolaris would truly flourish, since all of the source code enhancements I had made to other projects were added back within days (and usually the authors were grateful). Since I knew contributing code was most likely not going to work, I decided to be active on the forums and propose changes that would better Solaris. This is when I started to get the impression that most of the design and development was happening behind closed doors, and not out in the open. Linux has prided itself on openness when it comes to design and development, so once again I started to question whether opensolaris would flourish.

So fast forward to the recent announcement by Oracle that opensolaris design and development would not be happening out in the open. In my opinion this never really occurred in the past, so I wasn’t one bit surprised by this announcement. They want to capitalize on the product (Solaris) they bought, and I can’t really fault them for that. Some people appear to have been caught off guard by this announcement, but the second Oracle bought Sun I figured open development would most likely stop. My only remaining question was what would happen to Solaris? Will Oracle eventually scrap it in favor of Linux? The cost to support two operating systems has to be relatively large, and I have to assume that there are some folks at Oracle who are evaluating this.

The Oracle announcement appears to have stirred some things up, and a number of new things came about as a result of it. The Illumos project was erected with it’s goal of making opensolaris development open. While this is a great idea in theory, I’m skeptical that the project can truly succeed without Sun/Oracle engineering. The amount of code in Opensolaris is rather large, and I have to assume that you would need an army of engineers to design, develop and QA everything to make it battle ready. I truly wish this project the best, and hope it gets the momentum it needs to succeed (Garrett D’Amore is a sharp dude, so the source is definitely in good hands!).

About a year ago I ditched Solaris in favor of Redhat Linux, which appears to be a growing trend amongst my SysAdmin friends. I like that Linux development is truly open, and the distributions I use (RHEL, CentOS and Fedora) provide the source code to the entire Operating System. The Linux distributions I use also have a large number of users, so getting answers to support or configuration issues is typically pretty easy to do. There is also the fact that the source is available, so I can support myself if no one happens to know why something is behaving a specific way.

This post wasn’t meant to diss Solaris, OpenSolaris or Illuminos. I was purely reflecting on the road I’ve traveled prior to embracing Linux and giving up hope in the opensolaris community. Hopefully one day Oracle will make all of the awesome Solaris features (DTrace, ZFS, Zones, Crossbow, FMA) available to the Linux community by slapping a GPLv2 license on the source code. I would love nothing more that to have all of the things I love about Linux merged with the things I love about Solaris. This would be a true panacea as far as Operating Systems go! :)

25 thoughts on “All good things (opensolaris) must come to an end”

  1. “About a year ago I ditched Solaris in favor of Redhat Linux, which appears to be a growing trend amongst my SysAdmin friends. I like that Linux development is truly open, and the distributions I use (RHEL, CentOS and Fedora) provide the source code to the entire Operating System.”

    Trends != brains.
    What “open development”? I challenge you openly to get your patch included in RHEL. Ditto for CentOS!

    For instance, take a look at redhat’s bug database, search for Anaconda and XFS related bugs, patches, requests for enhancements… ZILCH. Except for XFS becoming an “add on product” for a hefty sum of money, redhat has ripped XFS out of Anaconda and hasn’t admitted any patches or work into Anaconda regarding XFS since.

    So much for “open” development. So much for “open” source.

    Yes I can patch Anaconda myself. Rebuild the SRPM. Should I churn out my own distribution too, because I can’t integrate my changes into RHEL? A fat lot of good does that do me.

    This whole “Linux has open development” spiel is a logical falacy. It’s no different than than anything else out there. If you’re lucky, work really hard and don’t go against one of the squires which control the access to the repository, you might get commit access someday. Maybe.

    “The Linux distributions I use also have a large number of users, so getting answers to support or configuration issues is typically pretty easy to do.”

    Yeah, and most of them are “me too! I have the same problem, does anyone know how to fix it?”

    Well, that’s really useful. There’s nothing like googling for a bug only to find that gazillion other people have the same problem, but nobody has any clue whatsoever how to fix it, or even what the root cause is.

    So at the end of the day, I’m stuck by myself, on that same Linux (and often am), and if I don’t roll up my sleeves and fix it, none of those clueless “me toos” will.

    That’s the same situation as on Solaris. So between Linux and Solaris, I choose an enterpriss, cooked OS – Solaris – any day of the week.

    “Me toos” can keep scratching their heads on trillion forums. So many users == good support is another fallacy.

    It doesn’t work that way in reality.

  2. This is what the “community” gets for subscribing to the CDDL license. Of course, we do have an opportunity to learn from our mistake here. The only way to avoid the same trap going forward is to use bits that are released under a free software license with strong copyleft protection.

  3. Wow, UX-admin. Harsh. The author is relating this story from firsthand experience, and has noticed a markedly different response with the opensolaris development community than with other open-source OS communities he’s experienced. Your comment makes me think you have an axe to grind.

  4. @UX-admin — I’ve actually had great success working with various Linux-related projects, though I typically target the upstream providers vs. the individual distributions. While what goes into RHEL is controlled by Redhat, at least I can get to the source RPMs to debug issues. Can’t say the same anymore for OpenSolaris. :(

  5. @UX-admin you are the reason Linux is whipping UNIX, keep it up!

    The fact you don’t seem to get a patch included != Red Hat’s problem and perhaps you need to understand how to communicate with upstream developers. Actually, no. Just keep it up :-P

  6. “So many users == good support is another fallacy.”

    Well, that explains Microsoft’s “retry, reboot, reinstall, upgrade” philosophy. :-/

  7. @UX-admin

    Actually I have written patches and had them accepted and used for RHEL and CentOS distributions. My atheros card in my laptop was not detected and unusable. I helped in that issue by testing development drivers when patches were supplied and once that was addressed I added also another patch for the LED lights on the wifi button and again tested the development drivers to prove that the patch worked.

    I If you supply well documented cases of the failure, supply a well written patch, and work with the devs (you have to watch the bug for updates and answer!) you will get your patch entered.

    Common things that cause bugs reports and/or patch submissions to get dismissed or overlooked:

    1. Lack of proof/documentation of the issue
    2. Low severity of issue when higher severity issues are pressing
    3. Failure to comply with the guidelines for writing/supplying a patch
    4. Attitude (sadly this is also true)

    From what I just saw in your previous post… I can guess at least one problem you had with the RHEL and/or CentOS dev team.

  8. @Tom — I haven’t used openindiana, so I can’t really talk intelligently about it. If I haven’t used it or broken/fixed it, you probably won’t see it on this blog. ;)

  9. @Steve — you are correct, though I’ve found a lot of sharp folks in the CentOS, RHEL and Fedora communities. Have yet to see/hear someone say “reboot the box and the machine will return to normal.” There are definitely cases were a reboot is the only thing you can do to address performance or reliability issues, but alas I haven’t seen that recommendation for other issues.

  10. @matty Thanks

    Also, one last point for UX-admin and others that say “” is better than Linux because “”…

    That is opinion and is subject to preference and the needs of that admin/user.

    I would say to you that as a matter of fact the majority of the admin populace does not share that sentiment.

    This is why wikipedia reports server usage in as follows:

    Linux 63.7%
    Microsoft Windows 33.7%
    All Unix 2.7%
    BSD 2.4%
    Solaris 0.1%
    Other Unix 0.2%
    Other <0.1%

    and supuercomputer os usage as follows:

    Linux 91.8%
    IBM AIX 3.4%
    Microsoft HPCS 2008 1.0%

    This does not support the idea that the Linux "community support" model and "open source" mentality does not work in real world usage. In fact it speaks to the contrary. The majority of datacenters show a way larger percentage of Linux in use than all other OS's combined. Note I said the majority and not all. There are those datacenters that have a high concentration of BSD or Unix as well as Windows. That is not the norm however.

    I pulled these numbers as is from:

  11. “I would say to you that as a matter of fact the majority of the admin populace does not share that sentiment.
    This is why wikipedia reports server usage in as follows:
    Linux 63.7%
    Microsoft Windows 33.7%
    All Unix 2.7%
    BSD 2.4%
    Solaris 0.1%
    Other Unix 0.2%
    Other <0.1%"

    What difference does it make what sentiment the majority shares? Is that something like Scott Adams' Dilbert "industry best practice"?

    At the end of the day, when work needs done, what sentiments are shared by whom is irrelevant.

    If the majority runs Linux because it's a trend, that doesn't validate Linux, it just speaks volumes about group think.

  12. And by the way, I wasn’t referring to myself when it comes integrating patches into RHEL. I often spend time digging through the redhat’s database because I push RHEL to the limit and hit a lot of bugs.

    As it turns out, so do other people, and when they include patches, they get excuses instead.

    I just have to shake my head in disbelief at a lot of those comments and answers coming from redhat’s engineers. Some of those are downright arrogant, some completely clueless.

    Meanwhile, the bugs get closed and no fix is forthcoming, although some of those fixes have patches from paying customers included.

    Simply unbelievable.

    I’m sorry, but the emperor is naked.

    Keep on glorifying “group think” and “sharing sentiments”.

    Good luck.

  13. @UX-Admin — we don’t run Linux because of any “trends,” we run it because it performs well and is reasonably well supported. The other big win is package and errata management. As a long time Solaris user that is something that was always lacking. The image package system is supposed to address some of these issues, but it’s not in a production release of Solaris so I have to use the existing tools for comparison.

  14. IPS is a disaster for Solaris.

    SVR4 packages in Solaris are extremely powerful, for example with class action script capability, but the learning curve is steep. They aren’t for everyone, but once one masters them, they are truly wonderful.

    IPS in contrast is severly broken by design and people who don’t really understand how Solaris customers use Solaris in production. One can’t create configuration packages because there is no preinstall/postinstall and no preremove/postremove, and there is no sufficent integration with SMF to be able to emulate those contexts.

    I use SVR4 packages in conjunction with JumpStart and Flash, and they work extremely well. Patching is a breeze because we use Flash(TM) images and clustering instead of patching live systems.

    But the learning curve is steep and it’s not for everyone. I note that a lot of Linux sysadmins aren’t cut out for that kind of engineering work.

    Configuration RPMs and zero interaction with a system are science fiction for most Linux sysadmins.

    As for redhat and patches, below are some examples, I could’ve dug up more. Without further ado:

    Like I wrote previously, good luck gentlemen. You’ll certainly need it.

  15. @UX-admin: Fine bugs you have there. Among the millions of entries you have picked two of the more famous ones. One got so much attention because some people thought they could play games by re-opening a bug which never has been one. You will find at least a hundred more of these. And by the way … Ulrich Drepper does not work for Red Hat anymore.

  16. @UX-admin:

    Come on do you really believe that companies or organizations (like mine) that have thousands of servers in multiple pops really run Linux on them because of trends of other system admins?

    If you do then you are sadly mistaken.

    My team has not logged on to most of these servers in 2+ years. We use puppet for updates and a central log server to keep individual disk usage down so there is no need, they just run and run well.

    A PXE server and kickstart combined with puppet and we can replace a production server in minutes if one does go bad due to hardware.

    There are unresolved bugs in every OS, yes even in Solaris. I have already shown that while you had a bad experience with redhats devs and getting patches included that this is not the norm. I myself have had patches included in releases.

    Just look at the closed bugs in bugzilla and read through them, you will find thousands that have been supplied by users and even more where a user only asked for a patch and then tested dev builds to prove it works.

    Maybe your point isn’t to troll or spread FUD but it certainly does not come across as such. You seem to have posted solely to bash Linux for issues that exist in every OS given the right bug.

    As to the numbers and why they matter, it does indeed show a “trend” but you mistake the trend as “group think” in stead of intelligent decision based simply on the fact that you personally disagree with their choice of OS.

    As if the “majority of the admin populace” are mindless sheep that cannot think for themselves enough to even pick an OS that is better for the job at hand. You choose to believe that the companies they are employed by and they themselves are powerless to choose unless other admins have given examples.

    Really UX-admin, just concede that others find Linux to be a better OS and that your views are obviously different than the majority of the world as clearly shown by factual research and not just by one admins opinion as your points are.

  17. ddreggors,
    So what does those percentage numbers say? Nothing.

    Sure, Linux runs on most Supercomputers, so what? What does that prove? That you can tailor Linux and throw out lot of code. It is not a custom Linux kernel. It is not common Linux, it is modified. That shows Linux kernel is easy to modify, but it proves nothing else.

    On rank no 5(?) on supercomputer list, BlueGene is very powerful. It uses 750MHz POWERPC cpus. Does that imply that 750MHz POWERPC cpu is among the best in the world? No, it means nothing. Linux running on those supercomputers mean nothing. Windows running on those supercomputers does not prove Windows is best and scales good. I heard one supercomputer with Windows preinstalled, they never booted Windows install, they only use the simple and fast modified Linux kernel to do the computations.

    Same with server percentage. Windows is very common, does that prove that Windows is very good? Hardly.

  18. I haven’t admined Solaris in years, but I still find it superior in some regards (introspection, for instance. There’s no DTrace/plockstat/intrstat/etc. for Linux). That said, it’s not worth putting up with Oracle for those few perks. Sad, really, since Solaris has so much potential, but is now in the hands of the worst vendor on Earth.

  19. @That Unix dude — I completely agree. Solaris is the bee’s knees when it comes to innovative technology! I’m super hopeful Oracle ports DTrace, ZFS, etc. over to Linux and licenses them under the GPLv2. That would be as good as it gets IMHO.

  20. Hi Matty,

    If you’d like to mail me your changes to dd, I’ll see if I can get someone to do a webrev for Illumos so that it gets integrated.

    Also if you have time, please do give OpenIndiana a go. Now that the Illumos integration work is mostly complete, we’ll be looking at producing a stable build with security patches sometime in the first half of this year.

    As project lead for OpenIndiana I can tell you that our top goal is being a community centered OS, with full open development, and we *do* follow up on developers expressing an interest in contributing to the project. I am not at all surprised that you lost faith in contributing under the old Sun model, but if you are looking for a project to contribute to, I can promise you helping OI/Illumos is highly rewarding.

    The future for the project is pretty bright. Big things start small and grow with time, especially when they’re driven forward by passionate believers who love what they’re doing, which we do.



Leave a Reply

Your email address will not be published. Required fields are marked *